By: Sara Owens, Senior Vice President of Analytics & Data Science
Myth: Your dataset tells the whole story.
Reality: Data without relevant, endemic knowledge (known as “priors” to Bayesian statisticians) is only half the story
Old-school (frequentist) statistics taught us that the right way to experiment, analyze data, and get sound results is to collect data, assume that the data represents all knowledge available, analyze results, achieve statistical significance, and only then get your answer.
The flaw in this approach is assuming that your data set represents all knowledge available. What if it doesn’t? Let’s think about a simple example. Let’s talk about Joe. He is an avid runner. He usually runs four to five miles, five days a week. Simple math tells us he runs about 20 – 25 miles per week. Joe logs his runs in Runkeeper. He has several weeks of runs logged.
Basic statistics tell us Joe ran an average 23 miles per week, with a standard deviation of 1.9 miles. And if we plot this data on a normal distribution, we can predict with a high degree of confidence how many miles Joe will run next week.
There is a 68% probability that Joe will run between 21 – 25 miles next week. There is a 95% probability that he will run between 19 – 27 miles.
But here is the catch. Joe is going on vacation next week and he is not packing his running shoes. He’ll run zero miles. And the frequentist forecast completely misses it.
Bayesian methods don’t make this mistake. Because they allow us to incorporate priors, existing knowledge outside of the dataset. If we know Joe is going on vacation, we can incorporate that as prior information, dramatically shifting the forecast.
This is the power of Bayesian thinking:
- It doesn’t treat data as a vacuum.
- It continuously updates predictions with both new data and known context.
- It delivers forecasts that are closer to reality, not just to theory.
That’s exactly how Agile Mix Modeling™ (AMM) works. It doesn’t just crunch campaign data in isolation. It incorporates context and prior knowledge about your brand, about the current state, and about how various media channels and formats perform for different audiences, product categories, and campaigns. This is why it’s so critical to have expert media practitioners at the wheel setting priors. What you get is continuously updated measurement and forecasting that can be acted on in real time. Because in today’s world, ignoring priors isn’t science — it’s dogma.
#DataVsDogma #MarketingMeasurement #Incrementality #MMM #AnalyticsLeadership #MediaPerformance #ArsXMachina #StayAgile #AXMAgency
