Google’s own documentation works the example, so we do not have to. Twelve media channels, six controls and eight time-effect parameters make 26 parameters. Two years of weekly data gives 104 data points. That is four data points per parameter, and the documentation’s verdict is direct: “This sample size is too low to estimate the model reliably.”
Marketing mix modeling is being sold hard right now, for an understandable reason: user-level tracking degraded, and a method that works on aggregate data looks like the answer. Sometimes it is. Whether it is for you is an arithmetic question, and it can be settled before anyone quotes you a price.
This page gives that arithmetic, what the tool authors themselves require, the failure modes their own technical papers document, and what to do instead if the numbers do not work.
What the method promises, and why interest returned
Understanding the appeal makes the limits easier to accept, because the appeal is genuine.
What it does. Relates aggregate outcomes over time, usually weekly sales or leads, to advertising spend by channel, plus controls for seasonality, price, distribution and anything else that moves the outcome. The coefficients are then read as each channel’s contribution.
Why it came back. It needs no cookies, no user identifiers and no cross-device tracking, so it survives everything that broke click-based attribution. That is a real and important property.
What it does that attribution cannot. It can, in principle, account for channels that produce no click at all: broadcast, print, sponsorship, and the share of digital effect that never generates a click, which the experimental literature suggests is most of it.
Where the difficulty starts. It is a regression on a small number of aggregate observations, and everything below follows from that single fact. Not from bad software, not from bad practitioners.
The arithmetic that decides before anything else
One ratio governs whether a model can be estimated at all, and both major tool authors state it.
The rule. You need several observations per parameter. Robyn’s guide puts it plainly: “The recommended ratio of data points is 1 independent variable : 10 observations”, restated in its source as “at least 10 observations to 1 independent variable”.
How many observations you have. Weekly data over two years is 104 observations. Over three years, 156. That is the entire dataset, and no amount of tooling increases it.
How many parameters you need. More than the channel count. Each channel typically needs three to four parameters to model delayed effect and diminishing returns, plus controls, plus seasonality terms.
What Google’s technical paper says about the resulting squeeze. “A typical MMM dataset, consisting of three years of national weekly data is only 156 data points. From this, the modeler is expected to produce a MMM often with 20 or more ad channels… the number of parameters in the model could exceed the number of data points available.”
And the standard it measures that against. “A rule-of-thumb for a minimum number of data points for a stable linear regression (putting aside whether causal effects are well-estimated) are 7-10 data points per parameter, of which typical MMMs fall short.”
Note the parenthesis. Seven to ten points per parameter is the bar for a stable regression, explicitly setting aside whether the causal effects are correctly estimated. That is the low bar, and typical models do not clear it.
What the tool authors themselves require
Both major open-source implementations publish their data requirements. They are worth reading before a vendor tells you what is possible.
Robyn. “In order to get robust results, an MMM will need a minimum of two years of historical weekly data. If monthly data is only available, then it is strongly recommended to collect more than two years (e.g. 4-5 years).”
Meridian. “As a general rule of thumb, historical data should be a minimum of two years’ worth of weekly data for geo-level models and three years’ of data for national-level models.”
Why the geographic distinction matters more than it sounds. A geographic model multiplies observations by the number of regions, which is how the data problem gets solved at scale. Meridian’s worked geographic example uses 105 regions. A company selling to a few hundred accounts across one country does not have 105 usable regions, so it is confined to the national model, which is the harder case.
The worked example, in full, because it is the whole article. “If you have 12 media channels, six controls, and eight knots, the total is 26 parameters… With two years of weekly data (104 data points), you have four data points per parameter. This sample size is too low to estimate the model reliably.”
And the rescue scenario the same page proposes. “You can combine the 12 media channels into three, lower your knots to two… If you also use three years’ worth of weekly data, you then have 156 data points to estimate 10 parameters. This is roughly 15 data points per parameter and now you might be able to glean some directional information from the MMM.”
Read what that rescue actually costs you. Three channels instead of twelve, three years of clean history, and the reward is “some directional information”. Not a return on investment per channel. Not a budget allocation. Directional information.
Why a model can fit perfectly and be wrong exactly where you use it
This is the failure mode that matters most, because it is invisible in every quality metric a vendor will show you.
The cause. Advertisers move their channels together. Budgets rise before a launch and fall in a quiet quarter, across every channel at once. As Google’s paper puts it: “Advertisers often allocate their spend across ad channels in a correlated way, which may make sense from the perspective of maximizing ad effectiveness.”
What that does to the estimates. “When fitting a linear regression model, highly correlated input variables can lead to coefficient estimates with high variance. This in turn, can lead to bad attribution of sales to the ad channel.”
The consequence stated in full, and it is the sentence to remember. “The modeler may find that many surfaces provide good predictive accuracy of out-of-sample sales, but perform poorly when the advertiser deviates from previous spend patterns. This is because the data contains little information about sales outcomes when one of the ad channels moves independently of the other.”
Read that against what you would use the model for. You commission a mix model in order to change the mix. The paper says the model is least reliable precisely when you deviate from previous spend patterns, which is the only reason to build one.
And the instability that follows. “The estimated relationship can change radically due to small changes in the data or the addition or subtraction of seemingly unrelated variables in the model.”
Why cross-validation does not rescue you here. The same paper notes in a footnote that “the scope for meaningful cross-validation of an MMM is typically limited due to small datasets and the time-series nature of the data which doesn’t allow for simple random sampling to produce a validation set.”
Five models, the same fit, answers 50% apart
The paper does not leave this abstract. It builds the demonstration.
The setup. Five plausible model specifications, fitted to the same data.
The fit. “Each model achieves an R2 value of either 0.98 or 0.99.” Four of the five also achieve an out-of-sample error of 6% to 8% in cross-validation.
The disagreement. “The models differ in their sales predictions by up to 50%… So despite fitting the data equally well, the models disagree on what sales are achievable under different ad spends.”
The conclusion the authors draw. “In this example, there are five plausible models that fit the data well, but that lead to different conclusions about the overall value of the ad channels and about how budget should be allocated.”
What this means for the deliverable you will be shown. An R-squared of 0.98 is not evidence that the allocation recommendation is right. It is evidence that the model describes the past well, which several mutually contradictory models also do.
The question to ask a vendor, and it is a fair one. How many alternative specifications did you fit, and how far apart were their recommendations? A vendor who fitted one model has not answered the question. A vendor who fitted several and shows you the spread is doing the job properly.
One more limit from a second paper by the same group. On the typical sample size of a couple of years of weekly national data, the model “may produce biased estimates”, with reported relative biases on response curves of roughly 18% to 40% at two years, against near zero on a very long series. And: “the optimal media mix based on the model has a large variance due to the variance of the parameter estimates.”
What happens when the model is checked against an experiment
There are few published side-by-side comparisons. The clearest one we could open is worth reading carefully, including its limits.
The design. Three advertisers, each of whom ran a randomised geographic experiment on their search advertising. The experimental result is treated as the source of truth and indexed to 1.0, and observational estimates on the same advertiser and period are compared against it.
The results. Case one: the naive regression estimate came out at 14.7 against an experimental 1.0. Case two: 8.4. Case three: 2.9. A demand adjustment based on category search volume narrowed the gap but left it large: 7.1, 7.3 and 1.4 respectively.
How the authors describe case one. “The naive estimate of ROAS is almost 15-fold larger than the experimental result. With the simple category-search-volume based demand adjustment, the gap shrinks but the estimate is still seven times as large.”
The mechanism. Advertisers spend more when demand is already rising, so the regression attributes the demand to the advertising. It is the same endogeneity that produced the +4,100% figure in the marketplace experiment discussed elsewhere on this site.
The limits, which matter and are usually dropped. These are three cases, on advertisers where “search ads are the dominant media channel in terms of spend”, using daily data on a single channel. This is not a full multi-channel weekly mix model, and three cases is three cases.
A correction worth making, because it circulates. The two large Facebook studies often cited as “MMM fails against experiments” do not concern mix modeling at all. They compare user-level observational methods, propensity matching and machine learning, against randomised trials. They are strong evidence that observational advertising measurement misses experimental truth, frequently by a factor of three. They are not evidence about mix models specifically, and citing them as such is a misattribution.
What a small company should do instead
None of this means measurement is hopeless. It means the instrument has to match the data you have.
Geographic holdout tests, one channel at a time. Turn a channel off in a set of regions, leave it on in matched regions, compare totals. This answers a narrower question than a mix model, requires far less data, and produces a causal estimate rather than a correlational one. It is what the mix model comparisons above use as their source of truth.
Time-based holdouts, if geography is not available. Less clean, because you cannot separate the pause from whatever else happened that month, but usable on a channel with a large enough share of spend to move the total.
The brand advertising pause. Two to three weeks off brand keywords, watching total enquiries rather than paid ones. Cheap, fast, reversible, and it settles a question worth real money.
The long-window ratio. Annual spend against annual new customers, ignoring attribution entirely. No confidence interval, no sophistication, and immune to every problem in this article.
When a mix model does become the right tool. Several years of clean weekly history, meaningful spend across several channels with genuinely independent variation, and preferably geographic granularity. That describes a company several sizes larger than most that are currently being sold one.
How to say no without dismissing the method. The method is sound. The question is whether your dataset can identify the parameters, and that question has an arithmetic answer that both tool authors publish. Ask a prospective vendor to work it with your actual number of channels and weeks, in the meeting.
Where to go next
You want the experimental evidence on whether advertising is incremental. Paid advertising vs organic.
Your counters disagree with each other. Why GA4 and Meta conversions don’t match.
You are choosing which metric governs the budget. ROAS vs MER vs CAC vs LTV.
Your deal volume is too low for stable averages. CAC at low volume.
You want the buying mechanics underneath all of this. Media buying explained.
You are splitting a budget across platforms. How to split budget between Google and Meta in B2B.
In short
- The arithmetic decides before anything else. You need roughly seven to ten observations per parameter, and weekly data over two years gives 104 observations in total.
- Google’s own worked example: 12 channels, six controls and eight knots make 26 parameters, giving four data points per parameter, which it calls “too low to estimate the model reliably”.
- The rescue scenario costs a lot and delivers little: three channels instead of twelve, three years of history, for “some directional information”.
- A technical paper from the same company notes that with 20 or more channels, “the number of parameters in the model could exceed the number of data points available”.
- Channels move together, so a model can predict the past well and “perform poorly when the advertiser deviates from previous spend patterns”, which is the only reason to build one.
- Five specifications, R-squared of 0.98 or 0.99 each, sales predictions up to 50% apart. Fit does not identify the right allocation.
- Checked against randomised experiments on three advertisers, regression estimates came out roughly 15, 8 and 3 times the experimental result.
- Use geographic holdouts instead at small scale. They answer a narrower question with far less data, and produce a causal estimate.
The method is sound. The question is whether your dataset can identify the parameters, and that has an arithmetic answer. Book a diagnostic, or see how we approach B2B paid acquisition.