Red Flags in Analytics

In this article we look at why no measurement is better than bad measurement, and how to distinguish good and bad measurement.

Marketing measurement is no longer a “nice to have”. It is a must for modern businesses to allocate budget, justify spend, and grow efficiently. Yet despite its importance, it is one of the most misunderstood and misused areas in marketing.

The uncomfortable truth: bad measurement can be worse than no measurement at all.

Why marketing measurement matters

At its core, marketing measurement exists to answer a simple question: what is driving results, and where should we invest next to grow further?

Done well, it enables:

Smarter budget allocation
Clear understanding of ROI by channel/campaign/platform
Confidence in scaling spend
Alignment between marketing and finance

Done poorly, it does the opposite.

When measurement destroys value

There is a tendency to assume that any form of measurement is better than none. That is not true.

If your model is flawed:

You shift budget away from what works
You double down on ineffective channels
You misread causality as correlation

In practice, this means negative ROI on the measurement itself. You are paying for the analysis and you are also losing money through bad decisions driven by it.

Historically, this risk was partially mitigated by cost. Traditional Marketing Mix Modelling (MMM) projects often came with a ~£50k price tag, which naturally limited adoption and encouraged scrutiny.

Today, the landscape has changed.

The cost shift: from £50k to £2k/month

Modern tools and open-source frameworks have significantly reduced the cost of running MMM and similar approaches.

This is, in many ways, a positive development:

More businesses can access measurement
Iteration is faster
Models can be refreshed regularly

However, it introduces a new risk.

Paying £2k per month for a poor model is arguably worse than paying £50k once for a rigorous one. The lower barrier to entry means it is easier than ever to deploy something that looks like measurement but lacks robustness.

At that point, the real question becomes:

How do you know if your measurement is actually good?

Asking the right questions

Most vendors and tools will present polished dashboards and confident recommendations. The real signal lies beneath that surface.

Here are the questions that matter.

1. What data is being used?

The foundation of any model is the data.

You should understand:

How many years of history are included?
- An analysis built on 2 years or less of data might not have enough information to distinguish marketing’s impact from other factors (e.g. seasonality, competitors, market conditions).
Are key variables highly collinear?
- If your campaigns go live on the same days, a model will not be able to confidently attribute merit to any of these (collinear) activities.

At Linea, we consider 3 years of data to be a minimum for most models.

2. What factors are included?

Marketing does not operate in isolation.

A credible model should incorporate:

Media activity (spend, impressions, etc.)
- A model will need media variables to calculate ROIs but a complete analysis should also review changes in cost per clicks/impressions to monitor changes in each campaign’s performance.
Non-media factors (seasonality, pricing, promotions, distribution, product changes)
- In order to isolate the incremental impact of marketing, other factors must also be measured. For example: “Am I seeing an increase in revenue relative to last month because of my marketing or because of a seasonal trend?”

If the model only looks at media, it is almost certainly over-attributing impact to it.

3. How granular is the analysis?

Granularity determines how actionable the results are.

Key considerations:

Is the breakdown at channel level only, or deeper (e.g. campaign, platform, or format level)?
- If the data provided is at a channel level, the analysis will only be able to deliver channel-level ROIs.
Is there enough data to support that granularity?
- A common trade-off in statistical models is number of variables (i.e. 3 variables, one for each channel, or 10 variables one for each creative) vs number of observations (i.e. modelling period, datapoints per variable). If a model with 150 weeks of data has 50 variables sophisticated techniques should be implemented to ensure model stability and prevent overfitting.

There is a balance here. Too coarse, and insights are vague. Too granular, and the model becomes unstable.

At Linea we use a hierarchical Bayesian approach to ensure reliable models with maximum granularity.

4. Are synergies measured?

Channels rarely operate independently.

For example:

TV may amplify search
An increase in product range might increase the effectiveness of Social
A worst economic condition might decrease marketing performance

Ask:

Are interactions between channels modelled?
What about the synergies of media and non-media variables?
If so, how are they quantified?
Are they assumed, or statistically estimated?

Ignoring synergies can lead to systematically undervaluing upper-funnel activity and misrepresenting performance over time.

Finding the usual approaches lacking when it comes to measuring synergies, we at Linea developed our own Bayesian framework and model specification to measure and quantify these interactions. We then feed these insights into ROIs as well as forward looking optimisations to plan for any scenario.

5. How is uncertainty handled?

All models are approximations. The question is whether uncertainty is acknowledged.

Look for:

Confidence intervals
Prior and posterior distributions (for Bayesian models)
Variance and stability of key parameters

If outputs are presented as precise and deterministic, that is a red flag. Stability tests and confidence measures are essential for a complete analysis. Otherwise, a model might show completely different results then next time you run it using the most recent data.

6. Can you inspect the model?

Transparency matters.

You should be able to access:

Coefficients
Priors (if Bayesian)
Functional forms (e.g. saturation curves, adstock)

If the model is a black box, you are being asked to trust results you cannot validate.

Model specification (i.e. coefficients and transformations) are often left out of deliverables. At Linea we provide a complete view on all inputs and outputs of our models.

7. How far do recommendations extrapolate?

Optimisation often pushes beyond historical data.

Ask:

Are recommendations within observed ranges?
If not, how far beyond?
What assumptions justify that extrapolation?

Models are strongest within the data they have seen. The further you move away from that, the more speculative the outputs become.

Our Scenario Tool, for example, has built-in constraints that will prevent an optimisation from recommending more than 2x the spend the model has seen, since going beyond that would be uncharted territory.

Final thought

Marketing measurement is not just about having a model. It is about having a model you can trust.

The shift towards cheaper, more accessible solutions is a positive one, but it places more responsibility on the user to ask the right questions.

Because in this space, the real risk is false confidence in the wrong answer.