Eric Seufert’s Post

View profile for Eric Seufert, graphic

Independent analyst & investor. Proprietor of Mobile Dev Memo.

Why media mix modeling fails While MMM can be a valuable tool in a suite of measurement solutions, it can't be the exclusive tool used for measuring the effectiveness of ad spend in promoting digital products. I've seen MMM be rejected by host organizations like an unsuccessful organ transplant. Integrating an MMM into a marketing workflow requires a tremendous amount of effort: in adapting that workflow to the limitations of MMM, in properly setting expectations across the broader organization, and in supplementing the MMM with other tools to service the use cases for which it is not suitable.

  • No alternative text description for this image
Eric Seufert

Independent analyst & investor. Proprietor of Mobile Dev Memo.

2w
Dale W. Harrison

Commercial Strategy & Marketing Effectiveness

2w

The tool isn't what's failing; it's the organization that's failing, generally because it's not ready for that level of sophistication. 90% of what makes a tool useful is the organizational acceptance and integration. This is why MMM is probably best reserved for companies with the sophistication to understand and use the outputs (and know what MMM is and isn't going to deliver). Most companies should probably focus on simpler things like using their existing attribution data to create predictive models for lead scoring (as opposed to the current approach of just making up numbers) and running randomized controlled experiments. These are easier to explain, easier to understand what they mean, and easier to get adoption for. Only after the organization becomes comfortable with the easy stuff will it be ready for complex and nuanced tools like MMM.

Dr Grace Kite

magic numbers & magic works

2w

This is why MMM has to be packaged with human beings that can support other humans with change.

Rhys Cater

Group Chief Solutions Officer at Precis

2w

Businesses are hooked on deterministic, fast, and accurate metrics to quantify return on marketing spend. The problem, like you say in your post, is that they never worked in the ways that marketers hoped. Even in the data utopia days of the mid 2010s, all attribution models were plagued with gaps, caveats, and biases. I knew very few companies who truly succeeded in implementing a better model than last click in a consistent way across their entire business (from tactical optimisation right up to boardroom). The result is that too many companies still converge to the method that presents the least friction: basic channel and last click data. The hurdle to implementing tools like MMM, or more holistic methods to evaluate ROI, is that they require the business to come together and make a decision and accept trade-offs. Even if the model they rely on today is flawed, they continue to rely on it because by doing so they avoid the need to do the hard work of creating organisational alignment and putting deep thought into better methods to evaluate performance.

Bradley Keefer

CRO | SaaS Revenue Leader | Start-up Executive

2w

Too often, people think in all-or-nothing terms. MMM isn’t meant to be the sole tool in a marketer’s arsenal but the central system that orchestrates and governs the entire marketing ecosystem. 1. MMM as the Governor: In a distributed system where various platforms operate independently, a central MMM is essential for aggregating data, analyzing cross-channel performance, and providing a unified view of marketing effectiveness. 2. Setting Budgets: We’ve seen the best results when MMM sets the budgets across channels, and then each channel is further optimized within the platform’s campaign measurement. I wouldn’t trust Facebook’s ROI as $10, but internally, it’s consistent. 3. Data Collection: MMMs using Bayesian methods can work with very little data and still produce predictable models. If you can’t say what you spent in the last 12 months by channel, you’ve got bigger problems than measurement. 4. Organizational Change: Integrating MMM can be challenging, but it’s a feature, not a bug. It gives marketing a seat at the table and ensures alignment across departments, making it easier to plug into the business’s financial metrics.

Brian Truman

Ad-Tech Consultant and Startup Advisor | Business Development, Revenue Growth, Strategic Planning, Partnerships, Advertising Technology, Mobile Games | Ex-Scopely

2w

Specifically for free to play mobile games, the business model is a statistical arbitrage. Successful businesses have grown by exploiting price inefficiencies - in a competition to buy users for a cost that is less than what they are worth. The tighter the competition and smaller gap in price inefficieny is causing havoc in these business models and systems. Can MMM and other tools do a better job at finding and exploiting price inefficiencies or does the business need to change their financial model to adapt to MMM and other new tools before they can be successful?

I find it hard to understand why anyone jumps to mix media modelling failing. Even if this is the case, it’s surely a learning to change how doing it or something else. As others mention, this is often the issue. Too many take mixed media modelling or other approaches as if they are magic bullets. We are doing the same with AI. Strip it back and it’s hypothesis testing at scale, but if you don’t know your objectives, how you measure success and are unwilling to isolate variables/data ( often with some controls or tests) it can be an excercise that confuses rather than informs. Other key mistakes include. Only thinking media affects the model Vs other business or real world influences(even the weather). The measurement data used is itself wrong or built on assumptions nobody questioned. Hiring a magic box provider, Vs working with people who’ve successfully worked through projects before who can explain, push back when needed and call BS. BS is a common occurrence as those who commissioned MMM look to find and show the data that shows it is working- even when it’s not true.

Like
Reply
Greg Dolan

Co-Founder and CEO | Inc. 5000 software-as-a-service company

2w

In the world of marketing, there’s often a quest for a "one stop shop" solution that can handle all measurement needs. However, the reality is that no single tool can do it all. Marketing Mix Modeling (MMM) plays a crucial role in understanding the holistic impact of marketing across channels, informing budget allocations, and predicting financial performance. Emerging Bayesian techniques, driven by software like Keen Decision Systems' solution, allow for the incorporation of information from other analytics solutions such as incrementality testing and Multi-Touch Attribution (MTA). This integration with the broader data and technology ecosystem helps reduce the ongoing challenges of updating and leveraging marketing mix results for decision-making. It’s time for marketers to take a strategic and intentional approach to their data and marketing technology stack. By identifying the best-in-class solution for each specific need, marketers can optimize their workflow and achieve superior outcomes.

John James

Executive Advisor Commercial Strategy - Champane aficionado, Freelance CCO/CMO/CRO/CGO

2w

Media Mix Modelling often fails for the same reasons all measurement efforts which seek accuracy fail. Politics

See more comments

To view or add a comment, sign in

Explore topics