It is easy to construct market models that visually impress with their goodness-of-fit, where the predicted and actual data match closely (e.g. Exhibit 27.9), the R2 value is high, and yet on analysis, the model is weak, incorrect or even nonsensical.
This may surprise you at first. Experienced market modellers, however, are only too familiar with such scenarios, and not easily fooled.
But, if you are not experienced, how do you assess the quality of a market model? The quality of a market model may appear deceptively good, so, as a user, you need to be aware of what could go wrong.
First and foremost, when it comes to developing market models, the knowledge of the market is as important as the knowledge of econometrics. The decision maker who uses the model and the econometrician, who builds it, need to work closely to create a practical solution based on market realities. It is very important that the market dynamics are clearly understood by the developer, that all of the variables that drive performance are included.
All too often in an era of commoditization of market modelling, data is shipped from the marketer to the market modeller, without the necessary information about the characteristics or nuances of the market. A modeller based overseas may have no knowledge of the Hungry Ghost festival, the exclusion of which may result in spuriously high discount elasticities for some FMCG brands in Singapore.
The exclusion from a model of any factor that significantly influences performance is likely to compromise the validity of the model. Unfortunately measures like R2 will still look good despite the omission. This is because marketing initiatives often occur concurrently, so the impact of the missing variables is attributed to other variables that comprise the model. The point to note is that all factors that significantly influence the dependent variable (sales), including external exogenous factors, should be included, irrespective of whether or not they are key to the objective of the research.
Greater levels of disaggregation provide for more robust and reliable market models. If 500 stores across 104 weeks (2 years) are modelled at store level, this yields 52,000 individual observations, i.e., this gives us very many degrees of freedom. Moreover, by modelling each store individually (Exhibit 27.10) the modeller is able to cut through noise, and isolate and measure the impact of price and promotional activities at store level.
When store level data is not available or accessible, modellers need to work with chain or channel level data, which introduces inaccuracies due to variations at the store level. Despite these imprecisions, chain/channel level data yields useful, fairly reliable models.
As regards the accuracy of the raw data from retailers, this is less of an issue now that relatively clean, weekly store level point-of-sale (POS) scan data is readily available in most markets.
Modelling works by correlating fluctuations in sales to those in the explanatory factors. If in the data there does not exist any variation in the movement of a factor, its potential effect is not calculable. (You cannot compute the discount elasticity of demand, if the product was historically not offered on discount).
Note also that when two or more factors always occur simultaneously and in similar proportions, it is not possible to untangle their individual influences.
Note: To find content on MarketingMind type the acronym ‘MM’ followed by your query into the search bar. For example, if you enter ‘mm consumer analytics’ into Chrome’s search bar, relevant pages from MarketingMind will appear in Google’s result pages.
Automated online solution for analysis of promotions.
Is marketing education fluffy too?
Marketing simulators impart much needed combat experiences, equipping practitioners with the skills to succeed in the consumer market battleground. They combine theory with practice, linking the classroom with the consumer marketplace.
The Plannogrammer is an experiential learning facility for category managers, trade marketers, and retailers in consumer markets. Ideally suited for hybrid learning programmes, Plannogrammer imparts hands-on training in the planning and evaluation of promotions and merchandising.
It supports a collection of simulation and analysis platforms such as Promotions and Space Planner for optimizing space and promotions, Plannogram for populating shelves and merchandising, a Due To Analysis dashboard that decomposes brand sales into the factors driving sales, and a Promotion Evaluator to evaluate the volume, value and profit impact of promotion plans.