It is easy to construct market models that visually impress with their goodness-of-fit, where the predicted and actual data match closely (e.g. Exhibit 20.9), the R2 value is high, and yet on analysis, the model is weak, incorrect or even nonsensical.
This may surprise you at first. Experienced market modellers, however, are only too familiar with such scenarios, and not easily fooled.
But, if you are not experienced, how do you assess the quality of a market model? The quality of a market model may appear deceptively good, so, as a user, you need to be aware of what could go wrong.
First and foremost, when it comes to developing market models, the knowledge of the market is as important as the knowledge of econometrics. The decision maker who uses the model and the econometrician, who builds it, need to work closely to create a practical solution based on market realities. It is very important that the market dynamics are clearly understood by the developer, that all of the variables that drive performance are included.
All too often in an era of commoditization of market modelling, data is shipped from the marketer to the market modeller, without the necessary information about the characteristics or nuances of the market. A modeller based overseas may have no knowledge of the Hungry Ghost festival, the exclusion of which may result in spuriously high discount elasticities for some FMCG brands in Singapore.
The exclusion from a model of any factor that significantly influences performance is likely to compromise the validity of the model. Unfortunately measures like R2 will still look good despite the omission. This is because marketing initiatives often occur concurrently, so the impact of the missing variables is attributed to other variables that comprise the model. The point to note is that all factors that significantly influence the dependent variable (sales), including external exogenous factors, should be included, irrespective of whether or not they are key to the objective of the research.
Greater levels of disaggregation provide for more robust and reliable market models. If 500 stores across 104 weeks (2 years) are modelled at store level, this yields 52,000 individual observations, i.e., this gives us very many degrees of freedom. Moreover, by modelling each store individually (Exhibit 20.10) the modeller is able to cut through noise, and isolate and measure the impact of price and promotional activities at store level.
When store level data is not available or accessible, modellers need to work with chain or channel level data, which introduces inaccuracies due to variations at the store level. Despite these imprecisions, chain/channel level data yields useful, fairly reliable models.
As regards the accuracy of the raw data from retailers, this is less of an issue now that relatively clean, weekly store level point-of-sale (POS) scan data is readily available in most markets.
Modelling works by correlating fluctuations in sales to those in the explanatory factors. If in the data there does not exist any variation in the movement of a factor, its potential effect is not calculable. (You cannot compute the discount elasticity of demand, if the product was historically not offered on discount).
Note also that when two or more factors always occur simultaneously and in similar proportions, it is not possible to untangle their individual influences.
Note: To find content on MarketingMind type the acronym ‘MM’ followed by your query into the search bar. For example, if you enter ‘mm consumer analytics’ into Chrome’s search bar, relevant pages from MarketingMind will appear in Google’s result pages.