Model Validity: How Good is the Fit?

The validity and reliability of market models must be assessed in terms of statistical accuracy as well as practicality. Is the model commonsensical? Does it meet the required accuracy standards?

• Adjusted R² (coefficient of determination): measures the proportion of the variation in the dependent variable (Sales) accounted for by the explanatory variables (Marketing Mix variables). It adjusts for the degrees of freedom associated with the sums of the squares.
• Estimated Standard Error: provides the average error for the model as a whole and can be used to calculate confidence intervals for forecasts and scenario simulations.
• Tests to assess the significance of the individual coefficients, i.e., the α and β values, which represent key parameters such as base and the elasticity of demand.
• Holdout tests: to assess the reliability of the model. These tests use the estimated model coefficients to predict the sales for a further 8 to 12 weeks. The sales prediction is then compared with the actual sales data of those weeks to assess the quality of the model.
• Tests to check for bias. Bias occurs when any of the variables that influence sales are omitted or when spurious variables are included. If a variable is overlooked, its effect is incorrectly allocated to the remaining variables that comprise the model. This results in the distortion of the importance of these variables.

Because it is easier to comprehend, greater emphasis is given to the R² measures. It is important however that due consideration is given to all the above tests, or else the conclusions derived from the model could well be misleading.