The validity and
reliability of market models must be assessed in terms of statistical accuracy
as well as practicality. Is the model commonsensical? Does it meet the required
accuracy standards?
- Adjusted R2 (coefficient of determination):
measures the proportion of the variation in the dependent variable (Sales)
accounted for by the explanatory variables (Marketing Mix variables). It
adjusts for the degrees of freedom associated with the sums of the squares.
- Estimated Standard Error: provides the average error for the
model as a whole and can be used to calculate confidence intervals for
forecasts and scenario simulations.
- Tests to assess the significance of the individual coefficients,
i.e., the α and β values, which represent key parameters such as
base and the elasticity of demand.
- Holdout tests: to assess the reliability of the model. These tests
use the estimated model coefficients to predict the sales for a further 8 to 12
weeks. The sales prediction is then compared with the actual sales data of
those weeks to assess the quality of the model.
- Tests to check for bias. Bias occurs when any of the variables that
influence sales are omitted or when spurious variables are included. If a
variable is overlooked, its effect is incorrectly allocated to the remaining
variables that comprise the model. This results in the distortion of the
importance of these variables.
Because it is easier to comprehend, greater emphasis is
given to the R2 measures. It is important however that due
consideration is given to all the above tests, or else the conclusions derived
from the model could well be misleading.
