Regression Analysis — Process

Sequence of steps to be followed in the development of a regression model.

1. Sample: The sample size should be adequately large to ensure statistical power and generalizability of the regression.
2. Model design: Choose the dependent and plausible explanatory variables. Check assumptions:
   • Normality
   • Linearity
   • Homoscedasticity
   • Independence of the error terms.
   Often there is also the need to create additional variables to meet some of the below requirements:
   • Transformations, in case some assumptions are not met.
   • Dummy variables to cater for nonmetric variables.
   • Polynomials required if curvilinear relationships exist.
   • Interaction terms required if there are moderator effects.
3. Run the model. Variable Selection is usually a combination of methods such as confirmatory, stepwise estimation, forward regression and backward regression.
4. Examine Results: Output need to be thoroughly examined in the context of the following:
• Issues: Influential observations. Check for multicollinearity.
• Practical Significance: Is it theoretically sound? Are the signs and magnitude of coefficient meaningful?
• Statistical Significance:
  1. R_adj²: Adjusted coefficient of determination.
  2. F-ratio: Standard error of estimation.
  3. t-test: Statistical significance of regression coefficients.
  4. Residuals: Ensure that residuals meet required criteria.
  If model lacks practical/statistical significance or there are modelling issues, the model design must be revised (i.e., back to stage 3).
5. Validation: Cross-validate the regression to assess how well the model can be generalized.