t-test

The t-test is a test to statistically determine whether the means of two normally distributed groups are different. It is commonly used with small sample sizes when the variances of the two normal distributions is not known. The test also assumes that the distributions are independent.

To understand how the t-test works, consider Exhibit 34.19. In both case scenarios, the difference in the means is the same. However, the probability that green is different from blue is much higher for Case I, where the sample variability is much lower, and, consequently, there is much less overlap between their distributions.

So, in order to identify the difference between the means of two groups, the t-test must measure the difference in means relative to the variability within each group.

The test summarizes the data down to a single value, the t-value, a signal-to-noise ratio accounting for the difference in means relative to the variability in the data:

$$ t = \frac {signal}{noise} = \frac {difference \; in \; group \; means}{variability \; of \; groups} = \frac {x̄_1-x̄_2}{σ_{x̄_1-x̄_2}} $$ $$ σ_{x̄_1-x̄_2} = \sqrt {\frac {s_1^2}{n_1} + \frac {s_2^2}{n_2}} $$

Since s₁ and s₂, the standard deviation for the two groups, is not known these values are derived from the group samples.

The significance of the t-value is related to the samples’ degrees of freedom, which is equal to n₁ + n₂ – 2, if the standard deviation for the two groups is the same (s₁ = s₂). Otherwise, the formula is:

$$ degrees\;of\; freedom\; (df) = \frac {\left(\frac {s_1^2}{n_1} + \frac {s_2^2}{n_2}\right)^2} {\frac{(s_1^2/n_1)^2}{n_1-1}+\frac {(s_2^2/n_2)^2}{n_2-1}} $$

Given the t-value and the degrees of freedom (df), we can estimate the probability (p-value) that the means of two groups are the same, i.e., the hypothesis (x̄₁=x̄₂).

The p-value is obtained from the t-distribution table by looking for the t-value in the row for degrees of freedom = df.

Note: A p-value from t-value calculator is provided on this webpage. In Excel, the data analysis add-in provides an easy-to-use facility to conduct the t-test.

If the p-value is lower than α, the significance level (usually set at 5% for market research studies), then the hypothesis that the two groups have the same mean is rejected.