Sample Size — Comparative Studies

Comparative studies aim to compare two or more groups to determine if there are statistically significant differences between them with respect to certain parameters. These studies can involve various comparisons, such as control group versus test group (e.g., controlled store tests), users versus non-users of a brand, or before-and-after analyses.

In a comparative study, the null hypothesis (H₀) assumes that there are no differences between the groups being compared. The alternative or research hypothesis (H_A) suggests that there is a difference, indicating a rejection of the status quo.

Regarding sample size determination in the context of comparative studies, we need to consider the desired level of accuracy and the non-null difference that the analyst aims to detect. By estimating the required sample size, researchers can ensure that they have sufficient data to draw meaningful conclusions.

Comparison of Two Means (Independent)

When comparing two means from independent groups, the sample size estimate is influenced by factors such as the desired significance level (α), the desired power (1 – β), the standard deviation of the population, and the smallest effect size or non-null difference that the analyst aims to detect.

The formula for estimating the sample size when comparing two groups with equal sample sizes is as follows: $$ n = \frac {2 (z_α + z_β)^2 }{(δ/σ)^2} $$

δ = |μ₀ − μ₁| is the detectable difference in the mean.

σ: population variance.

z_α: standardized value associated with α, the level of significance, the type I error.

z_β: standardized value associated with β, the type II error.

For α = .05, z_α = 1.96; for β = .20, z_β =0.84.

$$ M (multiplier) = 2 (z_α+z_β)^2=2(1.96 +0.84)^2=15.68 ≈16 $$ $$ n=\frac{M}{∆^2},\;\;\;where\; ∆ =\frac{|μ_0- μ_1|}{σ}=δ/σ $$

Δ is the standardized difference between means, or the effect size (ES). It is measured in units of the standard deviation and represents the magnitude of difference in means.

The multiplier, M, varies with power as depicted in Exhibit 35.7.

Power (1 – β)	One sample	Two sample
0.5	4	8
0.8	8	16
0.9	11	21
0.95	13	26
0.975	16	31

Exhibit 35.7 Multiplier for α =.05, for different power settings.

The detectable difference, |μ₀ − μ₁|= σ√(M/n)

For the conventional setting of power = 0.8, M = 16:

$$ n=\frac{16}{∆^2}, |μ_0- μ_1|=4σ/\sqrt{n} $$

This is the sample size for each of the two groups. If there is only one sample, the multiplier is 8, and n = 8/Δ².

Consider, for example, a study on the weight of fresh recruits in the army. Suppose we are interested in detecting a change in weight compared to a previous batch of recruits, and we consider a difference of 400 grams or more to be of practical relevance. If the standard deviation of the weight distribution is 3.2 kg, we can proceed with the following calculations: $$ ∆=\frac{400}{3200}=0.125; \;n=\frac{8}{∆^2} =512 $$

We need a sample of 512 recruits to detect a difference of 400gm in weight.

Comparison of Two Means (Dependent)

In cases where observations are paired, such as before and after measurements, the sample size formula for comparing two means is as follows: $$ n = \frac{(z_α + z_β)^2}{(δ/σ)^2} $$

Note that the multiplier (i.e., the numerator), M, has been halved compared to the formula for independent samples. Additionally, the standard deviation (σ) in this formula refers to the standard deviation of the differences within pairs, which is often not known in advance and needs to be estimated from the data.

Comparison of Two Proportions

When comparing two proportions, p₀ and p₁, the sample size required can be estimated using the following formula: $$ n = \frac{M p(1 - p)}{(p_0 - p_1)^2} $$ $$ \text{Here M is the multiplier and p is the average proportion, } p = (p_0+ p_1)/2 $$

To obtain a conservative estimate, it is common to assume p = 0.5, which yields an upper limit on the required sample size. If the desired power is set at the conventional level of 0.8 and the multiplier M is 16, then the required sample size, n, is: $$n = \frac {M}{4×(p_0 - p_1)^2} $$ $$n = \frac {4}{(p_0 - p_1)^2} $$

For small proportions, p<.05, use: n = 4/(√p₀ - √p₁)²