What it is
The intraclass correlation coefficient (ICC) measures how similar outcomes are among units within the same cluster — a village, a school, a health facility catchment area. An ICC of 0 means cluster membership provides no information about individual outcomes (knowing one person’s outcome tells you nothing about their neighbour’s). An ICC of 1 means everyone in the same cluster has identical outcomes (all the variation is between clusters, none within). In practice, ICCs for economic and social outcomes in development research settings are typically between 0.01 and 0.30.
The ICC matters for two related purposes: power calculations for cluster-randomised trials (a higher ICC requires more clusters for equivalent power) and regression adjustment when individual-level analysis ignores clustering, inflating the precision of standard errors. The design effect — by how much a clustered sample is less informative than a simple random sample of the same size — equals 1 + (m−1) × ICC, where m is the average cluster size.
When to use it
ICC estimation is appropriate when: designing a cluster-randomised evaluation and needing to determine the number of clusters required; adjusting standard errors in analysis of clustered data; or diagnosing the degree of within-cluster homogeneity in a completed survey as a measure of data quality (very high ICCs in some outcomes can indicate enumerator clustering effects or contamination).
Donner and Klar (2000) provide the theoretical foundations for cluster randomisation, including ICC estimation and its implications for sample size. Bloom, Richburg-Hayes and Black (2007) estimate ICCs for school outcomes in the US and show how they drive sample size requirements in education evaluations. In India, Muralidharan and Sundararaman (2011) report ICCs from their teacher incentive experiment in Andhra Pradesh — ICCs of 0.2–0.4 for test scores, implying substantial design effects.
How it works
The ICC is estimated from a one-way random effects ANOVA model:
y_ij = μ + u_j + ε_ij
where y_ij is the outcome for individual i in cluster j, u_j is the cluster-level random effect with variance σ²_b (between-cluster variance), and ε_ij is the individual-level error with variance σ²_w (within-cluster variance).
The ICC is:
ρ = σ²_b / (σ²_b + σ²_w)
This is estimated by fitting a random effects model and extracting the variance components. The design effect (DEFF) for a cluster of size m is:
DEFF = 1 + (m − 1) × ρ
The effective sample size of a clustered sample of N individuals in clusters of size m is N / DEFF.
For binary outcomes, the ICC is typically estimated using a linear probability model or a logistic mixed model. For continuous outcomes, the linear variance components model is standard.
Confidence intervals. The ICC estimate is a statistic with sampling variability. Confidence intervals — typically computed using the F-distribution for the one-way ANOVA or using bootstrap methods for more complex designs — should be reported alongside point estimates, particularly when using the ICC for power calculations.
Key decisions
Level of clustering. If the study has multiple levels of clustering (students within classrooms within schools), the ICC should be estimated at each level relevant to the design. Treatment assignment at the school level produces a design effect based on the school-level ICC for the full school variance; classroom-level ICC and school-level ICC are separate parameters.
Outcome-specific ICCs. The ICC varies by outcome — test scores, income, and health outcomes have different ICCs, and the ICC for the same construct can differ substantially across settings. Using a single ICC from a different study or context for power calculations is risky; if data from the same setting or population are available, estimating the ICC from those data is strongly preferred.
Pre-specified ICC in power calculations. Power calculations for cluster RCTs should document the assumed ICC, its source, and a sensitivity analysis showing how sample requirements change across a range of plausible ICC values. An ICC of 0.05 vs. 0.15 can require nearly twice as many clusters for equivalent power in some designs.
Caveats & common mistakes
High ICC from enumerator clustering. If enumerators are systematically assigned to geographic clusters (each enumerator covers one or a few villages), the ICC may confound genuine outcome clustering with enumerator effects. An unusually high ICC — especially one that differs markedly across enumerators — should prompt investigation of whether enumerator behaviour is contributing to within-cluster homogeneity.
Ignoring ICC in analysis. Treating clustered observations as independent underestimates standard errors by a factor of approximately sqrt(DEFF). In studies with moderate clustering (ICC = 0.10, cluster size = 20), the DEFF ≈ 2.9, meaning standard errors are underestimated by about 70% if clustering is ignored. Cluster-robust standard errors or multilevel models must be used.
Assuming ICC is constant across subgroups. ICCs may differ between treatment and control groups, between genders, or across regions. Reporting average ICCs without investigating heterogeneity can mask important variation. For heterogeneous treatment effects analysis, checking whether the ICC differs by subgroup is informative.
Analysis Guide
import pandas as pd
import numpy as np
import statsmodels.api as sm
from statsmodels.regression.mixed_linear_model import MixedLM
# 1. Fit null random effects model — no covariates, so all variance is
# attributable to cluster or individual, not predictors; the variance
# components from this model decompose total variance into between-cluster
# and within-cluster components needed for the ICC
model = MixedLM.from_formula('outcome ~ 1', groups='cluster_id', data=df).fit()
var_between = float(model.cov_re.iloc[0, 0])
var_within = float(model.scale)
# 2. Compute ICC from variance components — proportion of total variance
# explained by cluster membership; values close to 0 mean clustering
# is negligible, values close to 1 mean within-cluster homogeneity
icc = var_between / (var_between + var_within)
print('ICC:', round(icc, 4))
# 3. Design effect — quantifies how much less informative a clustered sample
# is than a simple random sample of the same size; effective N is the
# actual N divided by DEFF and is what determines statistical power
m = df.groupby('cluster_id').size().mean()
deff = 1 + (m - 1) * icc
print('DEFF:', round(deff, 2), '| Effective N:', round(len(df) / deff))
# For binary outcomes: use BinomialBayesMixedGLM or pingouin.intraclass_corr;
# ICC from logistic mixed models is on the latent scale, not the probability scaleReading the output
- ICC < 0.05: low clustering — standard power calculations hold
- ICC 0.05–0.15: moderate — verify the power calculation assumed this range; if not, recalculate required clusters
- ICC > 0.15: high — required cluster count may be substantially larger than planned; recheck sample size
- DEFF > 2: effective N is less than half your actual N — a common and costly oversight
- ICC > 0.30 on individual-level outcomes (attitudes, self-reports): probable enumerator effects — check whether enumerators were systematically assigned to clusters
- Wide CI spanning your assumed ICC: insufficient data to pin down clustering; treat as a prior, not a confirmed value
References
Bloom, H. S., Richburg-Hayes, L., & Black, A. R. (2007). Using covariates to improve precision for studies that randomize schools to evaluate educational interventions. Educational Evaluation and Policy Analysis, 29(1), 30–59. https://doi.org/10.3102/0162373707299550
Donner, A., & Klar, N. (2000). Design and Analysis of Cluster Randomization Trials in Health Research. Arnold.
Muralidharan, K., & Sundararaman, V. (2011). Teacher performance pay: Experimental evidence from India. Journal of Political Economy, 119(1), 39–77. https://doi.org/10.1086/659655
Murray, D. M. (1998). Design and Analysis of Group-Randomized Trials. Oxford University Press.