Spillover & Saturation Design

What it is

Standard RCT designs assume the stable unit treatment value assumption (SUTVA): the potential outcome of each unit depends only on that unit’s own treatment, not on the treatment of others. This assumption fails whenever treatment of one unit affects outcomes of others — through information diffusion, market price effects, disease transmission, or social norms. When SUTVA fails, the standard ITT estimator mixes the direct effect of treatment with the spillover effects onto control units, and the estimated effect depends on the density of treatment in the study area.

A saturation design addresses this by randomising both the treatment rate within clusters (saturation) and individual treatment within clusters. This two-level randomisation enables separation of: (1) the direct effect of being treated; (2) the spillover effect of being untreated in a high-treatment environment; and (3) the total effect of increasing treatment density in a community.

When to use it

Saturation designs are appropriate when spillovers are theoretically plausible and policy-relevant. They are essential when: the intervention works through information diffusion (deworming, vaccination); when treated and control units interact in markets (cash transfers with market price effects); or when the policy question concerns optimal treatment saturation. Kremer and Miguel (2004) on deworming in Kenya is the canonical reference — they found that the benefits of deworming included massive positive spillovers that conventional designs would have missed.

The design is more expensive than a standard cluster RCT because it requires sufficient clusters to estimate effects at multiple saturation levels, and because individual-level randomisation within clusters must be maintained.

How it works

Design structure. Randomly assign clusters to different treatment saturation levels (e.g., 25%, 50%, 75% treatment rate, or a pure control cluster with 0%). Within each non-zero-saturation cluster, randomly assign the specified proportion of individuals to treatment.

Estimation strategy. Let T_i be the individual treatment indicator and S_j be the saturation level of cluster j. Four estimands can be defined:

• Direct effect: Y(T=1, S) − Y(T=0, S) at a given saturation level S.
• Spillover effect: Y(T=0, S_high) − Y(T=0, S_low) — the effect on untreated individuals of being in a high-saturation cluster.
• Total effect: E[Y(S_high)] − E[Y(S_low)] — the overall effect of increasing treatment density, combining direct and spillover effects.
• Dose-response: how the direct and spillover effects vary with saturation level.

Regression approach. Regress outcome on individual treatment status, cluster saturation level, and their interaction:

Y_i = α + β₁·T_i + β₂·S_j + β₃·(T_i × S_j) + ε_ij

β₁ = direct effect at zero saturation; β₂ = spillover effect for untreated individuals; β₃ = how the direct effect changes with saturation.

Key decisions

Number of saturation levels. Two levels (low and high saturation, plus pure controls) are sufficient to estimate linear dose-response. Three or more levels enable estimation of non-linear spillover effects. The tradeoff is that more levels require more clusters for adequate power at each level.

Pure control clusters. Including clusters with zero treatment (pure control) is important for identifying spillover effects on untreated individuals — by comparing untreated individuals in treated-saturation clusters to individuals in pure control clusters. Without pure control clusters, spillover effects are not separately identified.

Geographic containment. The design assumes that spillovers operate within clusters but not across them. This requires that clusters are geographically contained enough to prevent cross-cluster contamination. If two adjacent villages are in different treatment saturation conditions but share markets and social networks, the saturation variation is contaminated.

Caveats & common mistakes

Ignoring spillovers in standard designs. The most common error is not accounting for spillovers at all, rather than designing a study to estimate them. If spillovers are positive, the standard design underestimates the treatment effect (because control units also benefit). If spillovers are negative (competition, displacement), the design overestimates the treatment effect.

SUTVA violation undermining IV estimates. When encouragement designs are implemented in settings with spillovers, both the ITT and the first-stage estimate are contaminated by spillovers, and IV identification is complicated. Saturation designs and IV designs should be considered jointly when both take-up is endogenous and spillovers are expected.

Cluster-level randomisation not protecting against individual-level spillovers. If individuals within the same cluster share information, resources, or disease, then within-cluster comparisons (treated vs. untreated in the same cluster) are contaminated by within-cluster spillovers. The design estimates the within-cluster direct effect minus within-cluster spillovers — a parameter that can be close to zero even if the true direct effect is large.

Analysis Guide

import pandas as pd
import numpy as np
import statsmodels.formula.api as smf

# 1. Spillover effect on untreated individuals: restrict to treat_ind == 0 to
#    compare untreated people in treated-saturation clusters against those in
#    pure-control clusters; a positive coefficient on sat_level means untreated
#    people benefit from being surrounded by more treated neighbours
df_untreated = df[df["treat_ind"] == 0]
spill = smf.ols("outcome ~ sat_level", data=df_untreated).fit(
  cov_type="cluster", cov_kwds={"groups": df_untreated["cluster_id"]})
print(spill.summary().tables[1])

# 2. Direct effect interacted with saturation — treat_ind * sat_level tests whether
#    the personal benefit of being treated changes as more neighbours are treated;
#    a positive interaction means treatments reinforce each other through social
#    channels; cluster SEs handle within-cluster correlation from the cluster-level
#    saturation assignment
inter = smf.ols("outcome ~ treat_ind * sat_level", data=df).fit(
  cov_type="cluster", cov_kwds={"groups": df["cluster_id"]})
print(inter.summary().tables[1])

# 3. Cluster-level total effect: collapse to cluster means before regressing —
#    this pools direct and spillover effects into a single coefficient on sat_level
#    and answers the policy question whether raising community-level treatment
#    density improves average outcomes for everyone in the cluster
df_cluster = df.groupby(["cluster_id", "sat_level"]).agg(
  mean_outcome=("outcome", "mean")).reset_index()
total = smf.ols("mean_outcome ~ sat_level", data=df_cluster).fit(cov_type="HC1")
print(total.summary().tables[1])

library(estimatr)
library(dplyr)

# 1. Isolate spillover: restrict to untreated individuals so the estimate reflects
#    the effect of treatment density on people who never received the treatment —
#    this is the externality component that standard RCT designs cannot identify
df_untreated <- filter(df, treat_ind == 0)
lm_robust(outcome ~ sat_level, data = df_untreated, clusters = cluster_id)

# 2. Direct effect with saturation interaction: treat_ind * sat_level tests whether
#    the personal benefit of treatment is amplified or diminished in denser treatment
#    communities; cluster SEs account for the within-cluster correlation induced by
#    the cluster-level saturation assignment
lm_robust(outcome ~ treat_ind * sat_level, data = df, clusters = cluster_id)

# 3. Cluster-level total effect: collapse removes within-cluster individual variation
#    so the regression captures only the between-cluster variation in saturation;
#    with few clusters this approach is preferred over individual-level regressions
#    because it avoids overstating the effective sample size
df_cluster <- df %>%
group_by(cluster_id, sat_level) %>%
summarise(mean_outcome = mean(outcome, na.rm = TRUE), .groups = "drop")
lm_robust(mean_outcome ~ sat_level, data = df_cluster, se_type = "HC2")

Reading the output

• The sat_level coefficient in the untreated-only regression is the spillover effect: a positive coefficient means untreated individuals in higher-saturation clusters have better outcomes than those in pure control clusters.
• In the interaction regression, the treat_ind coefficient is the direct effect at zero saturation; the treat_ind × sat_level interaction tests whether the direct effect changes as saturation rises — a positive interaction means the direct effect is larger in denser treatment clusters.
• The cluster-level total effect regression on sat_level combines direct and spillover effects; a significant positive slope means higher treatment density benefits the cluster on average, regardless of individual treatment status.
• If treat_ind and sat_level coefficients have opposite signs (positive direct effect, negative spillover), the intervention may involve competition or resource displacement — total effect estimates will be smaller than direct effects alone.
• Standard errors must be clustered at the cluster level in all individual-level regressions; failing to cluster will produce confidence intervals that are too narrow.

References

Kremer, M., & Miguel, E. (2004). Worms: Identifying impacts on education and health in the presence of treatment externalities. Econometrica, 72(1), 159–217. https://doi.org/10.1111/j.1468-0262.2004.00481.x

Baird, S., Bohren, J. A., McIntosh, C., & Özler, B. (2018). Optimal design of experiments in the presence of interference. Review of Economics and Statistics, 100(5), 844–860. https://doi.org/10.1162/rest_a_00716

Halloran, M. E., & Struchiner, C. J. (1995). Causal inference in infectious diseases. Epidemiology, 6(2), 142–151. https://doi.org/10.1097/00001648-199503000-00010

Sobel, M. E. (2006). What do randomized studies of housing mobility demonstrate? Causal inference in the face of interference. Journal of the American Statistical Association, 101(476), 1398–1407. https://doi.org/10.1198/016214506000000636