What it is
The Anderson (2008) index — also called the inverse-covariance-weighted (ICW) index or the summary index — is a method for aggregating a family of related outcome variables into a single composite score. The key feature is its weighting scheme: variables that are highly correlated with other variables in the family receive lower weights, because they contribute less independent information. Variables with unique information — low correlation with the rest — receive higher weights.
The index was developed specifically for the problem of multiple hypothesis testing in programme evaluations: when an intervention is expected to affect several related outcomes (different dimensions of health, multiple aspects of education, various components of women’s empowerment), testing each outcome separately inflates the false-positive rate. Summarising the family into a single index reduces the number of tests while retaining power by combining information across all outcomes.
When to use it
The Anderson index is appropriate when: a programme evaluation has multiple pre-specified primary outcomes in the same conceptual domain; the outcomes are all expected to move in the same direction under the treatment; and the goal is a single test of whether the treatment affected the overall domain.
Anderson (2008) introduces the method in a re-analysis of a randomised evaluation of the JOBS programme, where multiple employment and psychological outcomes are combined into summary indices. Anderson indices are widely used in randomised evaluations for domains such as women’s empowerment, children’s health, household economic outcomes, and financial inclusion. Kling, Liebman and Katz (2007) use a conceptually similar approach in the Moving to Opportunity study.
The index is less appropriate when outcomes in a domain are expected to move in different directions (some increase, some decrease), when precise estimates of individual outcomes are the primary interest, or when the outcomes in a family measure conceptually distinct constructs that should be analysed separately.
How it works
The index is constructed in four steps:
Step 1 — Standardise each outcome. Each outcome variable is transformed to have mean 0 and standard deviation 1, standardised relative to the control group mean and standard deviation at the relevant wave. Using control group statistics preserves the counterfactual interpretation: the index measures deviations relative to what would have been observed without the treatment.
Step 2 — Sign-align outcomes. Ensure that higher values on all outcomes correspond to better outcomes (or all to worse outcomes). Outcomes coded in the opposite direction are multiplied by −1 before standardisation.
Step 3 — Compute inverse-covariance weights. The weight for each variable is the sum of the elements of the row corresponding to that variable in the inverse of the covariance matrix of the standardised outcomes. This gives lower weight to variables that are highly correlated with others (which are already represented by those other variables) and higher weight to variables carrying independent information.
Step 4 — Compute the index. The index is the weighted mean of the standardised variables, where weights are the inverse-covariance weights. The result is standardised to have mean 0 and standard deviation 1 for interpretability.
Key decisions
Defining the index family. The choice of which outcomes to include in a given index family should be specified in the pre-analysis plan before data collection. Post-hoc inclusion or exclusion of outcomes that do or do not show effects defeats the purpose of the index as a multiple-testing correction. The family should be defined by a conceptual criterion — “all outcomes measuring women’s economic empowerment” — not by observed correlations.
Handling missing values. The ICW index requires a complete covariance matrix. If some variables have substantial missing data, the covariance matrix may be poorly estimated. Imputing missing values before index construction (using control-group means or multiple imputation) or dropping variables with high missingness are the standard choices. Anderson (2008) recommends imputing missing values to the control mean when constructing the index to avoid bias from differential attrition.
Direction of outcomes. All outcomes must be signed in the same direction before computing the index. Reviewing the sign of each variable — and documenting the reasoning for sign reversals — is a necessary pre-analysis step. If some outcomes are expected to increase and others to decrease with treatment, the Anderson index is inappropriate and the outcomes should be analysed separately.
Comparison to equal-weight alternatives. The ICW index can be compared to a simple mean of standardised outcomes (the SUR/GLS index). When all outcomes are equally correlated, ICW and equal-weight indices give identical results. When correlations are heterogeneous, ICW down-weights clusters of similar outcomes. Reporting both and discussing differences builds transparency.
Caveats & common mistakes
The index is not theoretically motivated. Unlike factor analysis, which estimates how latent constructs explain observed item variation, the ICW index is a pure statistical construction. The weights reflect empirical correlations in the specific dataset, not any theory about the relative importance of outcomes. This means the index weights will differ across waves, sites, and samples, making the index itself not directly comparable across contexts.
Covariance matrix instability with small samples. The inverse of the covariance matrix can be unstable or undefined when the number of variables is close to the sample size, or when some variables are nearly perfectly correlated. A rule of thumb is to have at least 10 times as many observations as variables. For small variable sets (2–5 outcomes), the equal-weight index is more stable.
Interpretation of the index. A one-standard-deviation increase in the Anderson index is not inherently interpretable. Unlike a single outcome with a clear unit, the index aggregates across different scales. Reporting the index effect size alongside the individual component effects — to show which outcomes are driving the index movement — is standard practice and necessary for policy interpretation.
Pre-registration is essential. The multiple-testing motivation for the Anderson index only holds if the index family and construction rules are pre-specified. Post-hoc construction of an index that “passes” the significance threshold is no better than selective reporting of individual outcomes. Pre-registration of the outcome families and the index construction method is strongly advisable.
Analysis Guide
# outcome_1 to outcome_5: related outcome variables
import pandas as pd
import numpy as np
import statsmodels.formula.api as smf
outcome_cols = [f"outcome_{i}" for i in range(1, 6)]
# 1. Standardise each outcome to control-group mean and SD — preserves counterfactual interpretation; deviations are measured relative to the no-treatment distribution
control = df.loc[df["treatment"] == 0, outcome_cols]
df_std = (df[outcome_cols] - control.mean()) / control.std()
# 2. Sign-align so higher values uniformly indicate better outcomes — flipping the negative-direction outcome before computing the index; mixing directions produces a meaningless composite
df_std["outcome_3"] = -df_std["outcome_3"]
# 3. Build the inverse-covariance matrix — the core of ICW weighting; the inverse down-weights variables that share variance with other outcomes (already represented) and up-weights variables carrying independent information
cov_mat = df_std.cov()
inv_cov = np.linalg.inv(cov_mat.values) # if this fails, two outcomes are near-collinear; drop one
# 3b. Row-sum the inverse — each row sum is the raw ICW weight; large row sums correspond to outcomes with low correlation with the rest (high independent information)
weights = inv_cov.sum(axis=1)
weights_norm = weights / weights.sum() # normalise so weights sum to 1
# 4. Compute the weighted mean index — the ICW composite is the weighted sum of standardised outcomes using the normalised inverse-covariance weights
df["icw_index"] = df_std.values @ weights_norm
# 5. Standardise the index to mean 0, SD 1 — makes the regression coefficient interpretable as a SD-unit effect on the composite
df["icw_std"] = (df["icw_index"] - df["icw_index"].mean()) / df["icw_index"].std()
# 6. Regress the ICW index on treatment — the coefficient is the programme's effect on the composite, in SD units, with a single degree of freedom rather than one test per outcome
fit = smf.ols("icw_std ~ treatment + age + C(female) + log_hh_expenditure", data=df).fit(cov_type="HC1")
print(fit.summary())Reading the output
- The ICW weights should vary across outcomes; if they are nearly equal, the outcomes are roughly equicorrelated and the index is close to an equal-weight sum of standardised outcomes.
- Outcomes that receive very low weights are highly correlated with other outcomes in the family — they contribute little independent information. This is expected and not a problem, but it is useful to document which outcomes drive the index.
- The regression coefficient on treatment is the effect of the programme on the composite index, in standard deviation units. Report it alongside individual component effects to show which outcomes are driving the result.
- If
np.linalg.inv()/solve()fails (matrix not invertible), two or more outcomes are nearly perfectly collinear; drop one of the near-duplicate variables before proceeding.
References
Anderson, M. L. (2008). Multiple inference and gender differences in the effects of early intervention: A reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects. Journal of the American Statistical Association, 103(484), 1481–1495. https://doi.org/10.1198/016214508000000841
Kling, J. R., Liebman, J. B., & Katz, L. F. (2007). Experimental analysis of neighborhood effects. Econometrica, 75(1), 83–119. https://doi.org/10.1111/j.1468-0262.2007.00733.x
List, J. A., Shaikh, A. M., & Xu, Y. (2019). Multiple hypothesis testing in experimental economics. Experimental Economics, 22(4), 773–793. https://doi.org/10.1007/s10683-018-09597-5