List Randomization (Item Count Technique)

A method for estimating the population prevalence of a sensitive attribute without requiring individual disclosure — respondents report a count, not which items are true.

What it is

The item count technique (ICT), commonly called list randomization, estimates the population prevalence of a sensitive attribute — a behavior, attitude, or characteristic that respondents would not report accurately if asked directly (Blair & Imai, 2012). Respondents are randomly assigned to one of two conditions. The control group receives a short list of non-sensitive items and reports only the count of true statements. The treatment group receives the same list with the sensitive item added, and reports a single count of true statements in the same way. The prevalence estimate is the difference in average counts between the two groups.

The method works because individual counts are not uniquely identifying — a treatment respondent who reports a count of 3 could be holding any combination of three true items from the list. This ambiguity is the source of privacy, and privacy is what makes honest reporting plausible.

When to use it

The ICT is appropriate when a sensitive attribute carries enough social stigma, legal risk, or contextual threat that direct questioning would produce meaningfully biased responses, and when the research question requires only a population-level prevalence estimate rather than individual-level data.

Typical uses span fields: vote buying and electoral coercion (Gonzalez-Ocantos et al., 2012), bribery and informal payments, intimate-partner violence, child labour, undeclared income, drug or alcohol use, harassment and misconduct at work, attitudes that respondents are afraid to share publicly, and compliance with rules respondents may have broken. Gonzalez-Ocantos et al. (2012) is a clean illustration: their ICT estimates of vote buying in Nicaragua are sharply higher than direct self-reports, quantifying the gap that social desirability bias produces.

The ICT is not appropriate when individual-level data are needed — the method produces an aggregate prevalence estimate and cannot identify which respondents hold the sensitive attribute. It also performs poorly when the true prevalence is very low (below roughly 5%) or very high (above 80%), where the estimates become unstable. And because the estimator is less statistically precise than a direct question (see Key Decisions), it is a poor fit when sample sizes are tightly constrained.

How it works

Respondents are randomly assigned to treatment or control, typically in equal proportions. The control group hears: “I will read you a list of statements. Please tell me how many are true for you — just the number, not which ones.” The treatment group receives the same instruction but with the sensitive item appended to the list.

The prevalence estimator is the difference in mean counts:

τ̂ = ȳ_T − ȳ_C

where ȳ_T and ȳ_C are the sample mean counts in the treatment and control arms. This estimator is unbiased for the population prevalence of the sensitive attribute under three assumptions:

Random assignment. List version is assigned independently of any respondent characteristic — guaranteed by design in an experiment.
No design effect. Adding the sensitive item does not change how respondents answer the non-sensitive items. If the sensitive topic primes respondents, triggers acquiescence shifts, or reframes how they interpret the rest of the list, this assumption is violated.
No liars. Respondents who hold the sensitive attribute report it truthfully through their count in the treatment arm — i.e., the privacy of the list format is sufficient to elicit honest answers.

Blair, Coppock and Moor (2020) review three decades of empirical evidence and find that list experiments reliably reduce bias relative to direct questioning on items with moderate stigma but can still under-report under heavy stigma (the no-liars assumption fails). The Caveats section discusses these failure modes in more detail.

Because the estimate is derived from two independent group means rather than a single proportion, its statistical precision is considerably lower than that of a direct question — for a four-item control list and prevalence around 20%, the ICT requires roughly four to six times the sample size of a direct binary question to achieve equivalent precision (Blair & Imai, 2012). This is the most important planning implication of the method, and it is routinely underestimated when designing studies.

Beyond the simple difference-in-means estimator, Imai (2011) and Blair and Imai (2012) develop a maximum-likelihood estimator that is more efficient and supports clean covariate adjustment in the prevalence (rather than count) equation. The ML estimator is the default when using the list R package (ictreg) or the user-written kict command in Stata. The two estimators answer the same question under the same assumptions and converge in large samples; in small samples they can produce visibly different point estimates, and reports should be explicit about which one is used.

Key decisions

List length. The norm of four control items traces to Droitcour et al. (1991) and has held up in subsequent work. Shorter lists provide less cover: with three items, a treatment respondent who reports 0 or 4 has effectively disclosed whether the sensitive item is true or false. Longer lists reduce efficiency because more of the total response variation comes from the non-sensitive items. Whatever length you choose, pilot data on how control item responses are distributed in your population is essential — the distribution must not be compressed near zero or the maximum.

Control item construction. Items should be unambiguous, phrased as affirmative statements, and selected to produce genuine variation in your study population. Avoid items that are nearly universally true or universally false: if most respondents are already near the top or bottom of the scale, the addition of the sensitive item shifts their count to a value that makes their answer obvious. Items should also be similar in social salience and grammatical structure to the sensitive item, to reduce priming effects and differential interpretation.

Sample size. Because the ICT estimate is a difference between two group means, its variance is larger than for a direct proportion estimate at the same sample size. As a rough planning guide: for a target prevalence of 20% and a desired margin of error of 5 percentage points, plan for 400–500 respondents per arm. Blair and Imai’s list R package (Blair & Imai, 2012) provides a simulation-based power calculator tailored to this estimator.

Single-list vs. double-list design. In the standard single-list design, each respondent receives either the control list or the treatment list. The double-list design, introduced by Tsuchiya, Hirai and Ono (2007), uses two distinct non-sensitive item lists (A and B): half the sample receives list A with the sensitive item added plus list B as control; the other half receives list B with the sensitive item added plus list A as control. Every respondent answers both lists, in counter-balanced order.

The design has two advantages over single-list. First, it supports a within-respondent estimator: each respondent’s difference between their treatment-list count and their control-list count is an individual-level estimate of their sensitive item status, and the mean of those differences is the prevalence estimate. Because respondent-level heterogeneity is differenced out, this estimator has substantially smaller variance than the between-subjects difference. Second, the two list-specific prevalence estimates can be compared formally: a significant difference between them is evidence of a list-specific design effect (Glynn, 2013).

The trade-off is respondent burden — each interview now contains two count tasks instead of one — and the risk that respondents detect the design across the two lists. Counter-balancing the order across respondents reduces but does not eliminate the latter concern.

Wording the sensitive item. Specific items with a defined reference period outperform vague ones. “I paid a government official without a receipt in the past 12 months” produces a more interpretable estimate than “I have engaged in informal financial arrangements.” The item should be a factual statement rather than a normative judgment, and the enumeration instruction — asking for a count, not identification — must be delivered exactly as scripted.

Caveats & common mistakes

Ceiling and floor effects. When the control list mean is close to the maximum or minimum possible count, treatment respondents whose answer shifts the total to an extreme value have disclosed their sensitive status. Blair and Imai (2012) recommend checking the distribution of control arm responses before fielding at scale. Careful item selection and piloting prevents this.

No-design-effect assumption. The ICT requires that adding the sensitive item does not change how respondents answer the non-sensitive items. This assumption can fail through acquiescence shifts, priming, or social desirability effects triggered by the sensitive topic. Single-list designs cannot detect this violation; the double-list design supports a formal test (Blair & Imai, 2012; Glynn, 2013) — implemented as ict.test() in the list R package. When there is strong theoretical reason to expect a design effect and the single-list design is used anyway, estimates should be interpreted with that limitation stated.

Deflation. This is the most important failure mode and the list format does not eliminate it. A treatment-arm respondent who holds the sensitive attribute can lower their count of non-sensitive true items to keep the total at a “safe” level — reporting 2 instead of 3 to avoid signalling that the sensitive item is true. The treatment mean drops, and the prevalence estimate under-reports the true rate. Holbrook and Krosnick (2010), Rosenfeld, Imai and Shapiro (2016), and Blair, Coppock and Moor (2020) document this pattern across many studies. The list format reduces social desirability bias relative to direct questioning but does not eliminate it. For heavily stigmatised topics, treat ICT estimates as plausible lower bounds, not point estimates.

Ethics and consent. ICT is often the chosen method precisely because direct questioning is too risky for the respondent. The indirect format is itself the privacy mechanism, and the consent process should reflect that. Do not debrief respondents by revealing which item was sensitive — it undermines their privacy and contaminates the population for future studies. Whatever oversight your study runs under (IRB, ethics committee, internal review), document the rationale for ICT over direct questioning, the storage and access plan for the count data, and the field protocol for handling distress.

Statistical power. An underpowered ICT study is the most common implementation failure. A study with 200 respondents per arm and a true prevalence of 20% produces a 95% confidence interval of roughly ±14 percentage points — wide enough to be uninformative in most applied contexts. Power calculations should use the correct variance formula for the difference-in-means estimator, not the simpler formula for a direct proportion.

Enumerator implementation. The privacy guarantee depends entirely on enumerators delivering the count instruction as scripted. Any deviation — rephrasing the question, prompting for individual items, or recording which items are true — destroys the protection that makes the method work. This should be a training priority and subject to back-checks.

Individual identification. The ICT produces a population estimate. Using it to identify which individual respondents hold a sensitive attribute — for screening or targeting purposes — is methodologically inappropriate and defeats the design.

Analysis Guide

import pandas as pd
import statsmodels.formula.api as smf
from scipy import stats

# 1. Check balance — assignment should be ~50/50; a large imbalance flags a
#    randomisation bug that biases the estimate even if the mechanism is unbiased
df['treatment'].value_counts()

# 2. Inspect the control-arm count distribution — under random assignment, the
#    treatment arm is the control arm shifted right by the sensitive prevalence;
#    floor (most counts = 0) or ceiling (most = J) effects defeat the privacy
#    guarantee AND inflate the SE of the prevalence estimate
control = df.loc[df['treatment'] == 0, 'count_items']
control.describe()
control.value_counts().sort_index()

# 3. Difference-in-means prevalence estimate (unbiased under random assignment +
#    no-design-effect + no-liars); for cluster-randomised designs use the OLS
#    form below with cov_type='cluster', not scipy's ttest_ind
stats.ttest_ind(
  df.loc[df['treatment'] == 1, 'count_items'],
  df.loc[df['treatment'] == 0, 'count_items'])

# 4. Equivalent linear-model form — treatment coefficient is the prevalence
#    estimate; cov_type='HC1' for heteroskedasticity-robust SEs, or
#    cov_type='cluster', cov_kwds={'groups': df['psu']} for cluster-randomised
#    designs; covariates absorb residual variance but do not change the target
#    parameter under random assignment
smf.ols('count_items ~ treatment', data=df).fit(cov_type='HC1').summary()
smf.ols('count_items ~ treatment + age + female + hh_size',
      data=df).fit(cov_type='HC1').summary()

# 5. Maximum-likelihood estimator (Imai 2011) and the design-effect test —
#    no Python equivalent. The ML estimator is more efficient than difference-
#    in-means and supports covariate adjustment in the prevalence equation;
#    use R's list package (ictreg, ict.test) — see the R tab — or call it
#    from Python via rpy2 if the analysis pipeline must stay in Python

library(list)

# 1. Check balance — assignment should be ~50/50; large imbalance flags a
#    randomisation bug that will not be fixed by analysis
table(df$treatment)

# 2. Inspect the control-arm count distribution — floor (most = 0) or ceiling
#    (most = J) effects here defeat both the privacy guarantee and the precision
#    of the prevalence estimate; report this distribution alongside any estimate
control <- subset(df, treatment == 0)
summary(control$count_items)
table(control$count_items)

# 3. Difference-in-means prevalence estimate (unbiased under random assignment +
#    no-design-effect + no-liars); use lm_robust from estimatr for clustered SEs
#    in cluster-randomised designs (clusters = "psu")
t.test(count_items ~ treatment, data = df)

# 4. Maximum-likelihood estimator (Imai 2011; default in ictreg) — more
#    efficient than difference-in-means and supports covariate adjustment in
#    the prevalence equation; J is the NUMBER OF CONTROL ITEMS (here, 4 control
#    items + 1 sensitive = 5 in the treatment list). Confirm by checking
#    max(df$count_items[df$treatment == 0]) <= J
fit_uncond <- ictreg(count_items ~ 1, data = df, treat = "treatment", J = 4)
fit_adj    <- ictreg(count_items ~ age + female + hh_size,
                   data = df, treat = "treatment", J = 4)
summary(fit_uncond); summary(fit_adj)

# 5. Formal test for design effects (Blair & Imai 2012) — null is that the
#    sensitive item does not change responses to the non-sensitive items;
#    p < 0.05 is evidence of a design effect and means the simple estimate
#    cannot be interpreted as unbiased prevalence
ict.test(df$count_items, df$treatment, J = 4)

SurveyCTO / XLSForm

For within-form randomization, use a calculate field evaluated once per submission: once(if(random() < 0.5, 0, 1)). The outer once() is critical — without it, the expression re-evaluates on every form recompute and the assignment changes mid-interview. For studies where exact arm balance matters or where assignment must be stable across multiple devices, pre-randomize on the server and load assignments via pulldata() from a uploaded dataset rather than randomising in the form.

Store the assignment in a saved field (not just a calculate) so the analyst can recover which arm each respondent was in — this is a routine field bug that breaks the analysis entirely. Use relevant conditions to display the appropriate list version based on the assignment. Record the count response as type: integer with constraint: . >= 0 and . <= ${list_length}. Do not display individual items as checkboxes — respondents must hear or read the list as a group and report only the count.

Randomise the order of items within each list across respondents using a shuffled index (one once() per item position), as Holbrook and Krosnick (2010) recommend, to neutralise order effects that can otherwise mimic design effects in the analysis.

Reading the output

The key number is the difference in means — not either arm mean alone; the treatment mean blends sensitive and non-sensitive items and is uninterpretable on its own
95% CI half-width ≤ 5 percentage points: informative; 5–10 percentage points: marginal, report cautiously and discuss the precision limitation; >10 percentage points: uninformative — the study is too small for the design and the estimate should not be acted on
Negative point estimate within sampling error of zero (CI crosses zero): consistent with low or zero true prevalence — report as such, do not assume a coding error. Negative point estimate well below zero (CI excludes zero on the negative side): not possible under the model and indicates a coding error, a strong design effect, or both
Control arm mean near 0 or near the maximum list length: ceiling or floor effect likely — report the full control distribution alongside the estimate and discuss the disclosure risk
Treatment-arm variance substantially larger than control-arm variance: consistent with a positive design effect (the sensitive item altering responses to the non-sensitive items); run ict.test() (R) or compare list-specific estimates from a double-list design before interpreting
ML estimate (ictreg / kict) and difference-in-means estimate diverging by more than a few percentage points: usually a small-sample issue, but check whether the control-arm distribution violates the ML model’s assumptions before relying on the ML number

References

Aronow, P. M., Coppock, A., Crawford, F. W., & Green, D. P. (2015). Combining list experiment and direct question estimates of sensitive behavior prevalence. Journal of Survey Statistics and Methodology, 3(1), 43–66. https://doi.org/10.1093/jssam/smu023

Blair, G., & Imai, K. (2012). Statistical analysis of list experiments. Political Analysis, 20(1), 47–77. https://doi.org/10.1093/pan/mpr048 — Author page (preprint)

Blair, G., Coppock, A., & Moor, M. (2020). When to worry about sensitivity bias: A social reference theory and evidence from 30 years of list experiments. American Political Science Review, 114(4), 1297–1315. https://doi.org/10.1017/S0003055420000374

Corstange, D. (2009). Sensitive questions, truthful answers? Modeling the list experiment with LISTIT. Political Analysis, 17(1), 45–63. https://doi.org/10.1093/pan/mpn013

Droitcour, J., Caspar, R. A., Hubbard, M. L., Parsely, T. L., Visscher, W., & Ezzati, T. M. (1991). The item count technique as a method of indirect questioning: A review of its development and a case study application. In P. P. Biemer et al. (Eds.), Measurement errors in surveys (pp. 185–210). Wiley.

Glynn, A. N. (2013). What can we learn with statistical truth serum? Design and analysis of the list experiment. Public Opinion Quarterly, 77(S1), 159–172. https://doi.org/10.1093/poq/nfs070

Gonzalez-Ocantos, E., Kiewiet de Jonge, C., Meléndez, C., Osorio, J., & Nickerson, D. W. (2012). Vote buying and social desirability bias: Experimental evidence from Nicaragua. American Journal of Political Science, 56(1), 202–217. https://doi.org/10.1111/j.1540-5907.2011.00540.x

Holbrook, A. L., & Krosnick, J. A. (2010). Social desirability bias in voter turnout reports: Tests using the item count technique. Public Opinion Quarterly, 74(1), 37–67. https://doi.org/10.1093/poq/nfp065

Imai, K. (2011). Multivariate regression analysis for the item count technique. Journal of the American Statistical Association, 106(494), 407–416. https://doi.org/10.1198/jasa.2011.ap10415 — Author page (preprint)

Rosenfeld, B., Imai, K., & Shapiro, J. N. (2016). An empirical validation study of popular survey methodologies for sensitive questions. American Journal of Political Science, 60(3), 783–802. https://doi.org/10.1111/ajps.12205

Tsuchiya, T., Hirai, Y., & Ono, S. (2007). A study of the properties of the item count technique. Public Opinion Quarterly, 71(2), 253–272. https://doi.org/10.1093/poq/nfm012