What it is
Item non-response occurs when a respondent completes a survey but leaves specific questions unanswered, or when an enumerator skips items. Unlike unit non-response (a respondent who was never interviewed), item non-response is selective — it tends to concentrate on sensitive questions, complex items, and modules where enumerators face the most resistance. Understanding the pattern of missing data before analysis is essential: if missingness is correlated with the outcome or treatment, it can bias estimates even if the missing share is small.
Item non-response visualisation maps these patterns — across items, respondents, enumerators, and survey waves — to distinguish random gaps from systematic ones, and to identify questions that should be revised or modules that require enumerator retraining.
When to use it
Item non-response diagnostics should be run at two stages: during data collection (as a monitoring tool to identify questions with high missing rates before they accumulate), and as part of pre-analysis data quality review (to inform missing data handling decisions and assess the potential for non-response bias).
Questions with non-response rates above 5–10% warrant investigation. Very high rates (>20%) may indicate that an item is poorly worded, culturally sensitive, too cognitively demanding, or being systematically skipped by enumerators. Little’s MCAR test (Little, 1988) provides a formal test of whether missingness is completely at random (MCAR) vs. correlated with observed data. If MCAR is rejected, the pattern of missing data should inform the imputation or analysis strategy.
How it works
Item-level non-response rates. For each question in the survey, compute the proportion of respondents with missing values. Sort items by missing rate and visualise in a bar chart or heatmap. Items with unusually high rates relative to other items in the same module stand out immediately.
Respondent-level missing count. For each respondent, count the number of missing items. Respondents with many missing items may have been rushed through the survey, may have been unable to answer due to language barriers, or may have a specific characteristic (illiteracy, disability) that was not accommodated in the instrument. Respondents with zero missing items on all variables may also be suspicious in surveys with legitimately hard-to-answer questions (suggesting fill-in responses).
Missingness pattern matrix. A matrix with respondents on one axis and variables on the other, with missing values shaded, reveals whether missingness is random (scattered across the matrix) or structured (concentrated in specific rows — respondents — or columns — items). Structured missingness is diagnostic of systematic problems.
Missingness by enumerator. If one enumerator has markedly higher non-response rates for specific items, they may be skipping those items consistently. This is a training or compliance problem that should be addressed immediately during active data collection.
Predictors of missingness. A logistic regression of a missing indicator (1 = missing, 0 = observed) on respondent characteristics (age, gender, education, treatment status) tests whether missingness is correlated with substantively important covariates. If treatment status predicts missingness, differential non-response is a potential threat to validity.
Key decisions
Distinguishing item types. “Genuine” missing values (respondent truly doesn’t know, question is not applicable to them) should be distinguished from “problematic” missing values (refusal, skip, skipped in error). CAPI instruments can record reason codes for missingness: “don’t know”, “refused”, “not applicable”, “technical error”. Analysing these categories separately is more informative than treating all missing values as equivalent.
Non-response imputation. The appropriate approach to missing data depends on the mechanism. If missing is completely at random (MCAR), listwise deletion produces unbiased estimates. If missing at random (MAR — missing depends on observed covariates), multiple imputation or inverse probability weighting is appropriate. If missing not at random (MNAR — missing depends on the unobserved value itself), neither approach resolves the bias without additional assumptions. Visualising the pattern and testing the MCAR assumption informs which regime is most plausible.
Outcome vs. covariate missingness. Missing values in outcome variables are more consequential than in covariates, because outcome missingness can directly bias treatment effect estimates if it is differential across treatment and control. Covariate missingness affects precision but not identification in randomised studies (assuming randomisation was not conditional on the covariate).
Caveats & common mistakes
Treating all missing as equivalent. Conflating “don’t know” with “refused” with “not applicable” in a single missing indicator discards valuable information. A high “don’t know” rate is a comprehension signal; a high “refused” rate is a sensitivity signal; a high “not applicable” rate may indicate problems with skip logic. Record and report these separately.
Not investigating patterns before imputation. Applying multiple imputation without first checking whether missingness is correlated with treatment assignment or outcomes is a common error. Imputation under MCAR assumptions when data are MNAR produces biased imputed values. The missingness pattern should be examined before any imputation strategy is chosen.
Ignoring item-level missing in index construction. When a composite index is constructed from multiple items (a scale score, a welfare index), observations with some missing items are often assigned a score based on the available items. This is appropriate if missingness is random but can produce biased scores if specific items are consistently missing for certain respondent types. Documenting which items were missing for each observation and how the index was computed is important for transparency.
Analysis Guide
import pandas as pd
import numpy as np
import statsmodels.formula.api as smf
import matplotlib.pyplot as plt
item_cols = [f'q{i}' for i in range(1, 51)]
# 1. Item-level missing rates — sort items from highest to lowest missing
# rate to immediately see which questions are structurally problematic;
# items above 5 percent warrant investigation before analysis begins
miss_rates = df[item_cols].isna().mean().sort_values(ascending=False) * 100
print(miss_rates[miss_rates > 5].round(1))
# 2. Missingness pattern matrix — visual check for whether missingness is
# scattered randomly across the matrix (consistent with MCAR) or structured
# in blocks of rows or columns (consistent with systematic skipping or
# module-level failures)
plt.imshow(df[item_cols].isna(), aspect='auto', cmap='gray_r')
plt.title('Missingness pattern'); plt.xlabel('Item'); plt.ylabel('Respondent')
plt.show()
# 3. Respondent-level missing count — respondents in the upper tail of this
# distribution were either rushed through the survey or had a characteristic
# the instrument did not accommodate; zero-missing respondents on hard
# questions can also be suspicious
df['n_missing'] = df[item_cols].isna().sum(axis=1)
print(df['n_missing'].describe())
# 4. Test whether outcome missingness is correlated with treatment — a
# significant treatment coefficient means follow-up is differential across
# arms; this directly threatens causal identification
df['miss_outcome'] = df['primary_outcome'].isna().astype(int)
fit = smf.logit('miss_outcome ~ treatment + age + C(female) + log_hh_expenditure',
data=df).fit(cov_type='HC1')
print(fit.summary())
# 5. By enumerator — enumerators with high outcome non-response may be
# systematically skipping sensitive items; flagging early during collection
# allows targeted retraining before the problem compounds
enum_miss = df.groupby('enum_id')['miss_outcome'].mean()
print(enum_miss[enum_miss > 0.1])
# 6. Little's MCAR test — Python has no canonical implementation; the
# pingouin and pyampute libraries provide partial support, or implement
# via Hawkins-style chi-square comparing observed and expected missing
# patterns across observed-data subgroupsReading the output
- Any item with a missing rate above 5% warrants investigation; rates above 20% suggest a structural problem (poor wording, sensitivity, enumerator skipping).
- A significant MCAR test (p < 0.05) means missingness is correlated with observed data — listwise deletion will be biased; use multiple imputation or IPW.
- A significant treatment coefficient in the logit/OLS of
miss_outcomeon treatment indicates differential non-response, which directly threatens causal inference. - Enumerators with
enum_miss_rateabove 10% on a key outcome should be reviewed for systematic skipping. - Respondents with
n_missingin the top 5% of the distribution are candidates for exclusion sensitivity checks.
References
Little, R. J. A. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198–1202. https://doi.org/10.2307/2290157
Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys. Wiley.
Sterne, J. A. C., White, I. R., Carlin, J. B., Spratt, M., Royston, P., Kenward, M. G., Wood, A. M., & Carpenter, J. R. (2009). Multiple imputation for missing data in epidemiological and clinical research: Potential and pitfalls. BMJ, 338, b2393. https://doi.org/10.1136/bmj.b2393
van Buuren, S. (2018). Flexible Imputation of Missing Data (2nd ed.). CRC Press. https://stefvanbuuren.name/fimd/