Enumerator Fraud Detection

A composite diagnostic that combines duration anomalies, response pattern irregularities, GPS metadata inconsistencies, and inter-item correlation analysis to flag surveys that show multiple signs of fabrication or enumerator misconduct.

What it is

Enumerator fraud — fabricating interviews, filling in questionnaires without interviewing respondents, or conducting interviews in ways that do not meet protocol — is a serious and underacknowledged problem in large-scale field surveys. Fabricated data looks like real data at the aggregate level, because an experienced enumerator will produce response distributions that roughly match the field population. Detection requires looking at the combination of signals that a fabricated interview uniquely produces: it is often too short, too similar to other interviews by the same enumerator, lacks GPS variation, or produces internal inconsistencies a genuine respondent would avoid.

No single signal is conclusive. Duration outliers, straightlining, and item non-response can all occur in legitimate interviews. Fraud detection relies on combining multiple weak signals into a composite score — an interview scoring high on several dimensions is more likely to be fraudulent than one scoring high on only one.

When to use it

Fraud detection diagnostics should be run routinely during active data collection as part of a real-time monitoring dashboard and as part of post-collection data quality assessment. They are particularly important in studies where enumerators are unsupervised in the field, where the questionnaire is long and cognitively demanding, and where payment is per-completed interview (creating an incentive to fabricate).

Standard field practice emphasises combining multiple paradata signals for fraud detection rather than relying on any single check.

How it works

Signal 1: Duration anomalies. Flag interviews with duration below the design minimum or below the 5th percentile of the distribution (see the Survey Duration Outlier Detector guide). Duration is the most sensitive single fraud signal because fabrication is almost always faster than genuine enumeration.

Signal 2: GPS metadata. Most CAPI instruments (SurveyCTO, ODK) record the GPS coordinates at interview start. Fabricated interviews are often conducted at the enumerator’s home or at a fixed location rather than at the respondent’s household. Flag interviews where: (a) the GPS coordinates match the enumerator’s home location; (b) multiple interviews share identical or near-identical GPS coordinates; or (c) coordinates are inconsistent with the assigned sampling cluster.

Signal 3: Response similarity to same-enumerator interviews. Fabricated interviews often resemble other interviews by the same enumerator more than genuinely collected interviews do. For each interview, compute the correlation between its response vector and the average response vector for all other interviews by the same enumerator. Unusually high correlation (above the 95th percentile of the distribution) is a fabrication signal.

Signal 4: Anomalous internal consistency. Some surveys include questions with known logical relationships (age vs. year of birth, household size vs. number of members listed, asset ownership and consumption). Fabricated interviews show elevated rates of inconsistency — enumerators filling in plausible values may not track the internal logic carefully.

Signal 5: Temporal patterns. Fabricated interviews are sometimes conducted outside plausible hours (before 7 am, after 8 pm) or in implausibly dense sequences (5 completed interviews within 2 hours). Timestamp analysis flags these.

Composite fraud score. Assign a binary flag (0/1) for each signal. Sum the flags across all signals to produce a composite fraud score (0–5). Interviews scoring 3 or above across five independent signals warrant manual review and verification.

Key decisions

Threshold calibration. Fraud detection thresholds (what counts as anomalously short, anomalously similar, anomalously timed) should be calibrated to the specific study context using pilot data and field experience. Conservative thresholds (flagging 2% of interviews) reduce false positives but may miss genuine fraud; liberal thresholds (flagging 15%) increase sensitivity but require more manual review resources.

Manual review protocols. Flagged interviews should be manually reviewed by a supervisor, not automatically excluded. Manual review involves: comparing the questionnaire to a typical interview from the same area, attempting to call or visit the listed respondent to verify the interview occurred, and reviewing other interviews by the same enumerator for corroborating patterns.

Confidentiality of flagging. The fraud scoring process should be kept confidential from enumerators during active collection. If enumerators know which signals are being monitored, they can adjust their fabrication to avoid detection (e.g., fabricating interviews at varied GPS locations, varying durations).

Caveats & common mistakes

Treating flags as verdicts. A high fraud score is a signal for investigation, not evidence of fraud. Some combinations of anomalies can arise from legitimate circumstances: a respondent answered very quickly because they know the subject well; multiple interviews were conducted in the same compound; GPS signal was poor and the recorded location is inaccurate. Investigation must precede any decision to exclude or take action.

Focusing only on outlier enumerators. Enumerators with the highest fraud scores are the easiest to detect, but fraud can also occur in moderate volume across many enumerators. An organisation-wide assessment of baseline fraud rates, based on back-checks and audit data, is more informative than focusing on obvious outliers.

Not building monitoring into the instrument design. Post-collection fraud detection is reactive. Proactive design — configuring SurveyCTO to require GPS capture, to log timestamps at module breaks, to include built-in back-check questions — makes fraud detection easier and deters fraud before it occurs.

Analysis Guide

import pandas as pd
import numpy as np

item_cols = [f'q{i}' for i in range(1, 21)]

# Signal 1: Duration — the most sensitive single fraud signal because
#   fabricating enumerators skip the time cost of genuine enumeration;
#   threshold should come from pilot data, not an arbitrary default
df['flag_duration'] = df['duration'] < 20

# Signal 2: GPS clustering — round coordinates to roughly 100m precision
#   and flag interviews sharing the same location within an enumerator;
#   multiple interviews at the same GPS point indicate a static fabrication
#   location rather than travel to separate respondent households
df['gps_key'] = (df['gps_lat'].round(3).astype(str) + '_' +
               df['gps_lon'].round(3).astype(str))
df['flag_gps'] = df.groupby(['enum_id', 'gps_key'])['gps_key'].transform('size') > 2

# Signal 3: Similarity to enumerator mean response vector — fabricated
#   interviews cluster around the enumerator's implicit template of a typical
#   respondent; genuine interviews should vary across the vector; the 95th
#   percentile flags interviews far closer to the template than peers
enum_means = df.groupby('enum_id')[item_cols].transform('mean')
def row_corr(i):
  obs = df.loc[i, item_cols].astype(float).values
  emn = enum_means.loc[i].values
  mask = ~np.isnan(obs) & ~np.isnan(emn)
  return np.corrcoef(obs[mask], emn[mask])[0, 1] if mask.sum() > 1 else np.nan
df['sim_to_enum'] = [row_corr(i) for i in df.index]
df['flag_similar'] = df['sim_to_enum'] > df['sim_to_enum'].quantile(0.95)

# Signal 4: Internal consistency — fabricating enumerators filling in values
#   quickly miss logical constraints genuine respondents would satisfy;
#   impossible values (negative age, zero household size, phone owners with
#   zero phones) are particularly revealing
df['flag_inconsistent'] = ((df['age'] < 0) | (df['hh_size'] <= 0) |
                         ((df['owns_phone'] == 1) & (df['n_phones'] == 0)))

# Signal 5: Timing — genuine field interviews occur during working hours;
#   timestamps outside that window suggest data entered at the enumerator's
#   convenience, not recorded in real time during the interview
df['interview_hour'] = pd.to_datetime(df['starttime']).dt.hour
df['flag_timing'] = (df['interview_hour'] < 7) | (df['interview_hour'] > 20)

# Composite fraud score — sum independent binary signals; interviews with
#   3 or more flags have converging evidence from distinct mechanisms and
#   should be prioritised for manual review; sort enumerators by their share
#   of high-scoring interviews to identify the most problematic collectors
flag_cols = ['flag_duration', 'flag_gps', 'flag_similar',
           'flag_inconsistent', 'flag_timing']
df['fraud_score'] = df[flag_cols].sum(axis=1)
print(df['fraud_score'].value_counts().sort_index())
print(df.loc[df['fraud_score'] >= 3,
           ['respondent_id', 'enum_id', 'fraud_score'] + flag_cols])

library(dplyr)

# Signal 1: Duration — the most sensitive single fraud signal because
#   fabricating enumerators skip the time cost of genuine enumeration;
#   threshold should come from pilot data, not an arbitrary default
df$flag_duration <- df$duration < 20

# Signal 2: GPS clustering — rounds coordinates to ~100m precision and
#   flags interviews sharing the same location within an enumerator;
#   multiple interviews at the same GPS point indicate a static fabrication
#   location rather than travel to separate respondent households
df <- df %>%
group_by(enum_id) %>%
mutate(
  gps_key  = paste(round(gps_lat, 3), round(gps_lon, 3)),
  flag_gps = duplicated(gps_key) | duplicated(gps_key, fromLast = TRUE)
) %>%
ungroup()

# Signal 3: Similarity to enumerator mean response vector — fabricated
#   interviews cluster around the enumerator's implicit template of a
#   typical respondent; genuine interviews should vary across the vector;
#   the 95th percentile flags interviews far closer to the template than peers
item_cols <- paste0("q", 1:20)
enum_means <- df %>%
group_by(enum_id) %>%
summarise(across(all_of(item_cols), mean, na.rm = TRUE), .groups = "drop")
df <- left_join(df, enum_means, by = "enum_id", suffix = c("", "_em"))
df$sim_to_enum <- mapply(function(i) {
obs  <- as.numeric(df[i, item_cols])
emn  <- as.numeric(df[i, paste0(item_cols, "_em")])
cor(obs, emn, use = "complete.obs")
}, seq_len(nrow(df)))
df$flag_similar <- df$sim_to_enum > quantile(df$sim_to_enum, 0.95, na.rm = TRUE)

# Signal 4: Internal consistency — fabricating enumerators filling in values
#   quickly miss logical constraints that genuine respondents would satisfy;
#   impossible values (negative age, zero household size) are particularly revealing
df$flag_inconsistent <- df$age < 0 | df$hh_size <= 0

# Signal 5: Timing — genuine field interviews occur during working hours;
#   timestamps outside that window suggest data entered at the enumerator's
#   convenience, not recorded in real time during the interview
df$interview_hour <- as.integer(format(as.POSIXct(df$starttime), "%H"))
df$flag_timing    <- df$interview_hour < 7 | df$interview_hour > 20

# Composite fraud score — sum independent binary signals; interviews with
#   3+ flags have converging evidence from distinct mechanisms and should
#   be prioritised for manual review; sort enumerators by their share of
#   high-scoring interviews to identify the most problematic collectors
flag_cols <- c("flag_duration", "flag_gps", "flag_similar", "flag_inconsistent", "flag_timing")
df$fraud_score <- rowSums(df[, flag_cols], na.rm = TRUE)
df[df$fraud_score >= 3, c("respondent_id", "enum_id", "fraud_score", flag_cols)]

Reading the output

A fraud score of 1 is common and not diagnostic on its own; scores of 3 or more across five independent signals warrant manual review.
flag_duration is the single most sensitive signal — fabricated interviews are almost always faster than genuine ones; calibrate the threshold using pilot data or field experience.
flag_gps with more than 2–3 interviews sharing identical rounded coordinates (±0.001°, roughly 100 m) is a strong fabrication signal, especially if those coordinates are outside the assigned sampling cluster.
flag_similar at the 95th percentile means that interview’s response pattern is more similar to the enumerator’s average than 95% of other interviews — not conclusive alone, but meaningful in combination.
Use the fraud score as a triage tool: sort enumerators by their share of high-scoring interviews, and prioritise those at the top for back-check and manual review.

References

Finn, A., & Ranchhod, V. (2017). Genuine fakes: The prevalence and implications of data fabrication in a large South African survey. World Bank Economic Review, 31(1), 1–16. https://doi.org/10.1093/wber/lhw024

Caeyers, B., Chalmers, N., & De Weerdt, J. (2012). Improving consumption measurement and other survey data through CAPI: Evidence from a randomized experiment. Journal of Development Economics, 98(1), 19–33. https://doi.org/10.1016/j.jdeveco.2011.09.004

McKenzie, D. (2012). Beyond Baseline and Follow-up: The Case for More T in Experiments. Journal of Development Economics, 99(2), 210–221. https://doi.org/10.1016/j.jdeveco.2012.01.002