Straightlining / Satisficing Detector

What it is

Straightlining is a form of satisficing in which a respondent gives the same (or nearly the same) answer to every item in a multi-item battery, regardless of item content. On a five-item Likert scale asking about different dimensions of satisfaction, a straightliner might mark “3 — neutral” for all five items. This behaviour reflects disengagement from the survey task: the respondent is minimising effort rather than reporting genuine opinions or experiences.

Straightlining degrades data quality because it introduces non-random measurement error in scale scores — a respondent’s scale score reflects their response style, not their true standing on the construct. It is most prevalent in long, repetitive batteries; in surveys administered without close enumerator supervision; in contexts where respondents feel social pressure to complete the interview quickly; and in CAPI surveys where the rapid auto-advance of items reduces engagement.

When to use it

Straightlining detection should be applied to any survey containing multi-item Likert batteries or rating scales with three or more items. It is especially important for composite measures (welfare indices, attitude scales, psychological measures) where straightliners can substantially alter the distribution of scale scores. Run the diagnostic before computing scale scores or indices.

The issue is widely discussed in survey methodology — Krosnick (1991) identified satisficing as a pervasive form of survey response behaviour, and later work has documented its prevalence in CAPI and online surveys in low-income settings.

How it works

Variance within respondent. For each respondent, compute the variance of their responses across all items in the battery. A respondent with zero within-person variance gave identical responses to all items. A respondent with very low variance (e.g., only two distinct values across 10 items) is a near-straightliner. Flag respondents whose within-person standard deviation falls below a threshold (typically the 5th percentile of the within-person standard deviation distribution).

Longest run of identical responses. An alternative measure is the length of the longest run of identical consecutive responses. A respondent who answered 3, 3, 3, 3, 3 on a battery exhibits a run of five — strong evidence of straightlining. A run of 4+ identical responses on a 5-item battery is a conservative flag.

Response entropy. Entropy measures the diversity of responses. For a respondent who used only one response category, entropy = 0. For a respondent who used all categories equally, entropy is maximised. Low entropy responses on a battery with heterogeneous items are flagged.

Enumerator-level straightlining rates. If straightlining clusters by enumerator — one enumerator has 30% of respondents with zero within-person variance while peers have 5% — this suggests the enumerator is rushing through the battery or filling in responses.

Key decisions

Distinguishing genuine from spurious straightlining. Some respondents genuinely have the same view on all items in a battery (e.g., all strongly satisfied). Short batteries (3 items) will produce more false positives than long ones (10 items). For batteries with fewer than 5 items, flag only zero-variance cases (all identical); for longer batteries, use a variance threshold.

Recoding vs. exclusion. Straightlined responses are informative about response quality but excluding all straightliners may bias the sample. A better approach is to: (1) flag straightlining as a quality indicator; (2) report how many scale scores are affected; (3) exclude flagged respondents from scale-dependent analyses in a sensitivity robustness check; and (4) investigate whether straightlining is correlated with the treatment variable, which would indicate a potential bias mechanism.

Battery design to reduce straightlining. Reversing the direction of some items (so agreement sometimes implies high scores and sometimes implies low) reduces straightlining and provides a diagnostic: a respondent who agrees with both positively- and negatively-worded items for the same construct is likely straightlining rather than genuinely consistent.

Caveats & common mistakes

Applying to heterogeneous batteries. Straightlining detection assumes that items in the battery measure related constructs and that genuine variance across items is expected. Applying it to a battery of factual questions (where uniform answers may be genuine) will produce false positives. Restrict the diagnostic to opinion, attitude, and rating batteries.

Not checking for acquiescence bias separately. Systematic agreement with all items (answering “agree” or “strongly agree” regardless of content) is a related but distinct problem from straightlining. Acquiescence bias inflates the mean without necessarily reducing within-person variance if some items are reversed. Both should be diagnosed.

Ignoring the timing dimension. Survey duration data can complement straightlining detection: respondents who complete a 10-item battery in 20 seconds almost certainly did not read the items. Combining low duration with low within-person variance provides stronger evidence of disengagement than either measure alone.

Analysis Guide

import pandas as pd
import numpy as np

item_cols = [f'q{i}' for i in range(1, 11)]

# 1. Within-person standard deviation — a respondent who engaged genuinely
#    with a heterogeneous battery should produce some variation across items;
#    zero SD means every item received the same response, which is the
#    cleanest possible signature of disengagement or enumerator fill-in
df['sd_battery'] = df[item_cols].std(axis=1)

# 2. Distinct values used per respondent — robust to batteries where genuine
#    responses could have low variance; a respondent using only 1 of 5 scale
#    values across 10 items is almost certainly not considering each item
df['n_distinct'] = df[item_cols].apply(lambda row: row.dropna().nunique(), axis=1)

# 3. Longest run of identical consecutive responses — a run of 5 or more
#    identical values in a 10-item battery is strong evidence of disengagement
#    because it is unlikely to arise from genuine opinions even when a
#    respondent has consistent views on some items
def longest_run(row):
  vals = row.dropna().values
  if len(vals) == 0:
      return np.nan
  runs = np.diff(np.where(np.concatenate(([1], vals[1:] != vals[:-1], [1])))[0])
  return runs.max()
df['max_run'] = df[item_cols].apply(longest_run, axis=1)

# 4. Flag complete and near-straightliners — zero SD is unambiguous; sd below
#    0.3 captures near-uniform respondents where one item differs slightly;
#    adjust the near-straight threshold to battery length
df['straightliner'] = df['sd_battery'] == 0
df['near_straight'] = df['sd_battery'] < 0.3
print('Straightliners:', df['straightliner'].sum(),
    f"({df['straightliner'].mean()*100:.1f}%)")

# 5. By enumerator — straightlining concentrated in specific enumerators
#    points to enumerator-level behaviour (rushing or filling in), not
#    respondent disengagement; this distinction changes the intervention needed
enum_rates = (df.groupby('enum_id')
              .agg(straight_rate=('straightliner', 'mean'),
                   n=('straightliner', 'size'))
              .sort_values('straight_rate', ascending=False))
print(enum_rates[enum_rates['straight_rate'] > 0.1])

library(dplyr)

item_cols <- paste0("q", 1:10)

# 1. Within-person SD and distinct value count — SD captures overall spread;
#    n_distinct counts how many different response categories the respondent
#    used; together they characterise engagement with the battery
df$sd_battery <- apply(df[, item_cols], 1, sd, na.rm = TRUE)
df$n_distinct <- apply(df[, item_cols], 1, function(x) length(unique(na.omit(x))))

# 2. Longest run of identical consecutive responses — a run of 5+ identical
#    values in a 10-item battery is strong evidence of disengagement because
#    it is unlikely to arise from genuine opinions even when a respondent
#    has consistent views on some items
longest_run <- function(x) {
x <- na.omit(x)
if (length(x) == 0) return(NA)
max(rle(x)$lengths)
}
df$max_run <- apply(df[, item_cols], 1, longest_run)

# 3. Flag straightliners — zero SD is an unambiguous flag; sd < 0.3 captures
#    near-straightliners where a single item breaks an otherwise uniform pattern
df$straightliner <- df$sd_battery == 0
df$near_straight <- df$sd_battery < 0.3

cat("Straightliners:", sum(df$straightliner, na.rm = TRUE),
  sprintf("(%.1f%%)
", 100 * mean(df$straightliner, na.rm = TRUE)))

# 4. By enumerator — straightlining concentrated in specific enumerators
#    points to enumerator-level behaviour (rushing or filling in), not
#    respondent disengagement; this distinction changes the intervention needed
df %>%
group_by(enum_id) %>%
summarise(straight_rate = mean(straightliner, na.rm = TRUE), n = n()) %>%
arrange(desc(straight_rate))

Reading the output

• Zero sd_battery (all identical responses) is an unambiguous straightlining flag; for batteries of 10+ items, flag respondents in the bottom 5th percentile of sd_battery as near-straightliners.
• n_distinct == 1 confirms complete straightlining; n_distinct == 2 on a 10-item battery is suspicious.
• A run of 5 or more identical consecutive responses on a 10-item battery is a strong indicator of disengagement.
• Enumerator straight_rate above 10–15% when peers are at 2–5% suggests the enumerator is rushing through the battery or completing it themselves.
• Cross-check straightlining flags with survey duration: short duration plus zero SD is stronger evidence of disengagement than either signal alone.

References

Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5(3), 213–236. https://doi.org/10.1002/acp.2350050305

Kaminska, O., McCutcheon, A. L., & Billiet, J. (2010). Satisficing among reluctant respondents in a cross-national context. Public Opinion Quarterly, 74(5), 956–984. https://doi.org/10.1093/poq/nfq048

Blasius, J., & Thiessen, V. (2012). Assessing the Quality of Survey Data. Sage Publications.

Zhang, C., & Conrad, F. (2014). Speeding in web surveys: The tendency to answer very fast and its association with straightlining. Survey Research Methods, 8(2), 127–135. https://doi.org/10.18148/srm/2014.v8i2.5453