Most Significant Change Method

A participatory monitoring and evaluation method in which program participants, field staff, and managers collect stories of meaningful change, which are then systematically reviewed and selected up the organisational hierarchy to identify what kind of change is most valued and occurring.

What it is

Most Significant Change (MSC) is a participatory monitoring technique developed by Davies and Dart (2005) in which no pre-defined indicators are used. Instead, participants and field staff regularly collect stories of the most significant change they have observed since the last collection period. These stories are passed up through the programme hierarchy — from participants to field workers to supervisors to programme managers — with each level selecting the stories it considers most significant and documenting why.

MSC is not a substitute for impact evaluation. It does not establish causality and is not representative — it samples the most dramatic changes, not the average experience. Its value lies in: (1) capturing unanticipated outcomes that pre-specified indicators miss; (2) surfacing what participants and implementers actually value, which may differ from what evaluators measure; and (3) generating hypotheses about mechanisms and unexpected effects for subsequent quantitative investigation.

When to use it

MSC is appropriate as a formative evaluation tool during program implementation — particularly in complex, adaptive programs where the theory of change is uncertain and the range of potential outcomes is wide. It complements pre-specified outcome measurement by expanding the aperture for what counts as a meaningful result.

MSC is least appropriate when the primary evaluation question is causal — whether the programme produced an effect — because stories cannot be attributed to the programme without comparison. It is best combined with a quantitative panel survey or administrative data system that tracks pre-specified outcomes alongside the MSC story collection.

How it works

Domain selection. Define the domains within which stories will be collected (e.g., “changes in people’s livelihoods,” “changes in women’s role in household decision-making”). Domains should be broad enough to allow unexpected stories to emerge but focused enough to make the collection manageable.

Story collection. Field staff ask participants: “Looking back over the past [month/quarter], what do you think has been the most significant change in [domain] for you or your family? Why do you consider this significant?” Stories are recorded verbatim or in detailed notes, including the participant’s explanation of significance.

Selection process. Stories are reviewed at each level of the organisation. Each level selects the story it considers most significant (or a small shortlist), documents why it was selected, and passes the selection upward. This creates a record of what different stakeholders value, not just what happened.

Periodic review. At defined intervals, a selection panel reviews the accumulated most-significant stories from all levels. Patterns across stories — recurring themes, types of change, populations disproportionately represented — are documented.

Verification. A sample of stories should be verified with the story-teller — confirming the story occurred, clarifying details, and checking whether the participant still views it as significant.

Key decisions

Collection frequency. Monthly collection is common in active programmes; quarterly collection is more sustainable for longer-term monitoring. More frequent collection risks participant fatigue and declining story quality.

Number of stories per collection. Asking for one story per participant per period is typical. Asking for multiple stories (best and worst change, or multiple domains) increases richness but increases burden.

Quantitative integration. MSC stories can be coded thematically (using qualitative coding — see the Qualitative Coding guide) and the frequency of themes tracked over time. Recurring story themes that do not appear in the quantitative outcome battery are candidates for adding to the formal indicator set.

Caveats & common mistakes

Using MSC to claim impact. MSC stories are not evidence of programme impact. They describe changes that occurred; they do not establish that the programme caused those changes. Conflating “significant change occurred” with “the programme caused significant change” is a common and serious misuse.

Restricting stories to positive changes. The MSC question asks for “most significant change,” not “most positive change.” Significant negative changes are equally valid — and more diagnostically valuable for program management. Training field staff to accept and document negative stories is important.

Not documenting the selection process. The selection process — who selected which stories and why — is as informative as the stories themselves. It reveals the theory of change in use: what participants, field workers, and managers actually consider valuable. If this process is not documented, half the value of MSC is lost.

Analysis Guide

MSC does not use statistical code. The primary outputs are a story database and a selection log. A simple tracking template:

Story collection form:
- story_id
- date
- collector_id (field worker)
- participant_id
- domain
- story_text (verbatim or detailed paraphrase)
- participant_explanation_of_significance
- collector_notes

Selection log (at each review level):
- review_date
- reviewer_level (fieldworker / supervisor / manager)
- selected_story_id
- selection_rationale

After accumulating 50+ stories, code them thematically and use the following to track theme frequency over time:

import pandas as pd

# stories_coded: DataFrame with one row per story
# vars: story_id, collection_period, theme, domain, selected (0/1), story_valence

# 1. Theme frequency by collection period — a theme appearing in more than 30%
#    of stories across two consecutive periods is a persistent pattern worth flagging
pd.crosstab(stories_coded['theme'], stories_coded['collection_period'])

# 2. Selection rate by theme — which themes get selected upward by reviewers?
#    Selection rates above 50% indicate strong alignment with what reviewers value
stories_coded.groupby('theme').agg(
  n_stories=('story_id', 'size'),
  n_selected=('selected', 'sum'),
  sel_rate=('selected', 'mean')).sort_values('sel_rate', ascending=False)

# 3. Negative vs positive stories (if coded) — zero negative stories selected
#    at the manager level across all periods signals suppression in the pipeline
pd.crosstab(stories_coded['story_valence'], stories_coded['collection_period'],
          normalize='index') * 100

library(dplyr)
library(tidyr)

# stories_coded: data frame with story_id + theme + collection_period columns

# Frequency of themes over time
stories_coded %>%
group_by(collection_period, theme) %>%
count() %>%
pivot_wider(names_from = theme, values_from = n, values_fill = 0)

# Selection rate by theme
stories_coded %>%
group_by(theme) %>%
summarise(
  n_stories  = n(),
  n_selected = sum(selected, na.rm = TRUE),
  sel_rate   = round(n_selected / n_stories, 2)
) %>%
arrange(desc(sel_rate))

Reading the output

A theme appearing in more than 30% of stories across two consecutive collection periods is a persistent pattern worth flagging to programme management; single-period spikes may reflect a local event rather than a trend.
Selection rates above 50% for a given theme indicate it aligns strongly with what reviewers value — cross-check whether this theme also appears in the pre-specified outcome indicators; if not, consider adding it.
A drop in story volume per collection period by more than 40% suggests participant fatigue or deteriorating field relationships, not a reduction in meaningful change.
Negative stories selected at the manager level at any rate above zero confirm that the selection process is functioning honestly; a zero selection rate for negative stories across all periods suggests suppression.

References

Davies, R., & Dart, J. (2005). The “Most Significant Change” (MSC) Technique: A Guide to Its Use. https://mande.co.uk/docs/MSCGuide.pdf

Dart, J., & Davies, R. (2003). A dialogical, story-based evaluation tool: The most significant change technique. American Journal of Evaluation, 24(2), 137–155. https://doi.org/10.1177/109821400302400202

Patton, M. Q. (2011). Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. Guilford Press.

Rogers, P. J. (2008). Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation, 14(1), 29–48. https://doi.org/10.1177/1356389007084674