Outcome Mapping & Harvesting

Two related developmental evaluation approaches — Outcome Mapping and Outcome Harvesting — that focus on changes in the behaviour, relationships, and actions of the people and organisations a programme directly influences, providing process-proximate evidence of programme contribution when distal outcomes are difficult to attribute.

What it is

Outcome Mapping (Earl, Carden & Smutylo, 2001) redefines what counts as a programme outcome. Rather than measuring changes in the ultimate beneficiaries (income, health, education), it tracks changes in the behaviour, relationships, actions, and activities of the programme’s direct partners (“boundary partners”) — the organisations, groups, and individuals whose behaviour the programme can most directly influence. The logic is that development programmes influence behaviour, which in turn influences systems, which eventually affects well-being — but attribution becomes progressively weaker as we move toward distal outcomes.

Outcome Harvesting (Wilson-Grau & Britt, 2012) starts from the other direction: rather than specifying expected outcomes in advance, it collects evidence of changes that have already occurred and then works backward to assess whether and how the programme contributed.

Both approaches are useful for adaptive programmes in complex systems — particularly when the causal chain from intervention to ultimate outcome is long, uncertain, or context-dependent.

When to use it

Outcome Mapping and Harvesting are appropriate for: advocacy and systems-change programmes where the primary pathway to impact runs through changing the behaviour of organisations and policymakers; complex adaptive interventions where pre-specification of outcomes is not feasible; and evaluation contexts where attribution to a specific programme is impossible but contribution analysis is possible.

They complement rather than replace quantitative impact evaluation. When an RCT measures a primary distal outcome (income, health), Outcome Mapping can track the intermediate behavioural changes that constitute the mechanism.

How it works

Outcome Mapping:

Intentional design. Before implementation, map the programme’s sphere of influence: who are the boundary partners? What changes in their behaviour, relationships, and actions would indicate progress? Express these as progress markers — a graduated scale from “least” to “more” to “most” expected behaviour change.
Monitoring. Track boundary partner behaviour against progress markers throughout implementation. This is formative monitoring, not summative evaluation.
Evaluation. Assess whether the changes in boundary partner behaviour have occurred, whether they are plausibly related to the programme, and whether they are on the pathway to systemic change.

Outcome Harvesting:

Harvesting. Field staff and programme participants identify changes that have occurred — in behaviour, relationships, policies, practices — during the programme period. These are “outcome statements”: descriptions of a change, its significance, and the programme’s contribution.
Verification. Each outcome statement is verified by checking with sources other than the programme itself: beneficiaries, partner organisations, secondary data, observation.
Analysis. Verified outcomes are clustered, analysed for patterns, and connected to the programme theory of change.

Key decisions

Boundary partner selection. The choice of boundary partners is critical for Outcome Mapping. Partners should be: (1) directly influenced by the programme; (2) in a position to create change for ultimate beneficiaries; and (3) few enough to monitor systematically (3–8 partners is typical). Including too many boundary partners dilutes the monitoring focus.

Progress marker calibration. Progress markers on the “most expected” end of the scale should be ambitious but plausible — achievable within the programme timeframe. Markers that are routinely achieved with no effort are uninformative; markers that cannot be achieved in the time horizon are demoralising. Calibrating markers against field experience from similar programmes is important.

Integration with impact evaluation. If an RCT is running alongside Outcome Mapping or Harvesting, the evaluation should use the qualitative process evidence to interpret quantitative findings — particularly null results (which might reflect failure at the mechanism level, visible in Outcome Mapping data) and heterogeneous effects (which might reflect variation in boundary partner behaviour change).

Caveats & common mistakes

Confusing contribution with attribution. Outcome Harvesting assesses the programme’s contribution to an outcome — not its sole causation. Contribution claims (“the programme contributed to this policy change”) are weaker than attribution claims (“the programme caused this policy change”) but are honest about the limits of evidence in complex systems. Overstating from contribution to attribution is a credibility-destroying error.

Using Outcome Mapping as a substitute for impact evaluation. Boundary partner behaviour change is a proxy for impact, not impact itself. A programme that achieves excellent progress markers but produces no measurable change in beneficiary well-being has failed at its ultimate objective. Outcome Mapping answers proximate questions; impact evaluation answers distal ones.

Insufficient verification in Outcome Harvesting. Outcome statements based only on programme staff reports are subject to selection bias and motivated reasoning. Independent verification — contacting boundary partners or beneficiaries who are not employed by the programme — is the minimum credibility standard.

Analysis Guide

These methods do not use statistical code. Tracking templates:

Outcome Mapping progress journal (simplified):

For each boundary partner:
- partner_name
- progress_marker (text of the specific marker)
- marker_level (least / more / most expected)
- date_achieved (if applicable)
- evidence_description
- assessor_notes

Outcome Harvesting database:

- harvest_id
- outcome_statement (what changed, in whom)
- date_of_change
- programme_contribution (description of how programme contributed)
- significance_to_whom
- verification_source
- verification_status (unverified / partially / fully)
- cluster_theme (for pattern analysis)

Once tracking data are in tabular form, use the following to summarise progress and harvested outcomes:

import pandas as pd

# progress_journal: DataFrame with one row per progress marker assessment
# vars: partner_name, marker_level, date_achieved, achieved (0/1)

# 1. Achievement rate by marker level — below 50% on "least expected" markers
#    after midpoint indicates a fundamental implementation problem, not slow progress
pd.crosstab(progress_journal['marker_level'], progress_journal['achieved'],
          normalize='index') * 100

# 2. Partners with no "most expected" markers achieved — these partners are
#    stalled at the lower tiers of the progress hierarchy; flag for review
most_achieved = (progress_journal.assign(
  is_most=(progress_journal['marker_level'] == 'most') & (progress_journal['achieved'] == 1))
  .groupby('partner_name')['is_most'].max())
most_achieved[most_achieved == 0].index.tolist()

# 3. Outcome Harvesting verification status by theme — unverified outcomes
#    exceeding 30% of a theme's harvest mean contribution claims need caution
# harvest_db: DataFrame with one row per outcome statement
pd.crosstab(harvest_db['cluster_theme'], harvest_db['verification_status'],
          normalize='index') * 100

library(dplyr)

# progress_journal: data frame with one row per marker assessment

# Achievement rate by marker level
progress_journal %>%
group_by(marker_level) %>%
summarise(
  n_markers = n(),
  n_achieved = sum(achieved, na.rm = TRUE),
  pct = round(100 * mean(achieved, na.rm = TRUE), 1)
)

# Outcome Harvesting: verification status by theme
harvest_db %>%
count(cluster_theme, verification_status) %>%
tidyr::pivot_wider(names_from = verification_status,
                   values_from = n, values_fill = 0)

Reading the output

A partner achieving fewer than 50% of “least expected” markers after the programme midpoint indicates a fundamental implementation problem at the boundary partner level, not just slow progress.
If “most expected” markers are achieved by more than one partner ahead of schedule, the programme theory may be underestimating what is achievable — consider revising progress markers upward.
Outcome statements with verification status “unverified” exceeding 30% of the total harvest should not be used for contribution claims in any report; address verification gaps before analysis.
A cluster theme with zero fully verified outcomes but many unverified statements signals a domain where the programme has limited independent evidence — treat contribution claims in that domain with caution.

References

Earl, S., Carden, F., & Smutylo, T. (2001). Outcome Mapping: Building Learning and Reflection into Development Programs. International Development Research Centre. https://www.idrc.ca/en/book/outcome-mapping-building-learning-and-reflection-development-programs

Wilson-Grau, R., & Britt, H. (2012). Outcome Harvesting. Ford Foundation. https://outcomeharvesting.net

Mayne, J. (2012). Contribution analysis: Coming of age? Evaluation, 18(3), 270–280. https://doi.org/10.1177/1356389012440912

Patton, M. Q. (2011). Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. Guilford Press.