Impact Measurement: Methods, Examples and Everyday Value

What modern impact measurement should make possible

Modern impact measurement should help the people running a program understand whether support is working, for whom and under what conditions. Its value is earlier learning, a fuller account of experience and evidence that remains useful across the participant relationship.

Learn while there is still time to respond

A result available only after a program closes can inform the next cycle. Evidence available during delivery can also improve the current one: staff can investigate non-attendance, adapt support or follow up with people whose needs have changed.

Understand the person behind the average

Quantitative measures show the scale and direction of change. Feedback, interviews and notes help explain the experience. Connecting both over time makes it easier to distinguish a group that benefits from one that is being missed, rather than treating an average result as everyone’s experience.

Use the same evidence for support and accountability

Applications, intake, mentoring and follow-up already contain much of the information needed to understand progress. Keeping them connected reduces repeated collection and lets a team use the evidence for participant support, program learning and reporting.

Make a claim the evidence can support

Outputs describe delivery; outcomes describe changes in circumstances, behavior or experience. An impact claim examines the program’s contribution, including other explanations and unintended effects. The OECD evaluation glossary provides a reference for these distinctions. More activity is useful to know about, but it is not sufficient evidence of better outcomes.

Why useful findings arrive too late

Separate collection creates separate fragments of the participant’s story. A staff member may know that someone stopped attending, while a survey contains an explanation and a mentor note records a related difficulty. If bringing these together requires an export, identity matching and manual coding, the organization learns slowly even when it collects plenty of data.

Open Play Foundation’s earlier process illustrates the delay. In his interview with Sopact, Marco Botha described paper activity records collected from facilities and entered into Excel. The information could be roughly six weeks behind the activity before he could use it. That limited its value for deciding what to change the following week.

The practical goal is to shorten the time between an observation and an informed response. Keeping evidence connected also reduces repeated requests to participants and preserves knowledge when staff change. An annual evaluation can then build on a record of learning rather than reconstruct the entire year.

Choose methods around the question you need to answer

Repeated measures show the direction of change

Baseline, completion and follow-up measures help establish a participant’s starting point and later circumstances. Following the same people matters: comparing different groups at each stage can disguise who improved or disappeared from the evidence. Report missing follow-up alongside the outcomes you observed.

Qualitative evidence explains the experience

Open-ended responses, interviews and staff notes can reveal barriers that a predefined scale does not capture. A learner’s assessment may improve while their feedback describes difficulty applying the skill at work. Combining the two helps the team decide whether the next support should address knowledge, opportunity or confidence.

Operational evidence shows where delivery needs attention

Attendance, response times, referrals and facility availability help explain whether a program reaches people as intended. They support current decisions and provide context for later outcomes. A quicker referral is an operational improvement; its effect on a person’s circumstances is a separate result to examine.

Evaluation design determines the strength of the claim

Before-and-after evidence can show observed change. Establishing how much the program caused may require comparison groups, experimental or quasi-experimental methods, or a carefully developed contribution analysis. Choose the design around the decision and the strength of the claim required. AI helps examine evidence; it does not create a missing comparison group.

Open Play: finding a participation problem nobody had asked about

While preparing a performance report, Marco asked AI Assistance to identify inefficiencies the team might be overlooking. The analysis surfaced declining retention among girls over roughly six weeks. It also raised a possible connection with late-afternoon sessions and getting home safely after dark.

He had not started with a question about transport. Exploring participation in context opened a more useful investigation: would earlier sessions or transport help? Those were recommendations to discuss with staff and participants, not an established cause or a verified improvement in retention.

This is the value of looking beyond a total. A facility can appear busy while a group gradually stops returning. The finding gave the team a specific question about access and delivery while it could still respond.

“I would say 80% of the time we spent on the system comes down to driving efficiencies.”

Marco Botha, CEO and co-founder, Open Play Foundation

Marco’s estimate describes how he uses the system, not a percentage of time saved. His team also examines differences across coaches and activities and meets on Mondays to review data and plan the week. See the Open Play Foundation account for the wider operational story.

Build measurement into the relationships you already manage

An application can establish the need and intended result. Onboarding adds the starting circumstances. Case notes, mentoring and assessments explain what happens during support. Follow-up shows whether the change lasts. Keeping that history together makes measurement useful throughout the relationship.

For training and accelerator programs, earlier application evidence can inform mentoring and pre/post review. For grants, the proposal and agreed outcomes give context to later partner reports. Membership organizations can connect recurring feedback with participation and changing member needs. A scholarship or award that ends at selection may need evidence about access and decision quality without imposing a continuing outcome study.

This approach does not require collecting every possible detail. It requires collecting information that serves a clear question and reusing it where appropriate. The team supporting a participant and the team preparing a report can work from the same evidence with access suited to their responsibilities.

Measurement changes with the program and the decision

For a nonprofit training program, useful context includes the participant’s starting point, support received and follow-up coverage. Measuring a later employment outcome means defining employment and the follow-up window, then keeping nonresponse visible. An investor examining employment at a portfolio company needs different context: headcount or full-time-equivalent definitions, company boundaries and investment timing.

A CSR team needs to distinguish the company’s contribution from a partner’s delivery and community outcomes. Sustainability measurement may concern operations and the value chain, with explicit site boundaries and calculation methods. A shared label such as “people reached” or “emissions reduced” cannot resolve these differences.

Design a small dictionary around the decisions the evidence must support. Specify the unit, population, period, source, calculation and rules for combination. Then retain the objectives and relevant history around those measures. Our impact reporting guide shows four worked designs for nonprofits, impact investments, CSR programs and sustainability.

Give the evidence shared meaning

A practical theory of change explains why the activities are expected to help. A data dictionary makes the measures interpretable: what counts as participation, what an outcome means, the reporting period and the people included. Together, they connect the program’s purpose with the information used to judge progress.

Consider an illustrative follow-up: 100 participants are due, 80 respond and 48 report employment. Employment among respondents is 60%, and response coverage is 80%. The remaining 20 outcomes are unknown. Reporting only “60% employed” would hide an important part of the evidence.

Qualitative findings need context too. A theme mentioned in ten comments is not necessarily experienced by ten people or representative of the whole cohort. Keeping the question, source and period available helps staff interpret the pattern and decide whether further follow-up is needed.

Where Sopact fits

Sopact provides decision intelligence for programs and portfolios. It connects survey responses, quantitative records, qualitative feedback, interviews and files with the context needed to analyze them together. Configured analysis becomes available as evidence arrives, and AI Assistance helps staff explore current questions with source material available for review.

The distinction is the continuing use of that analysis. A summary of a document is useful for reading it; connected analysis lets the team revisit a question as new evidence arrives and carry what it learned into support, management and reporting. Your team owns the routine work with personalization and continuing Sopact support.

Sopact’s typical setup estimate is two days to two weeks for an agreed workflow; historical imports and integrations affect scope. Start with a program decision that currently requires repeated preparation. For platform evaluation, see impact measurement software. For how findings become action, continue to impact measurement and management.

Frequently asked questions