This review is useful when you have an outcome claim to examine, especially across programs or investments. You do not need it before every operational report. Bring one defined outcome and its supporting records; leave with a clearer claim and a prioritized evidence gap.
Review one outcome, not the entire organization.
Our fictional training provider has 40 completers, 25 follow-up responses and 15 people reporting skill use after 30 days. A report says the course helped people put learning into practice. Before accepting that wording, look at five different questions.
What: Is skill use the change people actually need?
Who: Whose experience is represented, and who is missing?
How much: What do we know about the number of people, degree of change and duration?
Contribution: What supports a role for the training, and what other explanations remain?
Risk: What could undermine the result or our interpretation?
Keep the finding within the evidence.
The responses describe reported skill use at one checkpoint. They do not establish improvement from a comparable baseline or lasting use. Fifteen completers did not respond. The team should examine that missing evidence before treating respondents as representative of everyone.
“How much” is broader than the headcount. Scale asks how many experience an outcome; depth concerns the degree of change; duration concerns how long it lasts. A single follow-up cannot answer all three.
Participant accounts can explain how training was experienced. They do not automatically establish what would have happened without it. For an investment portfolio, the investor’s contribution is also a separate question from the enterprise’s own contribution.
Find the uncertainty that affects the decision.
If the decision is whether to change practice support, the team may need to understand opportunity and barriers. If the decision is whether to claim lasting improvement, it needs suitable evidence about change and duration. More responses to the same question may not resolve the second problem.
- Write the exact outcome claim.
- Record the evidence and its limits for each dimension.
- Identify missing perspectives and plausible alternative explanations.
- Choose the gap most likely to change the decision.
- Assign a proportionate next inquiry and review date.
Collect only justified information. Group comparisons can reveal unanswered questions, but small or sensitive groups need appropriate access and disclosure controls. Do not infer the reason for a difference from a demographic label.
Avoid a reassuring score that hides a serious gap.
An internal scoring rubric can help organize review if its criteria are explicit. It is not an official rating just because it uses the five dimensions. An average can conceal a decisive missing source or an unsupported causal claim.
For this exercise, write plain notes: what is supported, what is partly supported and what is not established. Self-reported evidence is not automatically weak; it may be the right source for a person’s experience. Judge its suitability for the particular claim.
Your output
One defensible sentence, one important limitation and one next action. For example: “15 of 25 respondents reported skill use at 30 days. We lack answers from 15 completers and evidence about sustained use. The course owner will review follow-up coverage before the next reporting cycle.”
Bring that result into the evidence-trail exercise, then use The Loop to act on the review. The framework should change a decision, not simply add another document.
Method references
- Impact Frontiers: Five Dimensions of Impact
- Impact Frontiers: Who experiences the outcome?
- Impact Frontiers: Scale, depth and duration
- Impact Frontiers: Reviewing impact risk
Add this to your plan
Record what you decided in this chapter in the same working evidence plan you started in Foundations, so definitions, sources and checks stay in one place.