Additional course links
You leave with: A data dictionary entry for Eastgate's placement-at-90-days measure, the outputs that sit beside it, and a reporting sentence that keeps missing data visible.
Where this fits: Lesson 3 drafts the reporting plan for A-31 from the onboarding call, and one line in it is still open: what counts as a job placement at 90 days. This deep dive turns Eastgate's promise into a definition both sides apply the same way. Bring back one data dictionary entry for the placement measure and the sentence you would use to report it when follow-up is incomplete. Both go into step 3 of the workbook.
An output is what a program delivers; an outcome is the change that follows for the people it serves. Twelve weeks of job-readiness sessions delivered is an output; participants in paid work 90 days later is an outcome. Report both: outputs explain what happened, and a defined outcome measure tells you whether the intended change occurred.
Separate activities, outputs, outcomes and impact
The OECD evaluation glossary distinguishes the products and services an intervention delivers from the effects that follow and the longer-term impacts. Use your funder's definitions where they differ. What matters is saying what each measure represents rather than relabeling every count as impact.
| Level | Question | Eastgate example (fictional) | Evidence to plan |
|---|---|---|---|
| Activity | What work will we do? | Run 12-week job-readiness sessions and employer introductions | Session schedule and records |
| Output | What was delivered, and to whom? | Unique youth enrolled; eligible participants attending at least 3 of the first 4 sessions | Eastgate's program report |
| Outcome | What changed for participants? | Participants in paid work 90 days after the program | Dated follow-up against a written definition |
| Impact ambition | What wider change might we contribute to? | More stable work for young people in Lake County | Broader evidence and an evaluation design suited to the claim |
The boundary depends on the program's purpose. An employer introduction is an output of a placement service; a job offer is a different result, and a job started is another. Explain the measure rather than trusting its label.
Follow the chain from enrolled to attended to placed
Moving toward outcomes does not mean dropping attendance or enrollment. Those counts tell a delivery problem from a theory that did not hold: low attendance points toward recruitment or access, while high attendance with low placement points toward skills, employer demand or the move into work.
Use one cohort across the chain. Eastgate's Q1 update reports 180 registrations and 150 eligible participants, of whom 108 attended at least three of the first four sessions.
| Stage | Count | Rate | What it answers |
|---|---|---|---|
| Registered | 180 | — | Interest; not yet enrollment |
| Eligible participants | 150 | — | Who the attendance measure counts |
| Attended 3 of first 4 sessions | 108 | 108 ÷ 150 = 72% | Did eligible participants stay past the start? |
| Placed at 90 days | Not yet due | — | The outcome; see the follow-up below |
The same 108 is 60% of registrants (108 ÷ 180) and 72% of eligible participants. Both are correct; they answer different questions. The plan defines attendance against eligible participants, so 72% is the figure Eastgate reports. The other 30 registrations need a status (not yet eligible, not started, withdrawn or unknown) before anyone calls them enrolled.
Write the outcome as an observable change
A promise to "help young people into work" is an intention, and a program officer cannot tell from it what evidence to look for. "Eligible participants are in paid work 90 days after completing the 12-week program" is closer. It still needs rules.
Settle them on the onboarding call or straight after. Does paid work include temporary roles, self-employment or a minimum number of hours a week? Does the 90-day clock start at each participant's completion or at the program's scheduled end? How are people who left early counted? Write the answers down before the first follow-up, not after the numbers arrive.
Resist making the statement more ambitious to impress a funder. If Eastgate can confirm placement at 90 days but not yet at six months, the plan should say so. A modest result that holds up is worth more than an impact claim nobody can check.
Build a small data dictionary for the indicator
The outcome is the change. The indicator says how you will observe it. A target is what you aim for, a baseline is the starting point, and the actual value is what a report submits. Keep all four apart.
| Field | Eastgate entry (fictional; adapt to your program) |
|---|---|
| Indicator and owner | Paid work 90 days after the program; Eastgate's program lead reports, Horizon's program officer checks |
| Cohort | Eligible participants in the named 12-week cohort, with the enrollment cutoff recorded |
| Numerator | Participants meeting the paid-work definition on the follow-up date |
| Denominator | Report both the eligible cohort and the number with known status |
| Paid-work definition | Hours, contract types, self-employment and the evidence accepted, agreed with Eastgate |
| Timing | 90 days after the agreed reference date; record the actual follow-up date |
| Source | Eastgate's follow-up records, uploaded with its quarterly program report |
| Missing data | Unknown status reported separately; never counted as placed or as not placed |
| Target | None agreed yet; if one is set, it sits apart from actual values |
| Version | Definition v1, the date agreed and the award it applies to (A-31) |
For retention at six months, write a separate entry. Being in work at a six-month check-in is not the same as six months of continuous work.
Read the result without hiding missing data
Placement at 90 days is not due in the Q1 update. When it comes, the follow-up will be incomplete; it always is. Suppose, for illustration, that 100 of the 150 eligible participants have a known status at 90 days, and 60 of them meet the paid-work definition.
Illustration · fictional
60 ÷ 100 = 60% of participants with known status are placed.
100 ÷ 150 = 66.7% of the eligible cohort has a known status.
60 ÷ 150 = 40% of the eligible cohort is confirmed placed.
Report the three together. "60% placed" hides the 50 people nobody reached. "40% placed" treats every unknown as a failure. The honest line is the confirmed share with the response gap beside it.
Check whether the unknowns cluster by site or start date. Missing follow-up may mean people facing the hardest barriers, or people too busy in new jobs to answer; the silence alone does not tell you which.
Use qualitative evidence to explain the numbers
Add one focused question to the follow-up: "What helped or got in the way of using the training in your job search?" Keep each answer attached to the participant, date and program stage.
A comment about getting to sessions may explain missed attendance, and Eastgate's Q1 budget already moved $2,000 from materials to transport stipends, approved with a reason. A comment about employer requirements may explain a placement gap. Read these beside the counts, and keep the voices that contradict the story along with the ones that fit it.
In Sopact Sense, an Intelligence Cell reads each answer as it arrives and tags it with the themes you define. The program officer checks the tagged passages before anything goes into a report.
Separate observed change from causal impact
If placement is high after Eastgate's program, that is an observation. Saying the program caused it needs more: the local labor market, prior experience, other support and who chose to enroll all play a part. A before-and-after count describes change for the people measured; it does not tell you what would have happened without the program. For a one-year, $24,000 grant, reporting the observed result, the plausible contribution and the limits is usually right. Causal claims need an evaluation design, and the reporting plan should not promise one it does not fund.
Hold the rubric to the same line. R-2's outcome pathway criterion, worth 30 points, rewards a credible pathway and a proportionate evidence plan, not certainty.
Link the application promise to the reporting period
Eastgate's answer to "how will you know it worked?" is part of the assessed submission. Keep it there, unchanged. The approved definition, v1 of the paid-work measure, goes on the award. Each quarterly update names the period, cohort, definition version, actual value and source file. If a definition or target changes, record the change as an amendment with a reason and an approval, beside the original.
Watch · 11:08–12:43 · agreeing metrics and checking reports
Unmesh describes the onboarding call that settles which metrics a grantee reports, grantees updating documents they already keep, and a rubric that checks each report for what is complete and what is missing. Watch for the unique ID: it is how a missing figure gets requested against the right grant. Watch on YouTube ↗
Jump to: 8:14 — transcript to theory of change · 12:43 — data dictionary and reporting requirements
Across a portfolio, compare only what is comparable. If another R-2 grantee counts job offers and Eastgate counts paid work started, the two do not add into one "placements" total.
Put it into practice
Define Eastgate's placement measure
- In step 3 of the workbook, write Eastgate's outcome as one observable change with a population, a change and a time point.
- Complete the data dictionary entry for placement at 90 days, including the paid-work definition, reference date and missing-data rule.
- List the outputs that sit beside it in the reporting plan and the denominator each uses. Check the 72% attendance figure against
eastgate-q1-update.csv. - Calculate the three follow-up measures from the illustration and write one sentence that reports them accurately.
- Give your definition to a colleague with two borderline cases: a participant in a six-week temporary role, and one who left in week 8 and found work. Revise any wording they applied differently from you.
Download the workbook (PDF)
Practice data: horizon-r2-applications.csv · horizon-rubric.csv · calibration-scores.csv · eastgate-q1-update.csv
Check your reasoning before moving on
A usable reporting sentence reads: "At the 90-day follow-up, 60 of the 100 participants with known status were in paid work under the agreed definition. Status was known for 100 of 150 eligible participants; 50 remain unknown. The figures describe observed placement, not the program's causal effect." Attendance is 108 ÷ 150 = 72%, not 108 ÷ 180. The two borderline cases should be settled by your definition, not by your colleague's judgment; if they had to decide, add the missing rule (minimum duration or hours, how early leavers count). Any placement target sits in its own field, apart from actual values. Take the finished entry back to the reporting plan in Lesson 3 and close the open item.
Questions grant teams ask
What is the difference between an output and an outcome?
An output is a service or product delivered: sessions run, young people enrolled, attendance. An outcome is a change that follows for participants, such as being in paid work 90 days after the program. Outputs explain delivery and access; outcomes show whether the intended change occurred. A grant report needs both, each with a clear definition and the same cohort.
Is attendance an outcome?
Usually not. Attendance measures participation or reach, not the change the program intends. It remains useful: high attendance with low placement points toward skills or employer demand, while low attendance points toward recruitment or access. Report attendance as an output with its denominator stated, such as eligible participants attending at least three of the first four sessions.
What is the difference between an outcome and an indicator?
The outcome describes the change you care about. The indicator defines how you will observe it: the cohort, numerator, denominator, timing, source and missing-data rule. A target is the value you aim for and the actual value is what the evidence shows. Keeping these separate lets someone else reproduce the figure and compare it across reporting periods.
Should missing follow-up count as failure?
No. An unknown status is neither a placement nor a failure. Report how many participants have a known status, the confirmed share of the whole cohort and the share among those reached, together. Then check whether the unknowns cluster by site or start date, because silence may mean either hardship or a busy new job.
Does an improved outcome prove impact?
No. An improved result shows what changed for the people measured. Attributing it to the program needs an evaluation design that accounts for the labor market, prior experience, other support and who chose to enroll. For most small grants, report the observed result, the plausible contribution and the limits, and reserve causal claims for funded evaluation.