To define an impact metric so everyone counts it the same way, write the decision rule behind the label. Specify who is eligible, what event or change qualifies, when it is observed, how the number is calculated, which evidence is acceptable, how exceptions and missing data are treated, and what decision the result should trigger. Then add a neutral qualitative question when the number can locate a problem but cannot explain it. The output is an approved Metric Definition Sheet ready to turn into governed fields and collection rules.
Your next step in the reporting course
Goal: Make one important metric clear enough for two people to calculate consistently.
Start with: Use the measure you selected in the previous lesson. The definition sheet explains eligibility, timing, calculation and missing data.
Carry forward: Leave with one tested metric definition. Carry it into the requirement-to-evidence map so you know what must be collected.
The related lesson aligned measures with funder requirements. This chapter makes those measures operational. A label such as “enrolled,” “participating,” “placed,” or “retained” is not yet a metric definition; reasonable people can count each one differently.
What you will produce
- One approved definition for every shared measure.
- A calculation whose numerator and denominator can be defended.
- A neutral follow-up question for context the number cannot provide.
- A clear handoff into the governed data dictionary.
How do you define a metric in seven steps?
- Start with the decision the metric must inform.
- Name the eligible population — who can enter the denominator.
- Define the qualifying event or change without relying on an ambiguous label.
- Fix the observation point and time window before collecting results.
- Write the calculation and evidence rule, including numerator, denominator, and acceptable source.
- Resolve exceptions and missingness so an unrecorded value is not silently treated as failure.
- Add context and approve the definition with program, data, and decision owners; include the source, stage and intended use so another team can interpret it.
Why can enrollment give a false sense of success?
Enrollment records intention; attendance records initial participation. If young people enroll but do not attend the first session, the program has achieved an administrative output but has not yet converted access into engagement. Treating both as “participants served” hides the earliest actionable gap.
Open Play Foundation creates and activates safe recreational platforms for children in Stellenbosch through community partnerships. Its work combines infrastructure, program support, community coaches, monitoring and evaluation, and continuing joint custodianship. Its published values emphasize empathy, dignity, trust, and listening to communities as full stakeholders. Open Play mission and values · Open Play investments
In a working conversation with Sopact, Open Play CEO Marco Botha described a related measurement problem. Program data was collected diligently, but analysis could arrive only weeks later. When enrolled youth did not appear for the first session, the team could discover the pattern too late to recover their participation — and the count did not explain why they had not come.
The response described in that working account was to reduce the monitoring burden to a small set of decision metrics, analyze the signal promptly, and ask one non-leading question that allowed young people to explain the barrier in their own words.
This operating chain is derived from Open Play’s public mission and the case experience supplied to Sopact; it is not presented as Open Play’s formally published Theory of Change.
Which metric should the program use?
Use first-session attendance rate to see whether confirmed enrollment converted into initial participation. Keep enrollment, continued participation, and outcomes separate because each describes a different stage and supports a different decision.
| Metric | What it answers | What it does not prove |
|---|---|---|
| Confirmed enrollments | How many eligible young people committed before launch? | That they participated |
| First-session attendance rate | Did confirmed enrollment become initial participation? | Why someone did not attend or whether they will remain |
| Continued-participation rate | Was participation sustained to a defined checkpoint? | That the intended participant outcome occurred |
| Outcome measure | Did the expected change occur for the defined population and period? | That the program alone caused the change |
First-session attendance is a leading program indicator, not an impact outcome. That does not make it less valuable. It makes it useful for a different decision: whether to intervene now so intended participants have a fair chance to benefit later.
What belongs in a Metric Definition Sheet?
This definition sheet uses seven practical elements: Decision, Who, What, When, How, Evidence and Exceptions. They are a course checklist, not a universal external standard. If one is missing, two programs can use the same label while producing incompatible numbers.
| Definition element | Question to resolve | Failure if omitted |
|---|---|---|
| Decision | What action, comparison, or judgment will this result inform? | Data is collected without a use |
| Who | Who is eligible to enter the numerator or denominator? | Populations are mixed |
| What | What observable event, condition, or threshold qualifies? | Staff apply different judgments |
| When | When is it observed, and over what window? | Exit, 30-day, and 90-day results are combined |
| How | What is the formula, numerator, denominator, unit, and rounding rule? | Reported percentages cannot be reproduced |
| Evidence | Which source is acceptable, and who records it? | Claims cannot be traced |
| Exceptions | How are cancellations, transfers, duplicates, late entry, and missing records treated? | Absence of data is mistaken for failure |
Worked example: define first-session attendance
Define both the attendance event and the eligible enrollment population. A missing attendance sheet is not evidence that everyone was absent, and an expression of interest is not necessarily a confirmed enrollment.
| Element | Illustrative definition requiring program approval |
|---|---|
| Metric name | First-session attendance rate |
| Decision | Which non-attendees require respectful follow-up, and do recurring barriers require a program change? |
| Who | Young people with confirmed enrollment before the first scheduled session and eligible for that cohort |
| What | Physical or otherwise program-approved presence recorded during the first session |
| When | At the first scheduled session; any grace period for late admission must be stated separately |
| How | Eligible confirmed enrollees attending first session ÷ all eligible confirmed enrollees × 100 |
| Evidence | Dated enrollment record joined to the dated first-session attendance record through a persistent participant ID |
| Exclude | Expressions of interest, duplicate registrations, cancellations recorded before the session, ineligible applicants, and people enrolled after the session |
| Do not classify yet | When the attendance record is missing or incomplete. “Not recorded” is not the same as “did not attend.” |
The definition above is illustrative. Open Play or any program using it must approve the actual attendance event, eligible population, grace period, exceptions, follow-up protocol, consent, and access rules.
Test the attendance calculation with incomplete records
Separate fictional exercise—not Open Play results. Suppose 100 people are eligible for the first session. Seventy have confirmed attendance, 20 have confirmed absence and 10 have unknown status because their records are incomplete.
- Confirmed attendance across all eligible people: 70/100 = 70%.
- Confirmed absence across all eligible people: 20/100 = 20%.
- Unknown status: 10/100 = 10%; status coverage is 90%.
- Attendance among people with known status: 70/90 = 77.8%, if that additional view is useful and clearly labeled.
Calling everyone outside the 70% attendance count a no-show would turn missing evidence into ten unsupported absence classifications. Resolve the records or retain the unknown category. A dashboard can flag the gap immediately, but staff should check the attendance evidence before contacting someone as a confirmed non-attendee.
This is a self-contained attendance exercise. Do not combine its counts with another lesson’s employment or training dataset.
Why add one neutral qualitative question?
The attendance rate tells the team that enrollment did not become participation; it cannot explain the barrier. Ask a neutral, open question before imposing categories so the participant can surface transport, safety, timing, communication, family responsibility, cost, health, work, school, expectations, or an unanticipated reason.
Ask first
“What, if anything, made it difficult for you to attend the first session? Please describe what happened in your own words.”
Do not begin with “Were fees the problem?” or “Which social or economic issue prevented you from attending?” Both insert an untested explanation. No question is completely unbiased: wording, language, interviewer, channel, and circumstance can influence the response. The practical goal is a non-leading question that leaves room for the team to be wrong.
If the first answer is too brief, staff can use optional probes without forcing a category: “Was the difficulty related to timing, transportation, cost, safety, family responsibilities, health, communication, work, school, the program itself, or something else?” Preserve the original response even when themes are added later.
| Evidence type | Permitted use | Do not do |
|---|---|---|
| Participant’s exact words | Retain as source evidence with consent and appropriate access | Rewrite the response to sound more convenient |
| Human-reviewed theme | Group recurring barriers while linking back to source text | Present a theme as a fact the participant stated |
| AI interpretation | Store as a candidate interpretation for human review | Invent a cause or infer a quantitative value |
How does the Theory of Change improve the metric?
A Theory of Change identifies the expected sequence of change and the assumptions that must hold. It helps the team see that enrollment, first attendance, continued engagement, and participant outcomes are different claims. The metric definition then states exactly how one claim will be observed.
A Theory of Change becomes “wallpaper” when it is commissioned for a grant and disconnected from everyday decisions. In this example, its practical value is simple: if enrollment is expected to lead to participation, then the enrollment-to-first-session transition is an assumption worth testing. The open question helps explore why participation did not follow enrollment; an account of a barrier does not by itself establish a causal explanation. Repeated review turns the theory into a learning loop rather than a static diagram.
How should a team approve a metric definition?
Approval requires the people who use, collect, interpret, and act on the measure. Test the definition against real edge cases before governing it.
- Ask the program owner whether the definition reflects how participation actually works.
- Ask field staff whether the evidence can be collected at the specified moment without harming service delivery.
- Ask the data owner whether the numerator, denominator, join key, missing states, and exceptions can be implemented.
- Ask the decision owner what result triggers follow-up and who has authority to act.
- Test cases such as cancellation, late entry, transfer, duplicate enrollment, excused absence, and a missing attendance sheet.
- Record unresolved disagreements instead of hiding them behind one label.
Prompt: draft definitions from an approved metric list
Where does Sopact Sense help?
The method works with a document and spreadsheet. The operational difficulty begins when definitions, enrollment records, attendance, follow-up responses, and reports live in different files or with different people. A configured Sopact workflow can collect attendance updates and follow-up responses against the relevant contact and program record. Apply the approved calculation and review source-linked themes as new information arrives. Test missing-data handling, permissions and calculation rules before relying on the results for follow-up.
People still approve the definition, decide what follow-up is appropriate, review AI-generated themes, protect youth data, and retain final authority. Programs should test theme coding across languages and participant groups, document overrides, minimize personal data, restrict access, set retention rules, and never contact or classify a young person solely because a model inferred a sensitive condition.
Watch: definitions and context work together
Watch the 1-minute 25-second introduction to defining measures once for different reporting frameworks. It explains the principle; it does not certify an exact standards mapping.
Watch the 6-minute 7-second context explainer. If you watched it earlier in the course, use it here to check whether your definition records the source, stage and intended use.
Frequently asked questions
How do you define an impact metric consistently?
Start with the decision, then define who is eligible, what qualifies, when it is observed, how it is calculated, which evidence supports it, and how exceptions and missing data are treated. Test the rule on real edge cases and obtain approval from program, field, data, and decision owners before turning it into governed fields.
What does quantifiable impact mean?
It means an aspect of change can be expressed with a defined measure. A number still needs a population, time period, calculation and evidence source. Quantifying an observed result does not establish that the program caused it, and a number does not replace participants’ accounts of their experiences.
Is enrollment an impact metric?
Usually, enrollment is an output or early participation-stage measure, not an impact outcome. It shows intended reach or commitment. It does not establish that someone attended, remained engaged, experienced the expected change, or benefited because of the program. It can still be useful when the decision concerns recruitment or access.
What is a first-session attendance rate?
It is the percentage of eligible, confirmed enrollees who attend the program’s first scheduled session. The definition must state what counts as confirmed enrollment and attendance, when late admission is allowed, which cancellations are excluded, and how a missing attendance record is treated. Its complement is the first-session no-show rate only when attendance status is complete and both measures use the same eligible population. If some statuses are unknown, show them separately.
Why not call a first-session no-show “dropout”?
Dropout normally suggests that participation began and later stopped. A person who enrolled but never attended has not yet entered active participation. Calling both situations dropout combines two different program problems: conversion from enrollment to attendance and retention after participation starts.
What is a good qualitative question for non-attendance?
Ask: “What, if anything, made it difficult for you to attend the first session? Please describe what happened in your own words.” It does not assume that fees, transport, safety, family responsibilities, or program design caused the absence. Optional probes can follow when the participant needs help elaborating.
Should missing attendance be counted as non-attendance?
No. A missing attendance record describes the evidence system, not the participant. Keep “did not attend” separate from “attendance not recorded,” “record incomplete,” and “not yet reconciled.” Otherwise, data-quality failures will inflate the no-show rate and trigger inappropriate follow-up.
Can AI determine why participants did not attend?
AI can propose themes from participants’ responses, but it cannot establish an unstated cause. Preserve the original words, cite the source response, label themes as coded interpretations, review them across languages and participant groups, and require human approval. Never infer sensitive circumstances or quantitative values that the participant did not provide.
What happens when a metric definition changes?
Do not silently rewrite history. Document what changed and when. If the construct, population, time window, unit, or calculation materially changes, create a new metric or field version and preserve the historical definition. Recalculation may be appropriate only when the method and disclosure allow a valid comparison and an authorized owner approves it.
How is a Metric Definition Sheet different from a data dictionary?
The Metric Definition Sheet resolves the meaning and decision rule for one measure: who, what, when, calculation, evidence, exceptions, and use. A governed data dictionary converts approved definitions into reusable fields with stable IDs, types, allowed values, validation, owners, access classifications, mappings, status, and version history.
Sources and case boundary
- Open Play Foundation — Our Story
- Open Play Foundation — Mission and Values
- Open Play Foundation — Our Impact
- Open Play Foundation — Our Investments
Open Play’s public pages support the organizational context, mission, values, investments, and people-centred measurement approach. The existing lesson attributes the delayed-analysis and first-session non-attendance account to a working conversation with Marco Botha supplied to Sopact by Unmesh Sheth; it is not a published outcome evaluation. Illustrative metric rules in this lesson are not represented as Open Play’s approved operating definitions.
Practice and continue
Complete a definition sheet for one approved measure. Give a colleague a test set containing a duplicate, a late entry and an unknown status. Compare your calculations and resolve any differences before the next reporting period.
Use the governed data dictionary reference to turn definitions into fields. For an evidence-collection handoff, use the reference to turn reporting requirements into collectable evidence.
For the final output, use How to Write an Impact Report and report examples.
By Sopact Academy · Revised September 12, 2026. Public customer sources and the limits of the illustrative definitions are stated above.