Choose measures that serve your actual work
An employment pathway, coaching program and afterschool service may all follow people over time, but they should not inherit the same definition of success. Begin with the decision the measure will support and the evidence the program can reasonably collect.
For P-24, attendance shows participation. A participant statement describes their experience. A practitioner observation describes something seen in a particular setting. None automatically proves a durable outcome. Define the role of each before combining them in a report.
Include the participant’s own goals where appropriate. A system that only records organizational targets may miss progress that matters to the person, or imply agreement where it has not been established.
How different instruments support a common comparison
Start with a question the participating teams genuinely share. For each required field, define the unit, eligible population, period, response options, missing states and calculation. Record how each local field maps to that definition, who reviewed the mapping and which version applies.
School A records mentoring visits; School B records coaching sessions. Both can use a participant identifier, service episode and dated observation. Compare a progress measure only when its definition and assessment method are compatible; local notes can remain different.
For example, a local field called visits can map to attendance occasions when both count each attendance. It cannot map to unique people without evidence that identifies and removes repeated attendance. If that evidence was never collected, label the unique-person result unavailable rather than asking AI to reconstruct it.
Before combining records, check coverage and overlap as well as definitions. State which teams supplied usable evidence, which are pending and whether the same person or transaction appears in more than one source. Report separate results when populations, methods or permissions make combination inappropriate.
In a Sopact workflow, the shared definitions provide context for analysis while the connected record retains local responses, files and history. Your team can review a source-to-field mapping instead of redesigning every local survey. Test that configuration with sample records before relying on a combined result; a data dictionary cannot supply missing evidence or make incompatible measures equivalent.
Write definitions another colleague can apply
For each pilot measure, record the question or observation, unit, period, source role and method. If a rubric is used, write concrete anchors and examples. A label such as “improving” needs an explanation of the evidence that supports it.
Retain unknown, not applicable and awaiting review as distinct states where relevant. Missing evidence should not become a zero by default. A change in wording, scale or rater may affect a comparison and needs a recorded interpretation decision.
Test two examples with a colleague. If you apply the definition differently, revise the wording or acknowledge the judgment involved. The goal is clearer interpretation, not a false appearance of precision.
Keep claims proportionate to evidence
A later observation can show a difference from an earlier one when the observations are comparable. It does not alone show that the program caused the difference. Other experiences, changes in participation and missing follow-up may matter.
When aggregating, define the eligible population, available records and matched observations. Do not describe a result among respondents as though it represents everyone served. Protect the appropriate level of detail for the reporting audience.
A shared dictionary helps staff and an assistant use the same context. Keep its version attached to analysis and review unclear interpretations. The dictionary supports reasoning; it does not guarantee the accuracy of every generated answer.
Work through the Pathways example
This is a fictional teaching example. Adapt the fields and rules to the question your own program needs to answer.
| Evidence type | What it can describe | What not to assume |
|---|---|---|
| Attendance | Participation at recorded occasions | Participation proves improvement |
| Self-report | The person’s account at that moment | It equals an independently observed outcome |
| Practitioner observation | Recorded behavior in a context | It describes every setting |
| Follow-up outcome | A later result with its source | The program necessarily caused it |
Build this part of your plan in more detail
These lessons address the next practical questions in this module. Choose the detail your workflow needs, then return to complete the exercise.
- Survey Attrition in Longitudinal Studies: Track Missing Waves
Show follow-up coverage alongside reported progress.
- Cross-Program Reporting: Combine Results Without Losing Meaning
Check what can be compared across locations or programs.
More practical questions in this module (7)
- How Do You Measure Change at Exit?
Compare starting and exit evidence while showing who has usable observations.
- How Do You Compare Two Cohorts Fairly?
Check population, timing and coverage before interpreting a difference between groups.
- How to Define Impact Metrics Your Team and Funder Can Use
For funder reporting, test which measures match, need a valid translation or require clarification.
- How Do You Analyze Survey Results by Demographic Subgroup?
Compare relevant groups while checking coverage, changing profiles, uncertainty and disclosure.
- Turn Reporting Requirements into Evidence: A Practical Mapping Guide
When audiences request different reports, map each requirement to an approved measure, source and unresolved gap.
- How Do You Estimate Impact With Thin Data?
When a claim exceeds the evidence, separate known results, missing coverage and planning assumptions before collecting more.
- How to Measure Youth Development Against a Framework
Apply a suitable framework when youth development is your program goal.
Add this part to your plan
On workbook page 5, define two measures appropriate to the fictional program. State how missing evidence and a changed rater affect comparison. Map one local field to a common definition. Give one example that must remain separate.
Download workbook (fillable PDF)Compare with a suggested answer
Define attendance by scheduled occasions and participation status. Define a chosen progress observation with its source and anchors. Keep missing values explicit and note rater changes; do not collapse all evidence into a single unexplained success score.
Self-check: Can another colleague explain what your measures mean and what claims they do not support?
Questions you may have
Is there one progress rubric for all programs?
No. Use a framework appropriate to the purpose and population, with explicit definitions and review. Do not copy customer-specific measures as universal standards.
Does a before-and-after difference prove impact?
It can describe observed change under suitable comparison conditions. A causal claim needs an evaluation approach that addresses other explanations.
Apply the method to your work
Use the five-part plan to assess the sources, analysis and permissions your team needs. The solution page shows where a connected platform can support that workflow.
Explore the case intelligence solution →