Additional course links
This lesson is for program and evaluation teams planning evidence across afterschool, mentoring, coaching or youth services. You will produce a small assessment plan: selected domains, suitable evidence, collection moments, review responsibilities and reporting limits. Start with one relevant domain and a fictional record before extending the plan.
Choose the decision before choosing the domains
Decide what the assessment will help someone do. A practitioner preparing a conversation, a young person reviewing a goal and a funder receiving aggregate results need different levels of evidence and detail. Write the decision and audience first. Do not collect every possible domain merely because a framework lists it.
Check whether the framework fits the age range, setting, language and intended use. If it includes a standardized instrument, follow its administration and scoring requirements and check permissions to use it. A locally written observation guide can support structured discussion, but should not be presented as a validated developmental scale.
Include the young person’s own priorities where appropriate. An organizational goal does not establish that the participant shares it. Keep their account distinct from a practitioner’s interpretation or a family member’s description.
Define what would count as evidence
For each selected domain, specify the behavior or experience of interest, who can describe it, the context in which it can be observed and the collection moment. Retain relevant narrative alongside the structured fields. A short quote may support an interpretation, but the number of quotes does not determine whether a rating is valid.
If the method uses rating anchors, define them from the chosen method. Do not automatically apply emerging, developing and demonstrated to every framework. Record not observed, not asked and insufficient evidence when those states apply; none is a low developmental score.
A participant might not have had an opportunity to demonstrate a behavior. Record that limitation rather than scoring the absence of an opportunity as inability. Likewise, a blank domain may be outside the agreed scope, not a collection failure.
Work through a fictional observation
A youth program wants to help participants ask for support during group work. Participant P-24 chooses this as a goal. During one session, a facilitator records that P-24 asked for help clarifying a task. At a later check-in, P-24 says asking is still difficult when working with unfamiliar people.
| Evidence | What it supports | What remains open |
|---|---|---|
| Dated facilitator note: a request for help during one task | The behavior was observed in that setting | How often it happens, and in other settings |
| Participant describes difficulty with unfamiliar people | The participant’s account of a contextual barrier | What support they want and what a later opportunity shows |
| No suitable intake observation | No measured starting comparison is available | Whether a retrospective account is useful, labeled separately |
A supportable statement is: “P-24 asked for help during the observed group task and described continued difficulty with unfamiliar people.” It is not “P-24 has demonstrated help-seeking across settings” or “the program caused growth since intake.” Agree the next appropriate opportunity and ask what support P-24 would find useful.
Connect observations without forcing one instrument across sites
Give each observation a participant reference where appropriate, program episode, source role, date and method version. Reuse suitable registration details instead of asking for them at every check-in. Refresh changing information with an effective date so a later school or location change does not rewrite the earlier context.
Several schools may use different local questions. Agree only the small shared core needed for a common decision. A dictionary can document domain meaning, compatible methods, populations and mapping decisions while preserving useful local notes. Two schools using the word confidence do not necessarily have a common measure.
Compare a participant over time only when the observations support that comparison. State a changed rater, setting or instrument. Without a baseline, report the current evidence; do not ask an assistant to invent an earlier score. At group level, show who supplied usable observations and who is missing.
Use AI to prepare a review, not decide a young person’s development
Configured analysis can help locate relevant passages, organize them under the selected domains and identify missing context. Give it the authorized sources and applicable definitions. Require it to distinguish observations, participant statements and interpretations.
Review original notes and any transcript uncertainty. Reject invented quotations and unsupported conclusions. A source citation lets a person inspect the basis of a statement; it does not certify the assessment. Consequential support decisions belong to the responsible people and the applicable professional process.
Do not treat sentiment, word choice or a missing note as an automatic diagnosis, risk classification or evidence of motivation. A concern requiring a response follows the organization’s established human procedure, not a wait for the next aggregate report.
Return findings that help the participant and team
Separate the working record from the participant summary and any funder report. A useful participant conversation includes what was observed, their own perspective and the next agreed action. A program report may describe patterns with coverage and limitations, without exposing private notes.
Assign responsibility for corrections and for checking whether the next action occurred. Record whether the participant found it helpful separately from whether staff completed a task. Test source links and exported views so a limited summary does not unintentionally expose the whole record.
Practice: build one domain assessment plan
- Name the decision, audience and one relevant domain.
- Specify the evidence method, source, setting and collection moment.
- Record what counts as sufficient evidence under that method and what remains unknown.
- Write a current-evidence statement using the fictional example.
- Define the next collection opportunity, owner and appropriate sharing.
- If multiple sites are involved, identify the shared fields and one local measure that must remain separate.
Check your work: another colleague should be able to explain what the observation supports without inventing a score or an earlier baseline. The participant’s voice should remain recognizable and the next action should have an owner.
Frequently asked questions
Does every framework domain need a rating?
No. Select domains for the purpose and use the chosen method appropriately. A domain outside the assessment scope should not be scored merely to fill a dashboard.
Do two quotations make a rating reliable?
No. Reliability depends on the method, evidence quality, interpretation and context. Several weak or repeated quotations do not establish a valid developmental rating.
Can different schools compare results?
They can compare selected measures when definitions, methods and populations support the comparison. Preserve local instruments and document mappings; report incompatible observations separately.
Can we show progress without a baseline?
You can describe current evidence and explicitly labeled accounts of perceived change. Do not present an unobserved starting score as measured progress.