5- How to Measure Change at Exit (Not Just Completion)

For: program leads and evaluators who can report that people finished — and then get asked by a funder how much anyone actually changed.

Why: traditional case management records completion — attendance met, credential earned, box checked — so it can describe delivery but can never prove movement, and it never asks whether the movement was yours.

Outcome: an exit wave that closes the pre/post pair — every baseline measure repeated on the same scale and joined by the same ID, change computed per participant the moment the response lands, attribution captured with one honest counterfactual, and a one-page exit report written for every graduate on arrival.

This is Chapter 5 of the Case Intelligence series — the post wave. In Chapter 4 you caught the people slipping mid-program, while the intervention still cost a phone call. Now the cohort is finishing, and exit is where the pre/post story the baseline opened and the midpoint tracked either closes or falls apart.

The running example stays the same: RiseWorks Foundation / Pathways 2027 (Train → Match → Place → Earn). RiseWorks can report the countable part without any of this: 80 enrolled → 62 completed → 58 credentialed. This chapter is about the numbers underneath the count: confidence 4.3 → 7.1 → 7.4 on the same 1–10 scale across intake, mid, and exit; a wage pair the exit hands to follow-up, paying off as median $9.96 → $25.11; and the comparison that shows completion was worth measuring at all — 82% of completers employed at follow-up versus 45% of non-completers.

As in every chapter, each step is tagged [DIY] or [SENSE]: designing the exit survey and writing the attribution question is thinking work you can do in any AI today; computing change per participant on arrival and building the cohort story from per-person joins is what the product does over the store — because change is a two-record fact, and a chat window has never seen the first record.

Completion is not change

Traditional case management is built to record that someone completed: sessions attended, a credential logged, a status set to “closed — successful.” That is a real fact and worth having. It is also exactly where a funder’s two hardest questions begin, and the software was never designed to answer either one:

How much did each person actually change? A credential proves attendance and a passed assessment. It does not prove movement. Two graduates can both finish — one arrives job-ready with confidence transformed, the other leaves precisely where they started — and a completion count scores them identically. “58 credentialed” is silent about both of them.

Was the change yours? Some participants would have improved without you — found the job anyway, built the confidence anyway. A funder paying for impact wants the change the program caused, not the change that happened to occur nearby. That question has to be asked at exit, of the participant, in a form that can be answered honestly. Case management never asks it.

The failure is structural, and it is the difference this series keeps returning to. Traditional CM platforms — the big case-management and grants suites — treat exit as one more form: collect it, export it, and hand a spreadsheet to whoever writes the report. Collect now, clean later, compare never — because the exit lives in one export and the intake in another, and nothing joins them but a weekend of VLOOKUP. Sopact Sense inverts that in three ways: the exit response is analyzed on arrival, not at report time; the analysis is adaptive — tune the attribution rubric on the first five or ten exits and every record re-scores; and the stores are centralized without being merged — Exit/Completion is one of eight stores joined by reference keys, and it joins Application/Intake on the persistent email ID, so every exit row knows its own baseline.

That join is the whole chapter. Exit is not a fresh survey; it is the post half of the pairs you opened at intake (Chapter 3) and checked at mid (Chapter 4). Every baseline measure reappears on the same scale, same wording, same ID — which is what lets change be computed per person, not as a cohort average that hides who never moved. And it sets the standard the rest of this article applies: a completer with no measured change is a flag, not a success. That participant is the one a checkbox system quietly counts as a win and a funder audit finds.

Step 1 — Repeat every baseline measure, same scale, same ID [DIY]

What you do. Take your intake form and build the exit from it, not alongside it. Every baseline measure reappears byte-for-byte: same wording, same scale, joined by the same persistent ID. Then add the completion facts the later chapters need. Nothing else earns a place.

Real example — RiseWorks. The exit carries the confidence question — the identical 1–10 item from intake and mid, the one that produces 4.3 → 7.1 → 7.4 — plus the 0–100 skills assessment administered at both ends of the program, and the completion facts: credential_earned, credential_name, skills_assessment_0to100, placement_ready (Yes/No). The readiness fields are not decoration; the matching and placement chapters read them later, through the same reference keys.

Why the discipline matters. A measure you reworded cannot be compared; a measure you rescaled is two orphans, not a pair. Chapter 3’s audit found the live example: RiseWorks’ 1–5 skill self-rating from intake was never re-asked at exit, so “self-assessed skills improved” is a claim the program permanently cannot make. That hole was caught at design time and cost one line to note. The same hole discovered at report time costs a year of unprovable claims. The rule for the exit designer is symmetrical: repeat every baseline measure exactly, and add no new outcome scale at exit that has no baseline — a post number without a pre is a dot with no line.

One more design decision belongs here: name where each pair pays off. Confidence closes at exit. The wage pair does not — baseline $9.96 meets its partner at six-month follow-up ($25.11), which means exit must capture consent and a contact channel, or the pair dies with the program.

Prompt 1: Exit Survey Designer

You are an evaluation designer. Your job is to build an EXIT (post-wave)
survey from my intake/baseline form so that change is computable per
participant. You audit for pairing, you do not invent.

TASK
From the intake form pasted below, produce the exit survey as a field
list, pairing every exit field to its intake partner.

Build it to these rules:
1. REPEAT every baseline measure byte-for-byte — same wording, same
  scale as intake — so each exit value pairs with that person's own
  baseline, joined by the persistent ID (email).
2. Add ONE counterfactual attribution question (I will design its
  wording with a separate prompt; reserve the slot).
3. Add the completion facts: credential_earned (Y/N), credential_name,
  skills_assessment_0to100 (only if the same assessment ran at
  intake), placement_ready (Y/N).
4. Add consent + preferred contact channel for the follow-up wave —
  any pair that pays off after exit (e.g. wage) dies without it.

INPUTS — paste between the markers:
<<< INTAKE / BASELINE FORM (field names, scales, exact wording) >>>

<<< LATER WAVES ALREADY PLANNED, IF ANY (e.g. six-month follow-up) >>>

OUTPUT FORMAT — one table, then a gap list:
Field | Type / scale | Pairs with (intake field) | Status | Purpose

Status values (fixed — use no others):
- PAIRED   — repeated identically from intake; change is computable.
- DRIFTED  — same construct, different scale or wording; include the
            one-line fix (restore the intake version).
- ORPHANED — an exit measure with no intake partner; keep only if it
            is a completion fact or feeds a later wave, and say which.
- CONTEXT  — completion facts and logistics; no pair expected.

Then: GAPS — every intake measure that does NOT reappear at exit,
each with one line on what claim becomes unprovable if it stays
unpaired.

RULES
- A measure you reworded cannot be compared. Flag it DRIFTED; never
 silently "improve" wording.
- Add NO new outcome scale at exit that has no intake baseline — a
 post number without a pre is a dot with no line.
- Do not invent fields that are not in my form or my listed waves.
- Deterministic: the same input must give the same output every run.
 No commentary beyond the table and the gap list.

Step 2 — Ask whether it would have happened anyway [DIY]

What you do. Write one counterfactual question — the attribution item most programs skip and funders most want. Its job is to separate program-caused change from change that was coming anyway.

The design problem is politeness. Ask a grateful graduate “was the program helpful?” and everyone says yes; you have measured gratitude, not attribution. The question has to pose the counterfactual in plain language — where would you be on this today if you had not been in the program? — with a response scale a participant can answer honestly: Entirely because of the program / Mostly the program / Partly the program / Would have happened anyway, plus a short open “why” so the reasoning arrives in the participant’s own words.

Real example — RiseWorks. The open “why” is where attribution strength actually lives. When a graduate writes “No way I’d have gotten certified on my own — I didn’t even know AWS D1.1 was the cert employers wanted,” that is a participant explicitly rejecting the would-have-happened-anyway alternative, with a mechanism attached (information the program supplied, a mentor who kept him enrolled). That is codeable evidence. “Great program, amazing staff” is not — it is praise with no counterfactual content, and an honest coding rule grades it as weak attribution no matter how warm it sounds.

Why it matters. Attribution is the difference between reporting outcomes and claiming impact. With the counterfactual captured per person, the cohort claim can be split honestly — changed and attributable versus changed-but-would-have-anyway — which is exactly the split a skeptical funder runs in their head anyway. Better to run it yourself, on evidence.

Prompt 2: Attribution Question + Coding Rule

You are an evaluation designer. Write ONE counterfactual attribution
question for my exit survey, plus the mechanical rule for coding the
answers. The question's job is to separate program-CAUSED change from
change that would have happened anyway.

INPUTS — paste between the markers:
<<< MY PROGRAM, IN ONE OR TWO LINES (who it serves, what it delivers) >>>

<<< THE MAIN OUTCOME THE EXIT MEASURES (e.g. confidence, credential, wage) >>>

REQUIREMENTS FOR THE QUESTION
1. Pose the counterfactual in plain language — where would the
  participant be on this outcome today WITHOUT the program — not
  "was the program helpful" (everyone says yes; that measures
  gratitude, not attribution).
2. Phrase it so a grateful participant can answer honestly. No
  leading wording that fishes for credit.
3. Response scale (fixed — use exactly these four):
  Entirely because of the program / Mostly the program /
  Partly the program / Would have happened anyway
4. Pair it with a short open-text "why", so attribution strength can
  be read from the participant's own reasoning.

CODING RULE FOR THE OPEN TEXT (fixed categories — mechanical, so the
same answer gets the same code every run):
- STRONG   — only when the text explicitly rejects the would-have-
            happened-anyway alternative with quotable words AND names
            a mechanism the program supplied (information, mentor,
            credential, access). Quote the words.
- MODERATE — program named as the main of several stated factors.
            Quote the words.
- WEAK     — praise or thanks with no counterfactual content ("great
            program, amazing staff"). Warmth is not evidence.
- NONE     — the text says or implies the change was coming anyway.
- MISSING  — no open text provided.
Never upgrade a code because the scale answer was generous; the open
text governs. Never invent a quote.

OUTPUT
1. The question wording.
2. The four-option response scale.
3. The open-text "why" wording.
4. The coding rule as a reusable instruction block.
Nothing else. Deterministic: same input, same output, every run.

A sample exit record

One RiseWorks graduate at exit, exactly as the fields arrive — Marcus D., 24, Skilled Construction Trades (welding), the same participant whose transport barrier surfaced at intake and whose confidence dipped mid-program:

email — marcus.d@… (the persistent ID — the same key on his intake, mid, and mentor-note records)

Completion facts: credential_earned Yes · credential_name AWS D1.1 Welding · placement_ready Yes

Repeated measures: confidence_1to10 — 7 (his intake baseline was 4; his mid reading dipped to 3) · skills_assessment_0to100 — 78 (intake: 40)

counterfactual_openended — “No way I’d have gotten certified on my own — I didn’t even know AWS D1.1 was the cert employers wanted. The mentor pushing me through the bus mess is the only reason I finished.”

Read that record the way a checkbox system would: credential Yes, file closed, one more tick toward “62 completed.” Now read it the way the joins allow: a confidence line that runs 4 → 3 → 7 — the mid-program dip Chapter 4 caught, recovered and passed; a skills assessment nearly doubled against his own baseline; and a counterfactual that names, in his own words, the exact intervention (a mentor, a transport fix) that kept the record from ending at the dip. None of that is visible in the completion count. All of it is in the record — if the record can see its own history.

Step 3 — The one-page exit report, written on arrival [SENSE]

From here, this is product output, not a prompt you run. The moment Marcus’s exit lands, an Intelligent Cell on each repeated measure joins back to his intake row by email and computes change from his baseline — never the cohort’s. A second Cell reads counterfactual_openended and rates attribution strength from what was actually said. An Intelligent Row assembles the result into a one-page Exit Report per participant. Here is Marcus’s, as Sense produces it on arrival:

INTELLIGENT ROW — Exit Report · Marcus D. · Skilled Construction Trades · exit

Notice what a standalone prompt could never have produced: every number in that row is a two-record fact. The +3 exists only because the exit row found the intake row; the “recovered” exists only because the mid row sits between them. A one-off prompt in a chat window has no store, no other waves, and no join key — it can summarize Marcus’s exit, but it cannot know he started at 4. And the flag cuts the other way too: a completer whose repeated measures show no movement — confidence flat, skills flat — gets a Row graded UNPROVEN, not a quiet tick in the completion column.

Same setup tip as earlier chapters: in the exit store’s table view, move the Row columns to the front — change, attribution, readiness first; raw answers after. Whoever reviews graduates works from the leftmost columns.

Prompt 3 — Change + Attribution on Arrival

WHERE IT SITS
Exit/Completion store. Two Intelligent Cells + one Intelligent Row,
firing on every exit response the moment it lands. The store joins
Application/Intake (and later, Six-Month Follow-up) on the persistent
email ID.

CELL A — CHANGE FROM BASELINE
Sits on: each repeated measure (confidence_1to10,
skills_assessment_0to100).
Instruction text of the Cell:
 "Join this response to the participant's intake record by email.
  Report before → after and the delta for this measure, computed
  against THAT PERSON'S baseline — never a cohort average. If a mid
  reading exists, include it as the middle point. If no intake value
  is found, return UNPAIRED (do not estimate a baseline)."

CELL B — ATTRIBUTION STRENGTH
Sits on: counterfactual_openended.
Instruction text of the Cell:
 "Code attribution from what was said, using the fixed rule:
  STRONG only when the text explicitly rejects the would-have-
  happened-anyway alternative AND names a program mechanism — quote
  the exact words. MODERATE when the program is named as the main of
  several factors. WEAK when the text is praise with no
  counterfactual content. NONE when the change is described as
  coming anyway. MISSING when blank. Never infer attribution from
  politeness or tone; the quote governs."

ROW — EXIT REPORT (one page per graduate)
Assembly spec: for each exit response, assemble
 per-measure change (Cell A, all repeated measures)
 + attribution code with its evidence quote (Cell B)
 + placement-readiness (credential_earned + skills + confidence
   recovered → READY / NOT-READY)
 + grade: EVIDENCED when movement shows on the repeated measures
   with a quotable basis; UNPROVEN when the participant completed
   but the repeated measures are flat — the completer a checkbox
   would silently count as a win.

SAMPLE RETURN — real record (Marcus D., Skilled Construction Trades):
 Confidence (1–10):        4 → 7 (+3), mid dip to 3 recovered
 Skills (0–100):           40 → 78 (+38)
 Attribution:              STRONG — "No way I'd have gotten certified
                           on my own… the mentor pushing me through
                           the bus mess is the only reason I finished"
 Placement-readiness:      READY (AWS D1.1 · skills 78 · confidence
                           recovered)
 Grade:                    EVIDENCED

WHY A STANDALONE PROMPT CAN'T DO THIS
Change is a two-record fact. A chat window holds the exit response
you paste into it — it has no store, no intake row to join, no mid
reading in between, and no way to run on all 62 completions as they
arrive or to re-score every record when you tune the rubric on the
first ten.

Step 4 — The cohort story, built from per-person joins [SENSE]

One Exit Report is a graduate’s story. The cohort’s story is what the funder reads — and the only honest way to build it is from the per-person joins, not from averages. The Sopact Assistant computes it across the exit and intake stores (and, for the wage pair, the six-month follow-up store), joined throughout by email:

The last two rows are the ones traditional reporting cannot produce. 82% versus 45% only exists because completers and non-completers alike stayed joined on the same ID all the way to follow-up — a program that stops tracking at the certificate ceremony has no comparison group and no way to show the credential did anything. And the UNPROVEN list is what makes the headline defensible: a cohort average can be arithmetically true while a slice of graduates moved zero, and an average quietly launders those non-movers into the win column. Because the Assistant builds 4.3 → 7.4 from every participant’s own before-and-after, the non-movers surface as named flags with their values attached — so the number you publish is one you can defend line by line, which is the only kind a skeptical funder accepts.

This is also why the step is [SENSE] and not a prompt: the computation spans three stores and every record in each. There is nothing to paste into a chat window, because the input is the workspace itself.

Prompt 4 — Cohort Change + Completion-Without-Change Flag

ASSISTANT PROMPT A — the cohort story from per-person joins

 "Across the Exit/Completion store, join every exit to its intake by
  email. For each repeated measure, compute change per participant
  against their own baseline, then report the cohort movement built
  from those per-person pairs — never from a raw average of exit
  scores. Include the funnel context (enrolled → completed →
  credentialed)."

WHAT IT RETURNS (RiseWorks / Pathways 2027):
 Funnel:      80 enrolled → 62 completed → 58 credentialed
 Confidence:  4.3 → 7.1 → 7.4 (intake → mid → exit; same 1–10 scale,
              same wording, same people, joined by email)
 Wage pair:   $9.96 (intake median) → $25.11 at six-month follow-up
              — exit hands this pair to the follow-up store

ASSISTANT PROMPT B — the flag a checkbox hides

 "List every participant who COMPLETED but shows no measured change
  from baseline on the repeated measures (confidence flat, skills
  flat). Grade each UNPROVEN and show the evidence: before/after
  values and any counterfactual text. Then split the cohort:
  completed + changed + attributable (the defensible wins) versus
  completed with no movement (the checkbox illusions)."

WHAT IT RETURNS:
 A flagged list — Participant | Before → After per measure |
 Attribution | Grade — plus the honest split. A cohort average can
 be arithmetically true while a slice of graduates moved zero; the
 per-person join surfaces those non-movers as named flags, so the
 headline number can be defended line by line.

ASSISTANT PROMPT C — completion versus change, proven

 "Across Exit/Completion joined to Six-Month Follow-up by email,
  compare employment outcomes for completers versus non-completers."

WHAT IT RETURNS (RiseWorks):
 Employment at follow-up: 82% of completers vs 45% of
 non-completers — the comparison that shows completion was worth
 measuring, and it only exists because non-completers stayed joined
 on the same persistent ID after they left.

WHY A STANDALONE PROMPT CAN'T DO THIS
The computation spans three stores — intake, exit, follow-up — and
every record in each. There is nothing to paste into a chat window,
because the input is the workspace itself.

Common mistakes (and what to do instead)

Reporting completion as impact. “62 completed, 58 credentialed” describes delivery. It says nothing about movement and nothing about cause. Keep the completion count — funders want it — but never let it stand where the change number should be.

Rewording or rescaling the exit questions. An exit item that “improved” the intake wording is an orphan, not a pair. RiseWorks’ unre-asked 1–5 skill rating is the standing example of what drift costs. Repeat the baseline measures exactly, or accept that the pair is dead.

Skipping the counterfactual because everyone loves the program. Gratitude is not attribution — that is precisely why the question is needed. Pose the without-us alternative plainly, give a scale that permits “would have happened anyway,” and code strength from the open “why,” not from warmth.

Publishing the cohort average without the per-person joins. A mean can be true and misleading at once. Build the aggregate from each person’s own before-and-after, and surface the non-movers as flags. If your average cannot survive its own line-by-line, it is not a number to publish.

Treating exit as the last contact. The wage pair — $9.96 → $25.11 — and the 82%-vs-45% comparison both live at six-month follow-up. An exit that fails to capture consent and a reliable contact channel closes the file and the evidence with it.

What you have now

An exit survey built from the intake form, with every baseline measure repeated on the same scale and joined by the same ID. One honest counterfactual question with a mechanical coding rule. An Exit Report per graduate, written on arrival — change from their own baseline, attribution strength with the quote that earned it, placement-readiness for the chapters ahead, and an UNPROVEN grade for any completer who didn’t move. And a cohort story assembled from per-person joins — 4.3 → 7.4, $9.96 → $25.11, 82% versus 45% — that can be defended record by record.

Why this works

Because the exit repeats the baseline byte-for-byte, change is computable at all. Because every wave hangs on the same persistent ID, change is computable per person — a trajectory with a name, not a mean with a margin. Because the counterfactual is captured at exit, the claim survives the funder’s hardest follow-up: was it you? And because the analysis runs on arrival, the completer who didn’t move is a flag in this cohort — someone to understand and follow up with — instead of an embarrassment in next year’s audit. The skeptic’s one-liner: completion counts what finished; a closed pair plus a counterfactual proves what changed and whose change it was.

The one thing to do this week

Open your exit survey next to your intake form and check one thing: does every intake measure reappear at exit, same scale, same wording? Fix the drifted ones, and add one question — “Where would you be on this today without the program?” with a scale that includes “would have happened anyway.” That single edit turns your next cohort’s exit from a completion record into a pre/post claim.

Video walkthrough (coming soon)

A screen-by-screen walkthrough — pairing the exit to the intake form, watching Marcus’s change and attribution compute on arrival, and pulling the cohort story with the UNPROVEN flags from the Assistant — is in production. Check back on the Academy.

Who this is for

Program leads whose annual report says “graduated” where the funder wants “changed.” Evaluators staring at an exit export and an intake export that share no key. Anyone who has published a cohort average and hoped nobody asked about the people inside it. If your program can prove attendance but not movement — and cannot say whether the movement was yours — the fix starts here.

Close your pre/post pair in Sopact Sense — sopact.com/academy.

Next in the series: How to Catch At-Risk Participants Early with Mentor Notes — exit measures the people you kept; the weekly mentor notes are where you keep them, catching the slide as it forms, weeks before the midpoint confirms it.