Watch · 11:08–13:19 · reports grantees already keep, checked on arrival
Unmesh describes the reporting half of the cycle: agree the metrics on the onboarding call, then ask grantees to update the social audit or financial documents they already keep instead of keying in numbers. A rubric checks what is complete, and the grantee's unique ID lets you go back for the missing piece. The closing section lists the portfolio context you start building here. Watch on YouTube ↗
Jump to: 4:49 — why dashboards cannot answer · 12:43 — data dictionary, requirements, timelines, red flags
The first update is in. The board's question is still open.
It is the Monday after Youth Pathways Q1 reports were due. Eastgate Youth Works, A-31, sent its update on time. It says 180 young people registered, and 108 of 150 eligible participants attended at least three of the first four sessions. It also notes that $2,000 moved from materials to transport stipends, which Horizon approved in Q1.
Your board chair has asked: "What did R-2 fund, and is it working?" Finance wants spend against budget for each grantee. Neither answer comes out of Eastgate's numbers as they stand. The update reports registrations; the award promised 200 enrolled youth; nobody has written down what "enrolled" means.
This is where Unmesh says dashboards disappoint. The data behind the Monday question was never collected, the explanation sits in a narrative the chart cannot read, and each grantee counts differently. So the answer leaves by export and gets rebuilt in a spreadsheet.
What the configured setup makes hard at this step
In Horizon's main system, each grant type has had a report template with metric boxes for eight years. The definitions live in the program officer's head or in last year's spreadsheet.
Grantees re-key metrics they already track elsewhere and attach a narrative PDF that waits to be read. Gaps are chased by email. The board summary is assembled in Excel, and the sources stay behind.
The reporting plan defines each measure. Eastgate uploads the program report and budget workbook it already keeps. Each upload is read against the definitions on arrival, a rubric flags what is missing, and one request goes back on the same record.
The pilot leaves the main system's templates alone. It tests whether one round's reports can arrive, be checked and be answered from without re-keying.
Freeze and version the round's decision rules
When the board asks what R-2 funded, the honest answer includes the rules that selected those grantees. Record the version that applied: the four eligibility gates, the rubric and its anchors, the clarification policy, conflict handling and decision authority.
R-2 has three entries to keep with that version: the feasibility anchor clarified in calibration after reviewers split on A-31's employer letter, A-44's single clarification request, and A-17's reassignment because R3 sits on its board. Each assessment carries the rule version it used, so an auditor can see why A-31 scored 70/100 without asking the reviewers.
If a correction is needed mid-round, document it and name the reviews to reconsider. The same discipline applies to reporting: if "enrolled" changes in Q3, the Q1 and Q2 figures keep the definition they were reported under.
Define each measure before the first number arrives
A reporting measure is ready when two people would count the same thing. That takes five parts: the unit, the period, the population, the source and the owner. Write them down on the record for each award, starting from the reporting plan you agreed on the onboarding call in Lesson 3.
Eastgate agreed four measures: unique youth enrolled (a target of 200 over the year), attendance, job placements at 90 days and budget against actual. Here are the first two, written the way Horizon wrote them.
Horizon example · fictional
| Measure | Unit and population | Period and source | Owner |
|---|---|---|---|
| Unique youth enrolled | Distinct young people, counted once per grant year, who are confirmed eligible (ages 16–24, Lake County) and admitted to a cohort. Registrations alone do not count. | Cumulative for the grant year, reported quarterly. Eastgate's participant roster in its program report. | Eastgate program lead reports; Horizon grants manager reviews |
| Attendance | Share of eligible participants attending at least 3 of their cohort's first 4 sessions. Report numerator and denominator. | Each quarter, for cohorts that have reached session 4. Eastgate's attendance register. | Eastgate program lead reports; Horizon grants manager reviews |
Now read the Q1 update against those rows. Attendance works: 108 ÷ 150 = 72% of eligible participants. Divide by all 180 registrations and you get 60%, which describes a different group and should not be reported as attendance.
Enrollment does not work yet. The update reports 180 registrations, which is not the agreed measure. If the 150 eligible participants were all admitted to a cohort, enrollment stands at 150 against the 200 target, but the report does not say so. The other 30 registrations need a status: not yet eligible, not started, withdrawn or unknown. Unknown is a status to report, not a zero and not a withdrawal.
Placements at 90 days have a period problem. A 12-week cohort finishing near the end of Q1 cannot show a 90-day placement until later in the year. The dictionary says when the measure first becomes due, so an empty Q1 cell reads as "not yet due", not zero placements.
Keep budgeted, committed and spent apart
Eastgate's award is $24,000. In Q1, Horizon approved moving $2,000 from materials to transport stipends, with the reason recorded. The total is unchanged, but the budget now has a second version, and Q1 spending is compared with the version in force when it was spent.
Keep budgeted, committed and spent as separate figures with a stated basis and period. Horizon's payment to Eastgate is a fourth fact: it says nothing about how Eastgate spent the money. Finance remains the system of record, and the dictionary names the report each figure comes from.
If Eastgate's workbook and Finance's figures do not agree, or you are unsure which total counts as "actual", work through the budget and actual spend deep dive, then come back.
Share a small common core, keep award-specific evidence
R-2 funded about ten organizations. The board question needs a few measures that mean the same thing across all of them: unique youth enrolled, attendance, placements at 90 days and budget against actual. Every R-2 award uses the same dictionary rows for that common core.
Everything else can stay specific to the award. Eastgate's condition, confirming the employer placement role before the second payment, is evidence only A-31 needs; it sits on A-31's record with a due point. Define award-specific measures with the same care and leave them out of cross-grantee totals.
Different metrics across grantees are not the problem the iceberg describes. Undefined ones are. One universal outcome forced on everyone produces a number nobody can explain.
Ask for the report they already keep, then read it on arrival
Eastgate already keeps a program report for its board and a budget workbook for its accountant. Copying figures into Horizon's boxes adds work and errors. In the pilot, Eastgate uploads both to A-31's record with a short note on what changed.
Each upload is read on arrival against the dictionary. In Sopact Sense that is an Intelligence Cell, a prompt you configure. Write the prompt with the care of a rubric anchor.
Read the uploaded quarterly program report and budget workbook for A-31. For each measure in the attached data dictionary, return the value as reported, the unit, period and population the report uses, and the page or cell it came from. If the report uses a different unit or population than the dictionary, say so and do not convert it. If a measure is absent, return "missing". If the dictionary says it is not yet due, return "not yet due". Never return zero for a value the report does not state. Then quote any explanation the narrative gives for a difference from plan, with its location.
Then a completeness rubric checks the result. It is a short list of pass-or-flag rules written in advance, the reporting equivalent of your review rubric. Horizon's has six checks.
| Check | Passes when | Flags when |
|---|---|---|
| 1. Coverage | Every dictionary measure is reported or marked not yet due | A due measure is missing |
| 2. Definition | Unit and population match the dictionary | A different unit is reported in its place |
| 3. Denominator | Every rate shows numerator and denominator | A percentage arrives alone |
| 4. Status | Every registration or participant has a status | Records are unaccounted for |
| 5. Budget version | Actuals are compared with the approved version in force | An old or unapproved budget is used |
| 6. Conditions | Each award condition has evidence or a due point | A condition has neither |
Arrival check · A-31 · Q1 · fictional
Coverage passes: a figure is offered for each measure, and placements are marked not yet due. Definition flags: 180 registrations reported where the measure is unique youth enrolled. Denominator passes: 108 of 150. Status flags: 30 registrations have no status. Budget version passes: Q1 actuals compared with version 2 after the approved $2,000 reallocation. Conditions pass with a due point: employer placement role to be confirmed before the second payment.
The grants manager opens the cited page, confirms each flag and decides what to ask. The check does not approve the report.
Send one clarification request, not a new report
Two flags, one request. Every document, reading and flag sits on A-31's record under the same persistent ID, so the request can name exactly what is missing without anyone rebuilding the context from email.
For Eastgate, it asks whether the 150 eligible participants are enrolled under the agreed definition, and for a status on each of the other 30 registrations. It does not ask for a resubmission. The original upload stays as submitted, and the answer attaches to the same record with its date.
It is the pattern Horizon used for A-44's missing letter: one request, a stated window, and a visible "clarification requested" status until the answer is checked.
Reuse verified evidence for Horizon's own reporting
Horizon has obligations of its own: board reporting on R-2, its annual return and what its auditor asks for. Those reports should draw on the evidence you have already defined and checked, not a new round of collection. The auditor asking for proof of the A-31 decision wants records you already hold.
A board report and a tax filing can define "grants" differently, so reuse needs a map. When you reach that step, work through the compliance reports deep dive, which shows how to explain two different totals instead of forcing them to match.
Build the context that later powers comparisons
At 12:43 Unmesh names portfolio context: a data dictionary, reporting requirements, timelines and red flags. You have started all four. The dictionary is versioned. The requirements say quarterly, placements due later. The timeline records when the update arrived and when the request was answered. The red flags are 30 registrations without a status and the placement-role condition.
Held on each grantee's record, that context lets a later question compare like with like. By Q3, "which R-2 grantees are behind their enrollment pace" is a question for the AI Assistant, checked against cited reports, because every grantee counted enrollment the same way.
Keep the limits in view. Software reads the reports; staff confirm what it found. A 72% attendance rate describes participation, not whether the program works. Placement at 90 days is closer to the board's question, and a claim that the grant caused a placement needs an evaluation design this reporting plan does not have.
In Sopact Sense, the award, dictionary, uploads, Intelligence Cell readings and requests share one record per grantee, and Claude or ChatGPT can query the same data through MCP. See the applications and grants solution page.
Put it into practice
Write A-31's dictionary, rubric and request
- Open step 4 of the workbook and
eastgate-q1-update.csv. - Write a dictionary row for each of A-31's four measures: unit, period, population, source and owner. Use the two rows above as a pattern for placements and budget against actual.
- Mark each measure as common core or award-specific, and add the placement-role condition as award-specific evidence with its due point.
- Run the six completeness checks on the Q1 update. Record each result as pass or flag, with the row or field it came from.
- Write one clarification request to Eastgate on A-31 covering every flag. Name the items, give the definition and say that the original report stays as submitted.
- Write one sentence the board chair could read today about Eastgate's Q1, with what it can and cannot claim.
Download the workbook (PDF)
Practice data: horizon-r2-applications.csv · horizon-rubric.csv · calibration-scores.csv · eastgate-q1-update.csv
Check your reasoning before moving on
Placements at 90 days: distinct eligible participants in paid work 90 days after completing the 12-week program, reported against the eligible cohort with the number of known statuses beside it; first due after Q1; source is Eastgate's follow-up log with employer confirmation. Budget against actual: US dollars by category against the version in force (version 2), quarter and year to date; source is the workbook reconciled to Eastgate's accounts; Eastgate's finance lead reports, Horizon finance reviews. All four are common core; the placement-role confirmation is award-specific.
Your checks should flag registrations reported in place of enrollment, and 30 registrations without a status. A sample request: "For A-31's Q1 update, please confirm how many of the 150 eligible participants were admitted to a cohort, which is how we agreed to count enrollment, and give a status for each of the other 30 registrations: not yet eligible, not started, withdrawn or unknown. Your submitted report stays as it is." A fair board sentence: "Eastgate's first quarter shows 72% attendance among 150 eligible participants; enrollment against the 200 target is being confirmed, and placements are first due later in the year." The practice pack has a fuller answer key.
Ask any vendor, including us
Upload one real grantee report as the grantee keeps it, not re-keyed, and ask the tool to return each of your defined measures with the page or cell it came from. Give it a report that uses registrations where you asked for enrollments and see whether it flags the mismatch or converts it silently. Ask it to mark a measure that is not yet due without writing a zero. Then ask for the clarification request and check that it lands on the same grantee record, with the original submission unchanged.
Questions grant teams ask
Can selection criteria change between rounds?
Yes. Give each round's eligibility rules, rubric and anchors a version, and record which version each assessment used. When you compare rounds, state what changed first. Scores from different rubrics are not directly comparable, and a higher average under a looser rubric does not show that applicants improved.
Must every grantee report the same outcomes?
No. Agree a small common core of measures that mean the same thing across the round, such as unique participants enrolled or budget against actual, and define them identically. Keep award-specific evidence, such as a condition on one grant, on that grantee's record. Inventing one universal outcome to fill a dashboard produces totals nobody can explain.
What should a grant reporting data dictionary include?
For each measure: the unit, the period, the population it covers, the source document or system, and the owner who reports and reviews it. Add when the measure first becomes due, whether rates must show numerator and denominator, and the version date. Keep the dictionary on the award record so every reviewer and every check reads the same definitions.
Why not let grantees type their metrics into a form?
Because they already hold the figures in their own program reports and budget workbooks, and copying them adds work and errors. Asking for the documents they keep, then reading them against your definitions on arrival, keeps the source attached to every number. Keep a form field for a short note on what changed.
How do you check a grantee report for completeness?
Write a short rubric of pass-or-flag checks before reports arrive: every due measure present, units and populations as defined, denominators shown, every participant accounted for, the right budget version, and evidence for each award condition. Run it on each report as it arrives, have staff confirm the flags against the cited page, then send one focused clarification request.
What if a grantee reports registrations instead of enrollments?
Do not relabel them or estimate a conversion. Record the registrations as submitted, flag that the agreed measure is missing, and ask for the count under the agreed definition plus a status for everyone registered but not counted. Until the answer arrives, report the measure as pending. Unknown records stay unknown rather than becoming zero or withdrawn.
Deep dives for this lesson
Open one when the exercise raises that question, then come back. Each uses the same Horizon example.