Additional course links
You leave with: An eligibility checklist, an anchored scoring table with a worked weighted score, and a calibration record for Horizon's R-2 rubric.
Where this fits: Lesson 2 asks you to rewrite Horizon's delivery-feasibility anchors and score A-31 before the live round. This deep dive answers the question that step raises: what belongs in an eligibility gate, what belongs in a score, and how precise an anchor has to be. You bring back an anchored rubric and a calibration record for step 2 of the workbook.
Watch · 0:47–2:13 · each field gets a prompt
Unmesh configures each field with a prompt so it is analyzed as it arrives, then designs the intake itself. Watch for the link between the two: a field can only be read against a criterion if the rubric says what evidence that criterion needs. That is what the anchors on this page supply. Watch on YouTube ↗
Jump to: 0:47 — configure each field with a prompt · 1:14 — intake design that listens
Design a grant rubric by stating what the funding should make possible, separating eligibility gates from merit criteria and missing evidence, and writing anchors that name the evidence behind each score level. Set weights, decide in advance how gaps are handled, and test the whole rubric on sample applications with more than one reviewer. Version it before the round opens so everyone knows which standard governed each application.
Start with the decision the grant supports
Write one sentence describing what the funding should make possible, for whom and over what period. For Horizon's Youth Pathways Fund: "We fund one-year programs that help 16–24-year-olds in Lake County prepare for, enter and keep employment." That is a starting statement, not an indicator.
Then ask what evidence would make a proposal credible: a pathway from training to placement, employer commitments, a delivery plan the organization can carry out, and a way to know whether it worked. Writing polish and a famous name should not quietly become extra criteria.
Check the form against the rubric. R-2 collects four narrative answers, a budget, an employer or partner letter and the most recent Form 990 where one is filed. Each should feed a criterion or a gate; if a field changes no decision, remove it.
Keep eligibility, merit and missing evidence separate
Eligibility is a decision against a mandatory rule. Merit is an assessment of quality among eligible applications. Missing evidence is an information state: it may justify a clarification request, or a rejection if your published policy says so, but it is not automatically ineligibility.
| Question at Horizon | Type of rule | What the reviewer records |
|---|---|---|
| Does the program serve Lake County and ages 16–24? Is the request $25,000 or less? | Eligibility gate | Pass, fail or unresolved, with the source and reason |
| Is the applicant a 501(c)(3) or fiscally sponsored? | Eligibility gate | Pass, fail or unresolved; the determination letter or sponsor letter as source |
| How credible is the placement pathway? | Merit criterion | Anchored score, cited passage and gaps |
| A-44's fiscal-sponsor letter is absent | Missing evidence | Held; one clarification request, owner and deadline |
| Reviewer R3 sits on A-17's board | Conflict and assignment control | Declaration and reassignment; never a lower applicant score |
Even a geography rule needs a definition. Does "serves Lake County" mean the applicant's address, the young people served or the delivery site? State which, and what source establishes it. Record an unresolved gate as unresolved, and give every applicant the same clarification opportunity and deadline. A-44 was held, answered one request within the window, and was then scored like everyone else.
Build the rubric in three passes
| Pass | What you decide | Horizon R-2 |
|---|---|---|
| 1. Name the criteria | A small set of distinct decisions; combine overlaps, keep any criteria a funder requires | Outcome pathway, delivery feasibility, budget justification, learning and reporting plan |
| 2. Describe evidence and levels | What a reviewer must see at each level, and which field or upload supplies it | Five-point anchored scale, each anchor naming its source |
| 3. Set weights and controls | Weights, missing-evidence treatment, conflicts, ties, who approves | 30 · 30 · 20 · 20; held status for gaps; committee decides |
Not every program uses a weighted sum. The NIH simplified peer-review framework scores two factors and assesses a third, expertise and resources, for sufficiency; all three inform an overall impact judgment. The weighted model below is Horizon's choice for a small round, not a rule for every funder.
Write scoring anchors that point to evidence
"Excellent delivery plan" does not tell a reviewer what to look for. An anchor describes the observable features that justify a rating. These are Horizon's delivery-feasibility anchors after calibration:
| Score | Evidence anchor · Horizon example, fictional |
|---|---|
| 1 — major gaps | Activities are listed, but responsibilities and sequence are unclear; no workable delivery plan is established. |
| 2 — partly developed | Some responsibilities and dates are named, but a major dependency is unsupported. |
| 3 — workable with gaps | Roles, milestones and sequence are described; at least one material dependency still needs a credible resolution. A letter of interest counts as a named dependency, not a confirmed commitment. |
| 4 — well supported | Roles, milestones and major dependencies are supported by confirmed commitments; only limited gaps remain. A letter of interest counts as a named dependency, not a confirmed commitment. |
| 5 — fully supported | Accountable roles, milestones, confirmed essential commitments and a plausible response to the main delivery risks. |
Each anchor points to the delivery-plan answer and the partner letter. A named partner is not a confirmed commitment, and a signed letter is not proof the partner can deliver. Ask for evidence in proportion to a $25,000 grant. Write separate anchors for each criterion, and check that the same fact is not rewarded twice, such as an employer commitment under both outcome pathway and feasibility.
Calculate a weighted score without hiding the reasoning
In R-2, scores run from 1 to 5 and weights sum to 100. Each contribution is weight × score ÷ 5. The scale's minimum total is 20, not zero; if you need a zero-evidence score, define that anchor before you use it. Here is A-31, Eastgate Youth Works, after calibration.
| Criterion · weight | Score → points | Reason to verify |
|---|---|---|
| Outcome pathway · 30 | 4 → 24 | Clear placement pathway; follow-up assumptions remain |
| Delivery feasibility · 30 | 3 → 18 | One essential employer commitment is an expression of interest, not a confirmed placement role |
| Budget justification · 20 | 4 → 16 | Costs linked to activities; one estimate needs support |
| Learning and reporting plan · 20 | 3 → 12 | Follow-up method described; treatment of missing responses unclear |
| Total · 100 | 70 | A review result, not an automatic award |
The total is 24 + 18 + 16 + 12 = 70. It summarizes the ratings; it does not explain them, so keep the criterion scores, sources and notes. Whether 70 is funded depends on the round's rules, the money available and the committee. Horizon's committee approved A-31 with a condition: confirm the employer placement role before the second payment.
Do not treat an unanswered question as zero or quietly renormalize the remaining weights. Decide in advance whether a gap prevents review, permits clarification or receives a defined rating.
Calibrate reviewers before the live round
Give reviewers the same sample application, ask them to score independently and cite the passage, and compare reasons before totals. Horizon's six reviewers scored A-31. On delivery feasibility one gave a 4 and another a 2, which at a weight of 30 is 24 points against 12, a 12-point difference.
Calibration record · A-31 · delivery feasibility · fictional
Original scores: 4 and 2. Cause: one reviewer read the employer's letter of interest as a firm commitment; the other required a confirmed placement role.
Resolution: the anchor was unclear, not either reviewer. The team added "a letter of interest counts as a named dependency, not a confirmed commitment" to the 3 and 4 anchors. Both reviewers rescored to 3.
Kept: both original scores, the reason, the anchor change and the rubric version it created.
The useful action was to clarify the evidence standard, not to average the disagreement into an 18 nobody held. Run a weak and a borderline sample the same way. Agreement exposes ambiguous anchors; it does not prove the rubric valid or free of bias.
Control changes after applications open
Approve and version the rubric, eligibility rules and application instructions together before launch, with the owner, approval date and the rounds they apply to. Reviewers should be able to tell which standard governed a submitted application. A rubric that lives as rubric_v6_FINAL(2) in someone's downloads folder fails this test.
If a defect appears mid-round, pause the affected decisions and use your approved change process. Assess who could be disadvantaged, whether applicants must be told, and whether the round needs an extension or restart; rescoring alone does not make a changed competition fair. Keep the earlier version, the reason, the approval and the list of affected applications.
Use the rubric as the reading instruction
A well-anchored rubric is also a good prompt. In Sopact Sense, the Intelligence Cell reads each answer and upload as it arrives against the criterion it feeds, and returns a draft score, the passage behind it and any missing evidence. The anchors do the work: a vague anchor produces a vague reading, for a person or for AI.
Keep the application ID, submission version, rubric version and source location with each finding. Reviewers verify the cited passage and approve or override the score. The application scoring rubric guide shows a wider selection workflow.
Put it into practice
Make Horizon's rubric usable by another reviewer
- Write R-2's funding-purpose sentence in your own words and list the four eligibility gates, including what "unresolved" means for each.
- In
horizon-rubric.csv, write an evidence anchor for every score level of one more criterion, budget justification or learning and reporting plan, naming the field or upload each level depends on. - Score A-31 from
horizon-r2-applications.csvand reproduce the weighted total by hand. - Compare with
calibration-scores.csv, explain the largest disagreement and write the anchor change that resolves it. - Record the rubric version, owner and approval date. Carry the result into step 2 of the workbook.
Download the workbook (PDF)
Practice data: horizon-r2-applications.csv · horizon-rubric.csv · calibration-scores.csv · eastgate-q1-update.csv
Check your reasoning before moving on
Another reviewer should be able to tell ineligible from incomplete: A-44 is incomplete, held and clarified, never failed for the missing letter. Your anchors should name sources, not adjectives. The A-31 arithmetic is 4 × 30 ÷ 5 = 24, 3 × 30 ÷ 5 = 18, 4 × 20 ÷ 5 = 16, 3 × 20 ÷ 5 = 12, total 70. The feasibility disagreement resolves through the anchor, to 3, with the original 4 and 2 kept. If a colleague cannot find the source behind a score, fix the anchor or the evidence request before adding questions to the form.
Questions grant teams ask
What is the difference between eligibility and scoring?
Eligibility tests the mandatory requirements for entering review, such as geography, organization type or a request limit. Scoring assesses quality among eligible applications against published criteria. Missing evidence needs its own status and follows the round's clarification or rejection policy, so an absent document is not silently turned into either a failed gate or a low score.
How many criteria should a grant rubric have?
Enough distinct criteria to represent the actual funding decision, and no more. Three to five is a useful drafting range, not a rule; Horizon uses four for a small round. Keep any criteria a funder requires and merge overlapping ones that reward the same evidence twice.
Why write evidence anchors instead of adjectives?
Adjectives like "strong" or "excellent" mean different things to different reviewers. Evidence anchors say what must be present in the application to justify each level and where to find it. They let reviewers tell a confirmed commitment from a letter of interest and discuss disagreements against the same standard. They support consistency; judgment still matters.
How do you calculate a weighted grant score?
On a five-point scale, multiply each criterion's weight by its score divided by five, then add the contributions. With weights totaling 100 the result is out of 100, with a minimum of 20 unless you define a zero anchor. Horizon's A-31 scores 4, 3, 4 and 3 on weights of 30, 30, 20 and 20, which gives 24 + 18 + 16 + 12 = 70.
Can AI make the funding decision?
Keep the decision with authorized people. AI can read each application against the rubric as it arrives and draft a score with the passage behind it, which saves weeks of reading. Reviewers still verify the evidence, resolve the gaps, record overrides and decide, with the rubric version and reasoning kept on the record.