1. Where to find each piece of evidence
For each criterion, name the application section where the evidence lives. Significance evidence might be in the narrative pages 1-3 and the supporting data attachment. Methodology evidence is typically in the narrative pages 4-8 and the budget detail. Sustainability evidence is in the narrative final pages, the partner letters, and the multi-year budget projection.
Naming the evidence location does two things: it speeds up review by directing the reviewer to where the answer lives, and it sets a uniform expectation. A reviewer who looks at only the narrative and ignores the budget cannot score methodology accurately. The instruction makes this explicit.
2. How to interpret mixed evidence
Applicants rarely present clean evidence states. The narrative may name a specific geographic area but cite no data source. The budget may include an evaluation line but at 2% of total. The methodology may describe the approach but skip the timeline. Evaluator instructions describe the decision rule for mixed evidence: which elements are essential, which are nice-to-have, and how to weight partial fulfillment.
Example rule: "If two of three required elements are present, score 3 (Satisfactory). If all three are present but one lacks specificity, score 4 (Good). If one of three is present, score 2 (Needs improvement). Score 5 (Excellent) only when all three are present with specificity AND the application names a source for each."
3. How to validate AI proposed scores
Reviewers using Sopact Sense receive AI-proposed scores with citation evidence. The validation instruction set covers: confirm that the cited evidence actually appears in the application as quoted, confirm that the cited evidence supports the proposed score, decide whether other evidence in the application changes the score, and either accept the AI score, adjust by one point with a reason, or override completely with a reason.
Adjustments and overrides leave a reasoning note in 1 to 2 sentences. Over time, these notes become a calibration signal: criteria where reviewers systematically adjust the AI score in one direction reveal anchors that need rewriting.
4. How to record reasoning
Every score gets a reasoning note. Not a paragraph. One to two sentences capturing what was decisive. "Methodology described clearly with named milestones; evaluation budget at 3.5% is below the 5% threshold for this criterion's level 5." Reasoning notes serve three purposes: they make the reviewer's score defensible if challenged, they provide raw material for applicant feedback, and they give the program team a window into how the rubric is being applied in practice.
Reasoning notes do not replace AI citation evidence. They sit alongside it. AI provides the evidence trail. The reviewer provides the judgment trail.
5. How to flag conflicts of interest and out-of-expertise applications
Evaluator instructions must include a clear flag mechanism for two situations. First, conflicts of interest: prior employment with the applicant organization, personal relationship with named staff, financial interest in the proposed work. The flag is mandatory; reviewers do not score the application. Second, out-of-expertise applications: a reviewer trained in early-childhood education cannot reliably score a clinical research proposal. The flag triggers reassignment to a more appropriate reviewer.
The flag mechanism should be one-click and visible at every stage of review. Hidden flagging is a failure mode: reviewers who feel uncomfortable but cannot find the mechanism either score uncomfortably or quietly disengage.