Additional course links
By Sopact Academy · Updated September 12, 2026. Role descriptions, records and numbers below are fictional teaching examples.
Make each review explainable
Bring: the employer-approved requirements checklist, its version and the evidence the candidate has agreed to provide.
Leave with: an assessment record, a list of unresolved questions and a documented next step.
This lesson is for placement coordinators and program teams. Use it with an approved role definition; it does not replace an employer’s selection responsibilities. The numerical example is a fictional exercise, not a validated hiring assessment.
Review the evidence in six steps
- Separate requirements, assessments, development and the next decision.
- Define observable scoring anchors and accepted evidence.
- Check the calculation and mark missing observations.
- Confirm development options with the participant.
- Review any AI-prepared summary against its sources.
- Compare and approve rubric changes before using them.
What consistency can—and cannot—solve
Two reviewers may interpret the same evidence differently. One may see a missing attachment as an unmet requirement; another may ask for the document. A shared rubric makes the expected treatment explicit and lets the team examine disagreements. It does not guarantee identical judgment or a fair result.
Bias can enter through the requirements, the evidence people have an opportunity to provide, the scoring anchors, the tool and the way staff act on the output. Applying the same flawed rule to everyone can preserve the problem. Review the process as well as the calculations.
For US employment, the EEOC’s AI and disability resources are relevant to tools used in applicant assessment. Make the appropriate accommodation and alternative-assessment process available. Do not use private case notes to infer disability, personality, citizenship or another personal characteristic for ranking.
Step 1 — Keep four review questions separate
Start by checking the role definition rather than designing a score. Confirm which requirements are approved, which are preferences and which remain unresolved. Then record what the available evidence supports.
| Question | Record separately | Avoid |
|---|---|---|
| Is a confirmed requirement satisfied? | Met, confirmed unmet, or awaiting evidence; source and date | Treating an empty field as a failure |
| What does a relevant assessment show? | Task, instrument, result, conditions and rubric version | Scoring an inferred trait or using an unrelated survey score |
| Is development an option? | Agreed route, availability, timing and confirmation status | Promising that a missing qualification will be completed |
| What happens next? | Reviewer decision, reason, owner and communication status | Treating an AI output as the final decision |
Do not add work authorization or citizenship to a weighted “fit” score. Where an eligibility check is appropriate, handle it through the approved process. The Department of Justice’s hiring guidance explains that work authorization and citizenship are different and that citizenship restrictions have limited grounds. A role description alone does not establish a lawful restriction.
Step 2 — Define evidence and scoring anchors
For any scored criterion, specify the job-related task, acceptable evidence and what each rating means. A label such as “excellent communication” leaves reviewers to supply their own definition. A task such as documenting a completed workshop procedure can be assessed against agreed, observable requirements.
Use a numeric score only where it helps the decision. Some requirements are better represented as a checked document or an unresolved question. Do not force every field into a weighted average, and do not allow a high average to erase an unresolved requirement.
Using this approved checklist and these assessment instructions, draft a review rubric. Identify the relevant task and evidence for each scored criterion. Define observable rating anchors and list anything that needs employer clarification. Keep eligibility and missing evidence outside the weighted score. Do not infer traits, invent evidence or recommend a hiring decision. Return a draft for a responsible reviewer to approve.
Test the anchors on several de-identified examples with different kinds of evidence. Include a missing document, a borderline task result and a correction to a record. Ask reviewers to explain disagreements before changing weights. A polished 0–100 scale does not establish that the assessment predicts job performance.
Step 3 — Check a worked calculation
In this fictional exercise, an employer has approved three practical tasks and a 0–4 rating scale. Four means the defined task standard was demonstrated; lower ratings correspond to documented levels of support or incompleteness. The actual anchors would need to specify what those levels mean for each task. For the fictional work-sequence task, an example anchor set is: 4, independently places every supplied step in the approved order; 3, reaches that order after one permitted prompt; 2, needs step-by-step guidance; 1, still leaves major ordering errors after that guidance; 0, attempts the assessment but demonstrates none of the specified sequence. An unattempted or missing assessment is unknown, not zero. These are teaching anchors, not a validated employment test; the responsible team must confirm task relevance, permitted assistance and appropriate assessment arrangements.
| Illustrative task | Weight | Observed task rating | Weighted contribution |
|---|---|---|---|
| Read the specified diagram | 40% | 4 of 4 | 40 × 4/4 = 40 |
| Prepare a defined work sequence | 35% | 3 of 4 | 35 × 3/4 = 26.25 |
| Document the completed procedure | 25% | 3 of 4 | 25 × 3/4 = 18.75 |
| Total | 100% | All three observations available | 85 of 100 |
The arithmetic is correct: 40 + 26.25 + 18.75 = 85. The number only summarizes this exercise under these weights. It does not mean an 85% chance of success, prove the person should be hired or establish a fair comparison with another assessment.
Now remove the work-sequence observation. Do not quietly enter zero or rescale the other two criteria to 100. Mark the score incomplete and show that observations cover 65% of the assigned weight. Ask for the missing assessment or use an approved alternative. If a protocol permits prorating, disclose that method and assess whether the resulting scores are comparable.
Keep the candidate–role pair ID, role version, source references, assessment date, reviewer and rubric version with the calculation. Another reviewer should be able to reproduce the arithmetic and understand the evidence without relying on the first reviewer’s memory.
Step 4 — Plan development without disguising a rejection
An approved development option should state what is needed, why it matters, when it must be completed and what support is available. Confirm the participant’s interest. A course seat that is unavailable, unaffordable or outside the person’s schedule is not a usable plan.
Distinguish an employer preference from a mandatory prerequisite. If an employer provides training after the start date, do not invent a requirement that it be completed before application. If an actual prerequisite remains unmet, record that specific issue and the responsible reviewer’s next step rather than labeling the person generally “not qualified.”
Offer a clear way to correct inaccurate or incomplete evidence. A document may be expired in the database because its replacement was uploaded elsewhere. Check the linked record before drawing a conclusion, and keep the correction history visible.
Step 5 — Use AI to organize evidence, with review
In a configured Sopact workflow, collection forms, uploaded documents and approved role information can be connected to participant and opportunity records. AI can prepare an evidence summary against the defined fields and highlight discrepancies. Reviewers check what the source supports and approve the next action.
For this candidate–role review, use only the approved checklist and supplied evidence. Return requirement ID, supported observation, exact source reference, date and unresolved question. Preserve “not stated” and “awaiting evidence.” Distinguish a missing document from a confirmed unmet requirement. Reproduce only the supplied scoring formula. Do not infer personal traits or select or reject the candidate. Identify what the reviewer must check.
A citation helps locate the evidence but does not guarantee that the summary is faithful. Check negation, dates, document versions and whether a statement belongs to the right person. Keep an assessment result distinct from a mentor’s opinion or a participant’s own account.
Test the workflow on examples with no relevant evidence as well as clear cases. Otherwise a tool may look accurate only because the pilot contains easy records. Record unsupported extractions, missed evidence and wrong-record matches separately so the team knows what needs correction.
Step 6 — Compare rubric versions before adopting changes
A changed weight can move scores without any change in the underlying evidence. Save a proposed version, compare its effects and explain the reason for the change. Do not tune weights to produce a preferred individual result.
Decide which pending reviews need reconsideration and whether completed decisions require a separate correction process. Keep original scores and decisions intact alongside the new version. Reprocessing every record without preserving the history would make earlier decisions harder to explain.
| Change under review | What to check | Record before release |
|---|---|---|
| A task weight changes | Which scores move, and whether the reason is job-related | Old/new rubric, rationale, affected reviews and approval |
| Evidence becomes available | Whether it resolves an unknown rather than changes a prior fact | Source, date and updated review status |
| Employer changes a prerequisite | Whether the new rule applies to pending reviews | Approved role version and communication plan |
| A reviewer overrides a summary | Whether the source supports the revised interpretation | Reason, evidence and reviewer identity |
Read the pool without mixing units
A participant can be considered for several roles, and a role can have several candidates. Candidate–role pairs, distinct people, openings and placements are different counts. A list of 108 pairs is not a cohort of 108 participants.
For example, 60 people might be associated with 24 roles through 108 recorded pairs. Twelve later placements would need their own verified outcome records and date range. You cannot calculate a participant placement rate by dividing those twelve placements by 108 pairs, or infer that a high match score produced the placement.
At pool level, review missing evidence, unresolved employer questions, development requests and time waiting for a decision. Look for inconsistent treatment and potential barriers in the process. A difference between groups can justify investigation; it does not by itself establish why the difference occurred.
Watch the wider training-to-placement workflow
4 minutes 13 seconds · Sopact demonstration using synthetic training records.
▶ Play: Training Program Data
Exercise: review the process before expanding it
- Choose five fictional or properly de-identified examples, including missing and contradictory evidence.
- Have two reviewers apply the draft rubric independently.
- Compare the specific criteria where they disagree and inspect the sources.
- Check the arithmetic and the handling of unknowns.
- Test the correction and accommodation routes with the responsible team.
- Approve a version and document what still requires human judgment.
Keep this pilot record. Repeat targeted checks when the role, assessment, model or rubric changes. Consistency is a process to maintain, not a claim established by writing a rubric once.
Frequently asked questions
How should two reviewers compare their scores?
Have each reviewer score the same evidence independently before discussing differences. Compare the cited evidence and the written anchors, then record why a score changed. The CIPD selection-methods guidance recommends pre-agreed criteria and independent scoring before group discussion. Agreement alone does not validate a rubric: also check whether it measures the job-relevant task and whether assessment arrangements are appropriate.
Can a candidate-matching score be bias-free?
No score or AI tool can guarantee that. Review the requirements, evidence, assessment design and treatment of candidates. Use testing, appropriate alternatives, correction routes and accountable decisions to identify and address problems.
Should missing evidence receive zero?
Not automatically. Unknown evidence and demonstrated performance at the lowest anchor are different states. Follow the approved assessment protocol and clearly show incomplete coverage.
What should a match score represent?
A defined summary of relevant assessment evidence under a particular rubric. It should not be described as a probability of success unless that use has been independently justified and validated.
Can a reviewer override an AI summary?
Yes. The reviewer should correct an unsupported interpretation and record the source and reason. Preserve the original output where needed for review rather than hiding the correction.
Does a high score count as a placement outcome?
No. Placement and retention require their own definitions, dates and evidence. Connect the later outcome to the review record without claiming the score caused it.
Reporting reference: use the placement investor-report lesson for outcome and financial reporting, or the grant-funder lesson for grant-funded training. Both should distinguish assessment activity from verified outcomes.