play icon for videos
← Academy
SOPACT ACADEMY · SURVEY INTELLIGENCE · ANALYZE

How Do I Measure the Impact of a Scholarship Program?

Measuring scholarship impact means tracking the outcomes an award is meant to produce — persistence, graduation, reduced financial stress, and post-graduation results — not just the dollars disbursed. Here is the five-step method, from baselining every applicant to following recipients after they graduate.

Sopact Academy · Course review

Academy / Applications, awards & grants / Deep dive

Practical Academy guide

How Do I Measure the Impact of a Scholarship Program?

To evaluate a scholarship, connect the award with the outcomes it is intended to support: for example, continued study, completion or reduced financial pressure. Plan appropriate follow-up and report what is known, what is missing and what the evidence can establish. Award counts and disbursements remain useful delivery measures; they do not by themselves show a change in recipients’ circumstances.

Additional course links

This lesson is for scholarship administrators, program teams and evaluators. Bring the purpose of one award and its current collection process. You will build a recipient follow-up plan, define one outcome and prepare a report that separates delivery, observed results and unanswered questions.

Start with the scholarship’s purpose

A completion grant, a living-cost award and a professional-development scholarship may aim at different changes. Select outcomes that match the award rather than requiring every recipient to report GPA, employment, debt and confidence.

Write the expected change, who it concerns and when it could reasonably be observed. Include how the recipient views success. A change in study mode, a transfer or a temporary pause may have context that a simple “retained/not retained” label misses.

Separate selection from evaluation

Application information can provide a starting point when it is appropriate and collected for a clearly explained purpose. But adding evaluation questions is not cost-free: it takes time and may request sensitive information that is unnecessary for selection.

Decide which information is required to assess eligibility and which belongs in voluntary or post-award evaluation. Explain whether responses affect the award and who will see them. Avoid exposing personal circumstances to reviewers who do not need that information.

If non-recipients are part of an evaluation design, establish a suitable purpose and collection process. Funded and unfunded applicants may differ because of the selection process. Comparing them does not automatically isolate the scholarship’s effect.

Connect the person, award and observation

Keep the student’s identity distinct from each application, award, payment and academic observation. A person may receive several awards or change institutions. A new award should not erase an earlier record, and a new term should not overwrite the previous term’s result.

RecordUseful context
RecipientStable identifier and appropriately maintained contact details
AwardProgram, cycle, decision, amount and conditions
DisbursementPayment date, amount and source record
ObservationOutcome definition, period, source and review status
Follow-upWhen it is due, response status and permitted next action

Use the relevant person and award relationships to join these records. Do not place all values in one row or sum the same award once for every term response.

Define a small, usable collection plan

For an award intended to support continued study, define what continuation includes: remaining at the same institution, an approved transfer, a different study load or another agreed status. Identify the checkpoint and eligible group. Keep completion, withdrawal, pause and unknown status distinguishable.

Across institutions, agree the few shared fields necessary for reporting. Local programs may ask different questions about support needs. Map compatible terms through a versioned dictionary rather than imposing one complete survey. Do not aggregate different academic calendars or grading systems without a defensible rule.

Reuse stable registration details. Confirm changing contact or study information at an appropriate interval, retaining the dates relevant to each observation. Collect academic or financial detail only when it serves the agreed purpose and access is appropriate.

Work through a continuation example

A fictional program has 40 recipients due for a next-term check. Thirty respond: 24 meet the agreed continuation definition and six do not. Ten have unknown status.

MeasureResultInterpretation
Response coverage30/40 = 75%One quarter of due records remain unknown.
Continuation among respondents24/30 = 80%Describes the responding group.
Known continuing share of the cohort24/40 = 60%Does not classify the ten unknowns as withdrawals.

If all ten unknowns continued, the cohort share would be 34/40, or 85%. The 60–85% range describes the unresolved statuses under this example’s definitions; it is not a confidence interval and does not estimate the scholarship’s causal effect.

Record a useful next action: confirm the ten missing statuses through the agreed process and review reasons for non-continuation where recipients choose to share them. Do not report 80% as an established rate for all 40 recipients.

Use reflections to understand reported experiences

A recipient may describe transport costs, paid work, caring responsibilities or a change in study plans. Code these accounts consistently and preserve the relevant source passages. Report how many people provided usable comments and whether several themes can apply to one response.

These accounts explain participants’ reported experiences. They may help design a better support response, but they do not prove that a theme caused withdrawal or establish its prevalence among non-respondents. Avoid inventing a reason when one was not given.

Check quotation permissions and disclosure risk before using a story. In a small award program, removing a name may still leave someone identifiable through their institution, course and circumstances.

Plan later outcomes when they serve a real question

If the award aims to support graduation or later employment, define when follow-up becomes due and what evidence is practical to obtain. Some recipients will not yet have reached that stage. Keep “not yet due” separate from missing and from an observed outcome.

A persistent identifier helps connect the records; it does not ensure recipients respond. Use an appropriate contact plan, clear purpose and reasonable burden. Reassess whether each later question is still useful to the program and recipients.

Prepare a balanced report

Show awards and disbursements, the outcome definition, eligible population, response coverage, observed results and limitations. Explain changes in the collection method or recipient mix before comparing cycles. Baseline-to-follow-up change is useful descriptive evidence but does not alone establish that the award caused it.

Do not apply red/amber/green labels to students based on incomplete outcomes. If the team uses a workflow status, make it describe the evidence or agreed follow-up action. A student’s circumstances require context, not an automatic score.

Configure the recurring process in Sopact

Connect recipient, award and dated evidence records. Use the approved dictionary to support comparable analysis while retaining local questions. Review candidate themes, missing statuses and conflicting observations before approving the reporting view.

The goal is to maintain the evidence through the award’s life, reducing repeated matching and assembly. Test the calculation on a small sample, review permissions and keep a dated reporting version. Human reviewers remain responsible for the interpretation and decisions.

Practice: design one follow-up

  1. Write the award’s purpose and one outcome definition.
  2. Identify which information can be reused and which must be collected later.
  3. Choose the eligible group, timing and response statuses.
  4. Draft a short question and an appropriate local follow-up.
  5. Use the fictional table to write a clear board-report finding without hiding unknowns.

Frequently asked questions

Must baseline data be collected from every applicant?

No. Choose the collection point from the evaluation purpose, burden and permitted use. Reuse suitable application data, but do not require unnecessary sensitive information for a later analysis.

Can we compare recipients with unsuccessful applicants?

That may support a carefully designed evaluation, but selection differences must be addressed. A simple comparison does not establish the award’s effect.

Can a small program evaluate outcomes?

Yes. It can describe known outcomes and learn from participants. Small numbers affect interpretation and disclosure, and do not justify stronger causal claims.

Return to your course →

Put this guide into practice.

Start with a decision your team needs to make. Decide what information belongs together, how you will check it, and who is responsible for acting.

See context in action →
Prepare Data Governance Before Using AI
Prepare your data governance
nonprofit-data-governance-before-ai
Feedback
Foundation
Prepare an evidence register and a tested governance baseline before applying AI to program data.
How to Turn Findings into Action and Check What Changes
The Loop — the method in one read
the-loop
Loop
The method
0
One continuous method for reliable, traceable AI reporting across case, application, grant, and program workflows: collect clean, analyze on arrival, and improve in time.
For growing data collection, connected analysis and recurring reporting
What Is Case Intelligence?
What Is Case Intelligence?
what-is-case-intelligence
Case
Foundation
1
One current, traceable record for each person—connecting intake, services, notes, surveys, documents, outcomes, decisions, and follow-up.
Workforce and training · Youth and mentoring · Case management · Scholarships · Accelerators · Education · Nonprofit programs
What is grant intelligence?
What Is Grant Intelligence?
what-is-grant-intelligence
Grant
Foundation
1
One connected evidence record from application and committee review through the awarded grant, grantee reporting, renewal, and board accountability.
Foundations and grantmakers · Public grant programs · Scholarships and fellowships · Accelerators
Connected Data Intelligence
Understand the approach
connected-data-intelligence
Feedback
Foundation
1
Connect recurring collection, relevant history, AI analysis, and governance in a workflow your team can maintain.
Growing organizations managing recurring data collection without a dedicated data team.
Measurement and Reporting: From Agreement to Evidence-Based Report
Measurement and reporting: from agreement to report
embedded-impact-measurement
Reporting
Start here
1
Turn the onboarding call into a reporting agreement, share one data dictionary, and write reports funders can compare and check.
For funders and the organizations they fund
Connect company context before collecting another return
Connect context
track-investees-impact-agreement-variance
Portfolio
Portfolio intelligence tools
1
Build a repeatable collect, review and improve cycle
Methodology — continuous, not annual
loop-methodology
Loop
The method
1
The continuous collect–analyze–improve cycle, adopted as an experiment: start with the step that already pays and add one data-collection step at a time.
Teams tired of rebuilding spreadsheets and forms who want a measurement system that compounds instead of resetting.
Agree the theory of change and core metrics on the onboarding call
Agree measures
onboard-portfolio-lock-impact-agreement-track-results
Portfolio
Portfolio intelligence tools
2
Test whether an AI-assisted result is repeatable and correct
Reliability — the same answer twice
loop-reliability
Loop
The method
2
Determinism as a feature: the same question over the same data returns the same answer every run — the opposite of a generic AI chat that drifts.
Anyone who has watched a general AI tool give two different numbers for the same question and needs results they can stand behind.
Design an application process around the decision, not the form
Design an Application Process
how-to-design-an-application-process
Grant
Foundation
2
How to Structure Stakeholder Data: Four Common Patterns
Choose your record structure
which-shape-is-your-data
Feedback
Foundation
2
Map the people, observations and relationships your workflow needs before collecting data.
Turn the onboarding call into a reporting agreement
Turn the onboarding call into a reporting agreement
onboarding-call-to-reporting-agreement
Reporting
Agree together
2
How to build a theory of change, then show it as a logic model, logframe or results framework
How to build a theory of change, then show it as a logic model, logframe or results framework
how-to-build-a-theory-of-change
Reporting
Agree together
3
How to Build a Logic Model: Steps, Example and AI Prompt
Build a Logic Model You Can Use
how-to-build-a-logic-model
Reporting
Align
3
How do you onboard a grant or RFP program?
Onboard a Grant or RFP Program
how-to-onboard-a-grant-rfp-program
Grant
Foundation
3
Turn your theory of change into a data-collection plan
Turn a Theory of Change into a Data-Collection Workflow
theory-of-change-to-data-collection-workflow
Case
Foundation
3
Run quarterly collection around each company's dictionary
Collect quarterly
collect-investee-reporting-without-burden
Portfolio
Portfolio intelligence tools
3
Keep a clear trail from a finding to its evidence
Traceability & Transparency
loop-traceability
Loop
The method
3
Every figure links back to the exact response, note, or document it came from — a full audit trail from headline result to raw evidence.
Teams whose numbers get scrutinized — by funders, boards, auditors, or standards — and who need to answer where did this come from on the spot.
Build Context: What Your AI Needs to Know
Build context
build-organization-evidence-model
Feedback
Foundation
3
How to Change Survey Questions Without Losing Comparability
Change questions with a clear history
change-questions-without-breaking-the-record
Feedback
Control
3
Create a question-change log and decide how old and new versions should appear in reports.
Programme & MEL leads · Teams whose questionnaire has ossified · Anyone evaluating a platform where configuration is a purchased service
Five Dimensions of Impact: How to Review Your Evidence
Use the Five Dimensions to Test the Evidence
five-dimensions-of-impact
Reporting
Align
4
How do you design a grant rubric and eligibility rules?
Design Your Rubric & Eligibility Rules
grant-rubric-eligibility-rules
Grant
Foundation
4
Review impact and financial evidence before it reaches the dashboard
Review and approve
read-investee-reports-multi-signal
Portfolio
Portfolio intelligence tools
4
Adapt the learning cycle to your workflow
Flexibility — one method, four workflows
loop-flexibility
Loop
The method
4
The same collect–analyze–improve cycle, shaped to four kinds of impact work — case, grant, portfolio, and feedback — each shown end to end.
Anyone deciding where the Loop fits their work, who wants to see the full path from messy input to a report they can defend.
How to Collect Feedback Offline and Keep Records Connected
Collect offline and reconcile the batch
collect-feedback-offline
Feedback
Connect
4
Build a field protocol and reconcile a test batch across devices, visits and delayed uploads.
Field & multi-site programs · Low-connectivity contexts · Nonprofits collecting in person
How Do You Design an Intake Form for a Baseline?
Design an Intake Form That Captures a Usable Baseline
intake-form-usable-baseline
Case
Nonprofit Track
5
Turn a proposed outcome into a reporting definition
Outcomes vs Outputs
frame-outcomes-over-outputs
Grant
Foundation
5
Plan and review your first workflow pilot
The Guarantee — first workflow in 2 months
loop-guarantee
Loop
The method
5
How to Analyze Documents as Evidence: Sources, Context and Review
Read documents as traceable evidence
read-documents-as-evidence
Feedback
Connect
5
Create a document register and reviewed findings with source locations, context and explicit exceptions.
Check each partner report against the agreement
Check each partner report against the agreement
check-partner-reports-against-agreement
Reporting
Funder road
5
Combine compatible metrics and explain every portfolio total
Build rollups
portfolio-impact-rollups
Portfolio
Portfolio intelligence tools
5
How to Clean Open-Ended Survey Responses Without Losing Meaning
Clean responses and define the denominator
clean-open-ended-survey-responses
Feedback
Clean
6
Create a cleaning log, response-status table and reproducible report statement.
How to Review Participant Support Needs Mid-Program
Spot At-Risk Participants Mid-Program
spot-at-risk-participants-mid-program
Case
Nonprofit Track
6
How to Write a Nonprofit Grant Application: Template and Example
Grant Application for Nonprofits
grant-application-for-nonprofit-organizations
Grant
Foundation
6
Personalize quarterly donor and annual LP reports
Report with evidence
portfolio-lp-board-impact-report
Portfolio
Portfolio intelligence tools
6
How to Build a Funder Context Profile: Research to Reporting
Build a Sourced Funder Context Profile
build-funder-context-profile
Reporting
Align
6
Roll up and benchmark portfolio results
Roll up and benchmark portfolio results
roll-up-and-benchmark-portfolio-results
Reporting
Funder road
6
How Do You Measure Change at Exit?
Measure Change at Exit (Not Just Completion)
measure-change-at-exit
Case
Nonprofit Track
7
How do you collect applications clean at the source?
Collect Applications Clean at the Source
collect-applications-clean-at-source
Grant
Collect
7
Turn portfolio findings into action and test the next cycle
Act and improve
portfolio-risk-monitoring-alerts
Portfolio
Portfolio intelligence tools
7
How to Analyze Multilingual Feedback Without Losing Meaning
Analyze and review multilingual feedback
analyze-multilingual-feedback
Feedback
Clean
7
Build a language review sheet and test software on original responses, translations, codes and reporting bases.
Multi-country programs · Multilingual survey data · Global networks & chapters
How to Define Impact Metrics Your Team and Funder Can Use
Define Measures the Organization and Funder Can Both Use
define-impact-metrics-funders-want
Reporting
Align
7
Write the portfolio report for your board or donors
Write the portfolio report for your board or donors
write-the-portfolio-report
Reporting
Funder road
7
Survey Attrition: How to Track Missing Waves in Longitudinal Studies
Track missing waves and matched outcomes
survey-attrition-longitudinal-studies
Feedback
Read
8
Build a wave-status register, compare response groups and report paired change with coverage and limitations.
How to Use Mentor Notes to Review Participant Support
Use mentor notes for support review
mentor-notes-early-warning
Case
Nonprofit Track
8
How do you reduce applicant burden?
Reduce Applicant Burden
reduce-applicant-burden-auto-clarification
Grant
Collect
8
Impact Metric Definitions: A Practical Worksheet and Example
Give Every Number One Definition
one-definition-for-every-number
Reporting
Define
8
Map every funder's ask to one evidence base
Map every funder's ask to one evidence base
turn-reporting-requirements-into-evidence
Reporting
Funded partner road
8
How to Connect Quantitative and Qualitative Survey Data
Connect scores and comments
connect-quantitative-qualitative-survey-data
Feedback
Read
9
Build a linked analysis view and joint display, with clear groups, reporting bases and evidence limits.
How to Calculate SROI as New Evidence Arrives
Calculate SROI — Live, Sourced, and Honest
calculate-sroi-live
Case
Nonprofit Track
9
How to Collect Grantee Reports with Less Burden
Collect Grantee Reports Without Burden
collect-grantee-reporting-without-burden
Grant
Collect
9
Capture each funder's taste, and your own
Capture each funder's taste, and your own
capture-funder-taste
Reporting
Funded partner road
9
How to Analyze Pre, Mid and Post Survey Data
Analyze pre, mid and post surveys
analyze-pre-mid-post-survey-data
Feedback
Read
10
Build a matched pre/mid/post analysis, interpret score movement and retain clear rules for missing waves.
How to Report a Job-Training Program to Grant Funders
Turn a Cohort into a Funder Impact Report
job-training-grant-impact-report
Case
Nonprofit Track
10
How to Follow Up on Missing Grantee Data
Chase Missing Grantee Data
chase-missing-grantee-data
Grant
Collect
10
How to Collect Clean Data Inside Your Workflow
Collect Clean Evidence Inside the Workflow
collect-clean-data-at-the-source
Reporting
Embed
10
Write each funder's report with AI, then check it
Write each funder's report with AI, then check it
assistant-writes-the-funder-report
Reporting
Funded partner road
10
Draft from approved sources, check the claims, and save an accountable report version. Bring the brief from the previous lesson.
Program managers, grant leads and reporting teams
How to Analyze Longitudinal Survey Data
Analyze longitudinal survey data
analyze-longitudinal-survey-data
Feedback
Read
11
Build a continuing analysis record with clear time scales, observed trajectories and limits.
How to Turn a Job Description into a Requirements Checklist
Clarify employer requirements
job-description-requirements-checklist
Case
Social Enterprise Track
11
Review applications without reviewer bias: one rubric, read on arrival, people decide
Review Without Reviewer Bias
review-applications-without-reviewer-bias
Grant
Analyze
11
How to Keep Impact Reporting Numbers Consistent
Get Stable Results From Governed Data
same-numbers-every-time
Reporting
Read
11
Compare your results with outside data: live queries and public datasets
Compare your results with outside data: live queries and public datasets
compare-with-outside-data
Reporting
Toolkit
11
How do you analyze a batch of grant applications?
Analyze a Whole Round
how-to-analyze-a-batch-of-grant-applications
Grant
Analyze
12
How to Measure Outcome Duration and Drop-Off
Measure outcome duration and drop-off
measure-outcome-duration-drop-off
Feedback
Read
12
Build a dated outcome claim, distinguish missingness from outcome loss and test forecast assumptions.
How to Score Candidate–Role Matches with a Clear Rubric
Review candidate–role evidence
score-candidate-role-matches-without-bias
Case
Social Enterprise Track
12
Make every number in your report match its source
Make every number in your report match its source
where-every-number-came-from
Reporting
Toolkit
12
How to Report Job Placements to Impact Investors
Turn a Cohort into a Social-Enterprise Investor Report
job-placement-investor-impact-report
Case
Social Enterprise Track
13
How do you track reviewer conflicts of interest?
Track Reviewer Conflicts of Interest
track-conflicts-of-interest-audit
Grant
Analyze
13
How to Write a Donor Report: Format, Evidence and Example
Design a Report for a Real Funding Decision
donor-report-funders-trust
Reporting
Decide
13
Build a report brief and claim-and-evidence table before drafting. Explain delivery, outcomes, spending, limitations and next actions.
Program managers, grant leads and reporting teams
How to put a credible dollar value on your results
How to put a credible dollar value on your results
credible-dollar-value-on-impact
Reporting
Toolkit
13
Prepare a valuation brief. Decide what the evidence supports, what needs more work, and when an outcome account is enough.
Program, evaluation and investment teams considering social-value estimates
AI Data Access Controls: What Your Assistant May See
Control what the assistant can access
what-the-assistant-may-see
Feedback
Prove
13
Define task-specific access, test synthetic records and verify report-sharing boundaries.
How to calculate the SROI ratio, step by step: value map, financial proxies and adjustments
How to calculate the SROI ratio, step by step: value map, financial proxies and adjustments
how-to-calculate-the-sroi-ratio
Reporting
Toolkit
14
Build a reproducible SROI calculation, test its assumptions and explain the result in a reviewed report.
Evaluation and reporting teams reviewing an SROI calculation
How to Write an Evidence-Based Impact Narrative for a Funder Report
Write a cited impact narrative
impact-narrative-funder-report-cited
Feedback
Prove
14
Build and check a report paragraph using a claim-and-source table, appropriate quotations and clear limitations.
How Do You Read a Grantee Report?
Read a Grantee Report
read-grantee-report-multi-signal
Grant
Analyze
14
How Do You Compute Grantee Variance?
Compute Grantee Variance
how-to-compute-grantee-variance
Grant
Analyze
15
Keep a person’s history connected across programs and staff changes
Follow one person over time
one-person-followed-for-years
Feedback
Shapes
15
Build a participant record that preserves episodes, dates, versions and missingness across repeated collection.
How Do You Build an SROI Value Map?
Build an SROI Value Map
how-to-build-an-sroi-value-map
Reporting
Optional method
16
Build a first value map, keep missing evidence visible, and give each unresolved outcome a next action.
Evaluation, program and investment teams preparing an SROI analysis
How do you track budget, invoices and actual spend for a grant?
Track Budget vs Actual Spend
how-to-track-budget-invoices-actual-spend
Grant
Analyze
16
Multi-Rater Feedback: Connect Perspectives and Protect Context
Connect several perspectives on one person
several-people-describing-one-person
Feedback
Shapes
16
Design subject-rater relationships, reporting rules and a tested multi-perspective feedback record.
How Do You Pick a Financial Proxy for SROI?
Pick a Defensible Financial Proxy
how-to-pick-a-financial-proxy-for-sroi
Reporting
Optional method
17
Compare candidate valuation sources and document why one fits your outcome, stakeholder and reporting period.
Evaluation and reporting teams selecting financial proxies
How Do You Analyze Grantee Reporting Longitudinally?
Analyze Grantee Reporting Over Time
analyze-grantee-reporting-longitudinal
Grant
Analyze
17
Cross-Program Reporting: Combine Results Without Losing Meaning
Combine evidence across programs
many-programs-one-picture
Feedback
Shapes
17
Build a defensible cross-program result with comparable measures, correct denominators and documented exclusions.
How to read Form 990 for a grant review
Read a 990 for Compliance
how-to-read-a-990-for-compliance
Grant
Analyze
18
Pick One Question. Keep Every System You Have.
Run a member-network survey
member-network-survey
Feedback
Shapes
18
Design and test a member reporting cycle with continuing records, coverage checks and authorized results.
Ask your whole grant round anything (Assistant + MCP)
Ask Your Whole Grant Round Anything
ask-your-grant-round-anything
Grant
Analyze
19
How do you build a grant audit trail?
Build a Grant Audit Trail
grant-audit-compliance-trail
Grant
Communicate
20
How do you produce grant compliance and regulatory reports?
Produce Compliance Reports
grant-compliance-regulatory-reports
Grant
Communicate
21
How Do You Roll Grantees Into a Board Report?
Roll Grantees Into a Board Report
roll-grantees-funder-board-report
Grant
Communicate
22
How Do You Build Grant Dashboards and Maps?
Grant Dashboards & Maps
grant-dashboards-geographic-mapping
Grant
Communicate
23