Build one governed definition for every program-data field — its meaning, type, unit, allowed values, missing-value rule, owner, collection point, validation rule, and version — so forms and reports remain comparable across programs and years.
SOPACT ACADEMY · DATA DICTIONARY
Watch (1:26): why one governed definition — population, unit, period, boundary, calculation — turns four framework reports into four views of the same dictionary.
In short: Build a data dictionary by inventorying the fields you already collect, consolidating duplicates, and documenting each one once — a stable ID, human-readable label, precise definition, data type, unit, permissible values, missing-value rule, collection point, calculation method, owner, validation rule, and version. For a multi-program organization the dictionary becomes the shared source for intake, surveys, case records, follow-ups, and reporting: programs use different subsets of fields but never independently redefine the same indicator. Map fields to a Theory of Change outcome, a Logframe indicator, or an IRIS+ metric where a defensible relationship exists — not for every operational field. Then version changes going forward without rewriting historical records.
This guide is for a program lead, MEL/evaluation manager, or data administrator at a nonprofit, funder, or multi-site program who needs one indicator to mean one thing across programs, sites, and reporting years. It shows the complete artifact — with a downloadable template — not just the idea.
Watch one application of the method: using an IRIS+-aligned data dictionary to support impact reporting (SoPact). It demonstrates the reporting workflow, not a field-by-field dictionary build.
AI extraction and API integration come later, as applications of the dictionary — not part of building it.
In short: A data dictionary is a governed catalog of the data fields an organization collects and what each field means. For every field it records a stable identifier, label, precise definition, data type, unit, permissible values, collection point, source, owner, validation rules, applicable framework mappings, and version history — which makes the field unambiguous but does not yet make the answer readable: knowing that a confidence score is an integer from 1 to 5 still will not tell you whether a 3 is a starting point or a fall from 5, and that reading comes from context the dictionary does not hold. (Data-element documentation guidance — e.g. NIH/FITBIR and the ISO/IEC 11179 metadata-registry concept — specifies that a data element carry a name, a precise definition, and permissible values: NIH data-element guidance.)
| Artifact | What it does | Scope difference |
|---|---|---|
| Data dictionary | Defines each field's meaning, format, rules, owner, and version | — |
| Codebook | Documents how one survey/dataset's variables and response codes were recorded | One dataset, not a cross-program catalog |
| Data catalog | Helps people discover and govern data assets across systems | Broader scope; may contain or link to field-level dictionary metadata |
Watch (6:07): where the dictionary stops and context starts — the four layers, data, framework, audience and expression, that turn a correctly defined field into an answer someone can act on.
In short: A field name and type are not enough to interpret a field consistently. A practical impact data dictionary carries these columns for every field:
| Column | Purpose |
|---|---|
| Field ID | Stable machine-readable identifier |
| Display label | Human-readable form label |
| Definition | Exact meaning of the field |
| Raw or derived | Collected directly, or calculated from other fields |
| Parent / source fields | For a derived field, the fields it is computed from |
| Data type | Integer, decimal, date, category, text, (nullable) Boolean |
| Unit | People, dollars, hours, percentage, score |
| Allowed values | Valid categories or range |
| Missing-value rule | Unknown, not applicable, declined, not collected |
| Validation rule | Conditions the value must satisfy before acceptance |
| Population | Participant, household, organization, employee |
| Applicable programs | Programs, sites, or services that use the field |
| Requirement status | Required, optional, conditional, not applicable |
| Applicability condition | Rule governing when the field appears |
| Collection point | Application, intake, exit, follow-up |
| Reporting period | Monthly, quarterly, annual, cohort-based |
| Calculation method | Formula for derived indicators |
| Disaggregation | Gender, geography, age, program, cohort |
| Source | Survey, CRM, document, administrative system |
| Owner | Person/team responsible for the definition |
| Sensitivity classification | Public, internal, confidential, restricted (access class) |
| Personal-data flag | Whether the field holds personal / special-category data |
| External mapping | IRIS+, Logframe, SDG, funder metric (where applicable) |
| Version · Effective date · Status | Current version, when it took effect, and Draft/Approved/Deprecated/Retired |
In short: The table below shows the core columns for readability. Download the complete template to see every governance, access, mapping, and version field.
| Field ID | Label | Type | Allowed / unit | Population | Applicability | Validation | Owner | Ver. |
|---|---|---|---|---|---|---|---|---|
participant_id | Participant ID | Text | UUID | Participant | Always | Unique; non-null | Data admin | 1.0 |
employment_90d | Employed within 90 days | Nullable Boolean | Yes / No | Participant | When follow-up done | Requires follow-up date | MEL lead | 1.0 |
hourly_wage_90d | Hourly wage at 90 days | Decimal | ≥0; currency/hr | Participant | Only if employment_90d = Yes | Numeric; currency code | MEL lead | 1.0 |
financial_security_theme | Financial-security theme | Category | Positive/Negative/Mixed/Not present | Participant | When narrative provided | Human-reviewed AI coding | Eval. lead | 1.0 |
One complete record, every attribute — easier to read than a 20-column row, and it shows the governance fields the compact table omits:
| Field ID | employment_90d |
| Definition | Whether a program completer held paid employment at the 90-day follow-up |
| Raw / derived | Raw |
| Data type | Nullable Boolean |
| Allowed values | Yes, No |
| Missing-value reason | Unknown, Not collected, Declined, Not applicable |
| Population | Program completers |
| Applicable programs | Workforce tracks |
| Requirement · applicability condition | Conditional — when the 90-day follow-up is completed |
| Collection point · period | Follow-up · cohort |
| Validation rule | Follow-up date required; value in {Yes,No} |
| Source · owner | Follow-up survey · Program MEL lead |
| Sensitivity · personal-data flag | Confidential · No |
| External mapping | Verify applicable IRIS+ metric |
| Version · effective date · status | 1.0 · 2026-01-01 · Approved |
Download the impact data-dictionary template — a spreadsheet with an instructions sheet, all columns above, four example rows, and blank rows to fill. (Ships with this chapter as .xlsx and .csv.)
In short: A Boolean has two logical values. "Unknown" is a missingness state, not a third Boolean — so store the value as Yes/No and capture the reason it's absent in a separate missing-value field. Prefer a null value plus a documented missingness reason (or a clearly governed categorical status). Never treat zero as missing, don't mix "not applicable" with "not collected," and avoid arbitrary numeric codes like -99 without metadata.
In short: Applicability — which programs and populations use a field, at which stage, required or conditional — belongs in the dictionary. Access — who may view, enter, or edit it, and whether it holds personal data — is an access-control feature around the system. Record both; don't collapse them into one "scope."
Applicability can be conditional: hourly_wage_90d applies only when employment_90d = Yes, which is exactly what a form generator needs. On access, follow least privilege — a wage field is restricted to roles authorized under the organization's privacy and data-governance policy; finance may receive aggregated wage analysis without access to identifiable participant records.
In short: This is a Sopact field-governance test, not a recognized external data-management standard. Framework mapping counts only where applicable; valid operational fields (participant ID, consent status, intake date, referral source) never map to a Theory of Change or IRIS+.
When a field grades Red or Amber, fix the shared definition — not just the one form — so every program using it inherits the correction.
In short: Use a new version of the same field only for changes that preserve semantic comparability — a clarified definition, corrected allowed values, non-substantive metadata. If a change materially alters the construct, population, unit, time window, or calculation, create a new field ID and retire or retain the old one.
"Employed at exit" and "employed within 90 days" are different measures — different time windows answering different questions — so they are separate fields, not versions of one:
| Field ID | Definition | Status |
|---|---|---|
employment_at_exit | Paid employment on the exit date | Retained or deprecated |
employment_90d | Paid employment at the 90-day follow-up | New field |
A true version change, by contrast, preserves the construct — for example tightening the allowed values on the same field:
| Version | Change to employment_90d |
Effective date | Historical treatment |
|---|---|---|---|
| 1.0 | Allowed values: Yes / No / Unknown | Jan 1, 2026 | Preserve as reported |
| 1.1 | Nullable Boolean (Yes/No) with "Unknown" moved to missing-value reason | Jul 1, 2026 | Comparable to 1.0; remap "Unknown" to missing |
In short: Reusing the same field label is necessary for comparability but not sufficient. Two sites' placement rates are meaningfully comparable only when they share the same approved definition, denominator, follow-up window, calculation method, and missing-data rule.
Comparability also depends on eligibility, whether self-employment or part-time counts, evidence source, and reporting-period boundaries. The dictionary fixes all of those on the definition, so "Job Placement Rate" means one thing — not merely hand two sites the same field name.
In short: AI can draft dictionary fields from existing forms and code open-text evidence into defined fields — but a narrative should never be silently converted into a quantitative value. Distinguish extracted facts, coded qualitative themes, and model-generated interpretations, and route interpretations to human review.
A participant writes, "This was my first paycheck where I wasn't choosing between rent and groceries." AI may tag it as financial security or perceived economic well-being — but it should not infer a wage amount or populate hourly_wage_90d. That value has to come from a wage question. The dictionary keeps the three outputs separate:
| Output | Example | Treatment |
|---|---|---|
| Explicit fact | "I now earn $22 per hour" | Extract as a candidate wage value; confirm against the wage field |
| Qualitative theme | "I can now afford groceries" | Code as a financial-security theme |
| Interpretation | "Participant appears more confident" | Store as an AI inference requiring human review |
Note the coded theme's own missing states: Not present means the text was reviewed and the theme was absent — different from no narrative provided, not analyzed, or analysis failed, which are missing-value states.
| Dictionary task | Sopact function | Human decision |
|---|---|---|
| Inventory fields | Import existing forms and data | Decide which fields are authoritative |
| Consolidate definitions | Shared field library | Approve merges and definitions |
| Define valid values | Types and validation rules | Set acceptable values |
| Reuse across programs | Program forms using shared fields | Decide applicability |
| Code narrative evidence | Intelligence Row candidate extraction & themes | Review interpretations |
| Manage changes | Version & effective-date controls | Approve changes |
| Report consistently | Traceable analysis & reporting | Interpret findings |
Once the internal definitions are approved, the next step is to map portfolio fields to IRIS+, GRI, and ESRS — begin mapping only after population, unit, period, boundary, and calculation are defined.
Inventory existing fields, consolidate duplicates and assign stable IDs, define meaning and format (type, unit, permissible values, missing-value rule, validation rule), document collection context (population, applicability, source, timing, calculation, disaggregation), assign governance (owner, sensitivity, access, status), map to outcomes or standards where defensible, and version changes going forward without rewriting historical records.
At minimum: field ID, label, definition, raw/derived, data type, unit, allowed values, missing-value rule, validation rule, population, applicable programs, requirement status, applicability condition, collection point, reporting period, calculation method, disaggregation, source, owner, sensitivity classification, personal-data flag, external mapping (where applicable), version, effective date, and status.
Version the same field only for changes that preserve the construct — a clearer definition, corrected allowed values, non-substantive metadata. Create a new field ID whenever a change alters the construct, population, unit, time window, or calculation (for example, "employed at exit" vs. "employed within 90 days" are separate fields).
Store the real value and record absence separately: prefer a null plus a documented missingness reason (unknown, not applicable, declined, not collected), or a clearly governed categorical status. Never treat zero as missing, don't mix "not applicable" with "not collected," and avoid arbitrary numeric codes like -99 without metadata.
A data dictionary defines each field's meaning, format, rules, owner, and version across the organization. A codebook documents how one survey or dataset recorded its variables and response codes. A data catalog helps people discover and govern data assets across systems, and may contain or link to field-level dictionary metadata — the distinction is scope.
Yes, as a draft. AI can inventory fields and propose IDs, definitions, and types, and code open-text evidence into defined fields. But a human owns the definitions, and AI-generated interpretations — not just extracted facts — must be reviewed before they populate governed fields. Never let AI infer a quantitative value from narrative.
Author: Sopact (Unmesh Sheth). Published Aug 2026; last reviewed Aug 2026. Green/Amber/Red is Sopact's practical field-governance test, not an external data-management standard.
ChatGPT, Claude, and Gemini are fine for a quick test — but not for an answer you'll put in front of a funder or board. When it has to hold up, run it in Sopact Sense.
Try it in Sopact →
