Secondary data analysis: where to find quality public data, how to validate before reuse, and how to integrate it with primary data via persistent IDs.
A practical guide to secondary data analysis
Secondary data offers speed and scale that primary collection cannot match. The trade-off is fit: someone else collected it for a different purpose. This guide covers where to find quality sources, how to validate before reuse, and how to integrate secondary with primary records at the participant level.
Reading time: 14 minutes · Updated May 14, 2026 · Part of the stakeholder intelligence series
Definition, without the textbook
Secondary data analysis is the practice of reusing data someone else collected, to answer a question different from the one it was originally collected for. Government statistics, peer-reviewed research, industry reports, internal administrative records, and published datasets all qualify. The defining property is reuse: the data already exists, and the current researcher is putting it to a new use. The opposite is primary data, which is collected directly for the current research question.
Five categories of secondary data sources
Category · 01
Government statistics
Census, BLS, HUD, CDC, federal and state agencies. The largest source by far and the most rigorously documented.
Category · 02
Administrative records
School districts, public health departments, court records. Operational data made available for research.
Category · 03
Peer-reviewed research
Academic studies, meta-analyses, archived datasets. Useful for prior effect-size estimates and methodology references.
Category · 04
Industry benchmarks
Trade association reports, sector dashboards, commercial market research. Useful for context; check for sponsor bias.
Category · 05
Your own historical records
Past cohort data, prior surveys, archived program records. Secondary to today's question even though you collected it.
Three properties separate a reusable source from one that produces misleading conclusions. Documented methodology: every variable has a definition, every sample has a frame, every metric has an update cadence. Without documentation, you cannot judge whether the source fits your question.
Geographic granularity: aggregate national statistics rarely match the participant population of a typical foundation program. Census tract, ZIP code, community area, county, MSA · the finer the granularity, the more useful the source becomes for context that actually fits.
API or MCP access: a source that requires downloading a CSV from a portal each quarter is operationally fragile. Sources with documented APIs (and, increasingly, MCP servers) integrate into a recurring evaluation workflow. Census, BLS, HUD, and major city portals all qualify.
The five sources worth knowing
Five sources cover most foundation evaluation work in the United States. Each one is publicly accessible, has documented methodology, and is increasingly available through MCP servers for AI-driven query. The table below names what each source carries, the geographic granularity it supports, the update cadence, and the MCP integration available today.
| Source | What it carries | Geographic granularity | Update cadence | MCP / API access |
|---|---|---|---|---|
| US Census Bureau | ACS demographics, household income, housing, employment, language, education | Block, tract, county, MSA, state | ACS 5-yr annual; decennial every 10 yr | Official MCP server on GitHub (uscensusbureau) |
| Bureau of Labor Statistics | Employment, unemployment, wages, occupation, industry | County, MSA, state, national | LAUS monthly; QCEW quarterly | Public API, MCP wrappers available |
| HUD User | Fair Market Rents, Income Limits, CHAS data, USPS Crosswalk | ZIP, tract, county, MSA, congressional district | Annual for most series | Free bearer token, 60 req/min |
| City open data portals | Local indicators: housing permits, public health, education, transit, equity | ZIP, community area, ward, neighborhood | Varies by city and dataset | OpenGov MCP server (Chicago, NYC, SF, LA) |
| World Bank Open Data | Development indicators: GDP, poverty, education, health, gender | Country, region, income group | Annual for most indicators | Public API, no key required |
★ Real MCP servers, open-source on GitHub. See the Census MCP repo and the OpenGov MCP repo for setup.
Bridge dimensions captured at intake. Secondary reference tables alongside primary records. Claude Code queries both layers in one prompt, with audit-ready provenance for every joined number.
Validation before reuse
Secondary data is rarely refused. It is over-trusted. A dataset arrives with a column that sounds right, the analysis moves on, and the report ships before anyone notices that "employed" meant something different in the source than in the program. The four-point check below catches most of these failures before they reach the published report.
Check 01 · Origin
who collected it, when, and why
The collector, the funder, the original purpose. Government agencies collecting tax data have different incentives than a trade association reporting on its own industry. Both can be useful; the bias is different.
Name the source organization, the original study or program, the publication date, and the funder if known. A reader should be able to retrace your trust chain.
Check 02 · Definitions
key variables and their meaning
How "employed" is defined. How "poverty" is calculated. How "low-income" is bounded. Definitions vary across agencies and across years within the same agency.
The definition is copied into your data dictionary, alongside the primary definition. If they differ, note the gap and the implication.
Check 03 · Time period and frequency
when the data covers, when it updates
The coverage period, the publication lag, and the update cadence. ACS 5-year estimates centered on 2021 are not current as of 2026. BLS LAUS is monthly with a one-month lag and is current enough for most quarterly analyses.
Vintage stamp on every joined number. If the lag exceeds two years, flag it explicitly in the methodology appendix.
Check 04 · Sample and exclusions
who is in the data, who is not
The sampling frame, the non-response rate, the excluded subgroups. Census ACS covers households; institutional populations are excluded. BLS Current Population Survey covers civilian non-institutional adults.
The frame is named explicitly. If your participant population includes groups that the source excludes (e.g., recently incarcerated, undocumented), the gap is flagged.
Worked example · the secondary side
The same TechBridge Chicago equity analysis from the primary data hub, walked from the secondary side. The primary data is in Sopact Sense. The question is whether the workforce program is reaching the highest-need community areas. The secondary source needs to provide community-area-level income context for 80 participant ZIP codes. The steps below show source selection, validation, and the join logic that pairs secondary with primary.
Three candidates carry community-area or tract-level income for Chicago. The Chicago Data Portal publishes "Per Capita Income by Community Area". The US Census Bureau publishes ACS variable B19013_001E (median household income) at the tract level. HUD's USPS Crosswalk bridges ZIP codes to either geography.
| Candidate source | Variable | Granularity | Vintage | Access |
|---|---|---|---|---|
| Chicago Data Portal | Per capita income | Community area (77 areas) | Updated annually | OpenGov MCP (no key) |
| US Census Bureau (ACS) | B19013_001E · median household income | Census tract | ACS 2023 5-yr | Official Census MCP |
| HUD User | USPS Crosswalk: ZIP → tract → community area | ZIP, tract, community area | Refreshed quarterly | HUD User API (bearer token) |
| Validation check | Finding | Pass |
|---|---|---|
| Origin | City of Chicago, Department of Public Health & Department of Planning. Republished from ACS by community area. | ✓ |
| Definition | "Per capita income" matches ACS Bureau definition. Aligned with B19301_001E. | ✓ |
| Time period | Latest vintage 2022 ACS 5-yr (centered 2018-2022). Lag is 3 years; acceptable for income context but flag in report. | ~ flagged |
| Sample / exclusions | ACS household sample. Institutional populations (correctional facilities, dorms) excluded. Cohort 2026 includes some recently-housed participants; gap noted. | ~ flagged |
The participant primary record carries zip_code.
The secondary income table carries community_area.
The HUD crosswalk maps between them. The join is a three-table operation: primary →
crosswalk → secondary. The result attaches community-area income to each participant
record.
For the full join walked from the primary side, including the disaggregated result and the insight Claude returns, see the TechBridge Chicago worked example on the primary data hub. The two pages describe the same analysis from opposite angles.
Architecture for integration
Secondary data does not integrate with primary by being copied into the same database. It integrates by being queryable from the same analytical workspace, via consistent interfaces. Primary lives at the participant level inside Sopact Sense. Secondary lives in reference tables, accessed via MCP servers or APIs at query time. Claude Code, BI tools, and notebooks see both surfaces and join them as if they were one table.
Primary inside Sopact
Participant-level, persistent
Secondary via MCP / API
Reference tables, aggregated
The persistent layer holds primary. The MCP layer queries secondary. Claude Code reads both.
Where secondary analysis breaks
Each mistake is technically computable and substantively misleading. Definition mismatch, outdated data treated as current, overgeneralization from narrow contexts, and missing provenance in the published report. All four are easy to commit and easy to prevent with validation discipline.
Mistake 01 · Definition mismatch
the column name is the same; the meaning is not
Census "poverty" uses federal thresholds. Your program's "low-income" criterion is 80% of area median income. The two are different categories, but the labels look interchangeable in the joined table.
The data dictionary names both definitions side by side. The join uses one consistent definition. The methodology appendix states which.
Mistake 02 · Outdated treated as current
two-to-three-year lag passed off as today
ACS 5-year estimates from 2023 are published in late 2024 and cover 2019-2023. Used in a 2026 report as "current," the data is three to seven years old at the tail. Labor market and rent shifts since are not reflected.
Vintage stamped on every number. For metrics that change quickly (rent, employment), use higher-frequency sources (BLS LAUS monthly) rather than ACS multi-year.
Mistake 03 · Overgeneralization
narrow context applied broadly
A peer-reviewed study on an urban youth program in three cities is cited as a benchmark for a rural adult workforce program. Effect sizes from the urban context do not transfer.
Source's population and context named explicitly in the citation. If the source does not match your population, either find one that does or acknowledge the limitation in the report.
Mistake 04 · No provenance in the report
reader cannot verify the number
The published report names regional unemployment at 6.2% without naming the source, the geography, or the period. A reader cannot tell if the number is national, state, MSA, or local; current or two years old.
Every secondary figure carries an inline citation: source, geography, period, vintage. The methodology appendix lists each secondary source in one table for quick verification.
Trade-offs that drive the design choice
Secondary data has three structural advantages and three real costs. The advantages: speed, scale, and cost. The costs: fit, currency, and methodology lock-in. Strong evaluation treats secondary as context for primary, not as a substitute. The disadvantages are reasons to validate before reuse, not reasons to avoid the source.
Advantage 01
Speed of access
The data already exists. Hours from question to data, not weeks. For context, baselines, and benchmarks, secondary access is the fast lane.
Disadvantage 01
Fit is approximate at best
Variables were defined for someone else's question. Definitions, sampling frames, and granularity rarely match yours exactly. Documentation gap is the largest risk.
Advantage 02
Scale of population coverage
Census ACS covers every household. BLS LAUS covers every county. A single primary collection cannot reach that scale. Secondary brings population-level context.
Disadvantage 02
Currency lag
ACS 5-year is two years lagged at publication; sector reports are often older. For fast-moving metrics, secondary is too stale. Match cadence to question.
Advantage 03
Lower marginal cost
Someone else paid for collection. Your cost is access and validation time. For most government sources, access is free; the only cost is methodology review.
Disadvantage 03
Methodology lock-in
You cannot change the sampling frame, the definitions, or the time period. You inherit them. Where the inheritance fails your question, you have to triangulate with primary or change the question.
For the head-to-head decision and the hybrid pattern, see the primary vs secondary data guide. For collecting primary data from scratch, see the primary data hub.
Frequently asked questions
Secondary data analysis is the practice of reusing data that someone else collected for a different purpose. Government statistics, peer-reviewed research, industry reports, internal administrative records, and published datasets all qualify. The defining property is reuse: the data already exists, and the current researcher is using it to answer a question different from the one it was collected for.
Five sources cover most applied work. The US Census Bureau (ACS, decennial, demographic surveys) for population and demographic data. The Bureau of Labor Statistics (LAUS, QCEW, occupation data) for employment and wages. HUD (Fair Market Rents, CHAS, USPS Crosswalk) for housing and geography. City open data portals (data.cityofchicago.org, data.cityofnewyork.us) for local context. The World Bank Open Data API for international comparisons. Each one is publicly accessible and increasingly available via MCP servers for AI queries.
BLS regional employment statistics reused to baseline a workforce program. ACS median household income at the census tract level used to contextualize program outcomes by neighborhood. HUD Fair Market Rents used to evaluate cost-of-living burden. A peer-reviewed study on similar programs reused to estimate expected effect size. Your own customer transaction records reused to study patterns the original system was not designed to surface. Each example shares the same structure: the data already existed, was collected for another reason, and is being reused for a new question.
Four checks. Origin: who collected the data, when, and for what original purpose. Definitions: how key variables are defined and whether they match your usage. Frequency and time period: when the data is updated and what time range it covers. Sampling and exclusions: who is in the sample and who is not. A dataset that passes all four checks is reusable; failing any one of them requires explicit limitation in the analysis.
Secondary data has no participant IDs, so the join cannot be on identity. It joins on shared dimensions: state, county, zip code, census tract, occupation code, age band, gender, year. The primary dataset aggregates to those dimensions, and the join becomes a structured query. A workforce program's primary data aggregates participant placement rates by state and occupation; BLS data provides the regional baseline at the same dimensions; the difference is attributable effect.
Three structural advantages. Speed: the data already exists. Scale: secondary sources often cover populations much larger than any single primary collection. Cost: someone else paid for the collection. The trade-off is fit: secondary variables rarely match primary questions perfectly, and the lag between collection and publication can run two to three years. Strong analysis treats secondary as context for primary, not as a replacement.
Three frequent errors. Assuming definitions match yours without verification (census poverty calculations may differ from your assessment criteria). Treating outdated information as current truth (public data often lags two to three years). Overgeneralizing from narrow source contexts (urban youth program results applied to rural adult services). Each error is technically computable and substantively misleading. Validation before reuse is the prevention.
Yes, increasingly via MCP servers. The US Census Bureau publishes an official MCP server on GitHub (uscensusbureau/us-census-bureau-data-api-mcp). HUD's APIs are accessible via MCP wrappers. The OpenGov MCP server (srobbin/opengov-mcp-server) covers any Socrata-powered city or state portal, including Chicago, NYC, San Francisco, and Los Angeles. Once configured, Claude Code can query these sources in natural language and join them with primary data exposed via the Sopact MCP interface.
Primary data analysis works at the participant level: each row is a person with full provenance back to the instrument and consent. Secondary data analysis works at the aggregate level: each row is a geography, demographic group, or time period. The two types analyze different questions, and the strongest evaluations combine them: primary to characterize the participants, secondary to characterize the counterfactual. The join produces attributable effect.
Currency requirement depends on the question. Demographic baselines (ACS 5-year estimates) at two years of lag are fine for most program evaluation. Labor market statistics (BLS LAUS) update monthly with a one-month lag and are acceptable for workforce comparisons. Pricing or capacity data needs to be within months. Long-term trend analysis can tolerate older sources. Always document the data's vintage explicitly in the final report so a reader can judge whether the lag affects the conclusion.
The full series
The integration pattern applied to grant management, training programs, impact portfolios, and nonprofit operations. The MCP integration walked through in depth, with worked examples across multiple secondary sources.
Read the stakeholder intelligence guide →Related guides in the cluster
Sibling guides for the primary/secondary cluster. Each one covers a different surface of the same persistent layer.
Hub
Primary data
Collection, analysis, and the TechBridge Chicago worked example with all four MCP sources joined in one prompt.
Comparison
Primary vs secondary data
The head-to-head decision and the BLS counterfactual that produces attributable effect.
Methods hub
Quantitative data analysis
The full methods catalog and the AI-era data layer thesis applied to numeric work.
Master pillar
Sopact Sense overview
The platform that holds the persistent layer. Architecture, capabilities, and integration patterns.
Engine pillar
Stakeholder intelligence
The category Sopact operates in. Why structured surveys alone produce snapshots rather than portraits.
Methods
Data collection methods
Seven methods compared, with the 2026 analysis-ready design lens.
Longitudinal
Longitudinal survey design
Persistent IDs, instrument versioning, and the cross-wave join pattern in depth.
Qualitative
Qualitative interview analysis
The five-stage workflow from transcript to outcome correlation with paired quant items.
AI tooling
AI data collection tools
Category map for AI-native data collection. What to look for, what to avoid.
Ready when you are
Bridge dimensions captured at intake. Secondary reference tables alongside primary records. MCP integration with Census, BLS, HUD, and city portals built into the analytical workspace. Provenance preserved from query to published report.
