play icon for videos
← Academy
SOPACT ACADEMY · SURVEY INTELLIGENCE · ANALYZE

How to Analyze Open-Ended Survey Responses

Good thematic analysis reconciles every count back to N — most don't. Here is the prompt that themes open-text, counts each theme, holds single mentions, and grades the result.

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1"> <style> @keyframes pulseDot{0%,100%{opacity:1}50%{opacity:.35}} *{box-sizing:border-box} body{margin:0;font-family:'Hanken Grotesk',-apple-system,BlinkMacSystemFont,sans-serif;background:#EEF2F9;padding:26px;display:flex;justify-content:center} .card{width:600px;max-width:100%;background:#FFFFFF;border:1px solid #E3E9F4;border-radius:20px;box-shadow:0 26px 64px -32px rgba(15,30,70,.34),0 2px 4px rgba(15,30,70,.05);overflow:hidden} .hdr{display:flex;align-items:center;gap:11px;padding:16px 20px;border-bottom:1px solid #EEF1F7;background:#F5F8FE} .hdr .ic{display:inline-flex;width:30px;height:30px;align-items:center;justify-content:center;border:2px solid #155DFC;border-radius:50%;color:#155DFC;font-size:15px;font-weight:800;flex-shrink:0} .hdr .t{min-width:0} .hdr .t .n{font-weight:800;font-size:14.5px;color:#0F172A;letter-spacing:-.2px} .hdr .t .s{font-size:12px;color:#64748B} .hdr .pill{margin-left:auto;display:inline-flex;align-items:center;gap:6px;font-size:11.5px;font-weight:700;color:#155DFC;background:#E6EEFF;padding:5px 10px;border-radius:20px;white-space:nowrap} .hdr .pill .d{width:6px;height:6px;border-radius:50%;background:#155DFC;animation:pulseDot 2s infinite} .tabs{display:flex;gap:0;padding:12px 16px 0;background:#F5F8FE} .tab{flex:1;display:flex;flex-direction:column;align-items:center;gap:5px;cursor:pointer;font-family:inherit;font-size:11.5px;font-weight:700;color:#64748B;background:transparent;border:none;border-bottom:2px solid transparent;padding:8px 4px 10px} .tab.on{font-weight:800;color:#155DFC;border-bottom-color:#155DFC} .tab .num{display:inline-flex;width:20px;height:20px;align-items:center;justify-content:center;border-radius:50%;font-size:11px;font-weight:800;color:#64748B;background:#EEF1F7} .tab.on .num{color:#fff;background:#155DFC} .divider{height:1px;background:#EEF1F7} .body{padding:20px} .eyebrow{font-size:10.5px;font-weight:800;letter-spacing:1.2px;text-transform:uppercase;color:#155DFC;margin-bottom:5px} .eyebrow.g{color:#15884F} .headline{font-family:'Inter',Georgia,serif;font-size:18.5px;font-weight:600;line-height:1.25;color:#0F172A;margin-bottom:12px} .lede{font-size:12.5px;line-height:1.55;color:#475569;margin:0 0 14px} .rows{display:flex;flex-direction:column;gap:8px;margin-bottom:14px} .row{display:flex;align-items:center;gap:11px;background:#F5F8FE;border:1px solid #EEF1F7;border-radius:11px;padding:11px 13px} .row .emoji{font-size:17px} .row .rt{flex:1;min-width:0} .row .rt .n{font-size:13px;font-weight:700;color:#0F172A} .row .rt .s{font-size:12px;color:#64748B} .pillsm{font-size:11px;font-weight:800;color:#155DFC;background:#E6EEFF;padding:4px 9px;border-radius:6px;flex-shrink:0} .pillsm.gray{font-weight:700;color:#64748B;background:#EEF1F7} /* chain / timeline */ .chainwrap{position:relative;padding-left:26px} .chainline{position:absolute;left:8px;top:8px;bottom:8px;width:2px;background:#E3E9F4} .cnode{position:relative;margin-bottom:14px} .cdot{position:absolute;left:-26px;top:4px;width:16px;height:16px;border-radius:50%;background:#fff;border:3px solid #94A3B8} .clabel{font-size:9.5px;font-weight:800;letter-spacing:.5px;text-transform:uppercase;color:#64748B;margin-bottom:4px} .cbox{background:#F5F8FE;border:1px solid #EEF1F7;border-radius:11px;padding:10px 12px;font-size:12px;line-height:1.5;color:#334155} .cbox.graded{display:flex;align-items:flex-start;gap:9px;border-left-width:3px;border-left-style:solid} .cnote{margin:0 0 14px 4px;font-size:11px;line-height:1.5} .cnote .plus{display:inline-flex;width:16px;height:16px;align-items:center;justify-content:center;background:#155DFC;color:#fff;border-radius:4px;font-size:10px;font-weight:800;flex-shrink:0;margin-right:6px} .csummary{display:flex;align-items:center;gap:12px;background:#0A1B3D;border-radius:13px;padding:11px 15px;margin-top:4px} .csummary .t{flex:1;font-size:11.5px;color:#CBD8F5} .csummary .t b{color:#F1F5FF} /* matrix */ .mxwrap{border:1px solid #EEF1F7;border-radius:12px;overflow:hidden;margin-bottom:10px} .mxhead{display:grid;grid-template-columns:118px 1fr 1fr;background:#F5F8FE;border-bottom:1px solid #EEF1F7} .mxhead div{font-size:9px;font-weight:800;letter-spacing:.5px;text-transform:uppercase;color:#64748B;padding:7px 9px} .mxrow{display:grid;grid-template-columns:118px 1fr 1fr;border-bottom:1px solid #EEF1F7} .mxrow:last-child{border-bottom:none} .mxrow>div{padding:9px;font-size:11.5px;line-height:1.4;color:#334155;display:flex;align-items:center;flex-wrap:wrap;gap:5px} .mxrow .mxlabel{font-weight:700;color:#0F172A;background:#FAFBFE} .mxchip{font-size:9px;font-weight:800;letter-spacing:.3px;text-transform:uppercase;padding:3px 6px;border-radius:5px;white-space:nowrap} .promptwrap{padding:0 20px 18px} .promptlbl{font-size:10px;font-weight:800;letter-spacing:1px;text-transform:uppercase;color:#94A3B8;margin-bottom:7px} .prompt{display:flex;align-items:flex-start;gap:9px;font-family:ui-monospace,'SF Mono',Menlo,monospace;font-size:11px;line-height:1.5;color:#1E293B;background:#F5F8FE;border:1px solid #EEF1F7;border-left:3px solid #155DFC;border-radius:9px;padding:11px 12px} .prompt .car{color:#155DFC;font-weight:800;flex-shrink:0} .flagrow{display:flex;align-items:flex-start;gap:10px;background:#F5F8FE;border:1px solid #EEF1F7;border-radius:11px;padding:10px 12px;border-left-width:3px;border-left-style:solid;margin-bottom:8px} .flagtag{font-size:9.5px;font-weight:800;letter-spacing:.4px;text-transform:uppercase;padding:4px 8px;border-radius:5px;white-space:nowrap;flex-shrink:0} .flagtxt{font-size:12px;line-height:1.45;color:#334155} .lp{display:flex;align-items:center;gap:12px;background:#0A1B3D;border-radius:13px;padding:14px 16px;margin-top:6px} .lp .ic{display:inline-flex;width:34px;height:34px;align-items:center;justify-content:center;background:#14295C;border:1px solid #26407A;border-radius:9px;color:#7FA8FF;font-size:16px;flex-shrink:0} .lp .rt{flex:1;min-width:0} .lp .rt .n{font-size:13px;font-weight:800;color:#F1F5FF} .lp .rt .s{font-size:11.5px;color:#9DB2D9} .lp .open{font-size:11px;font-weight:800;color:#0A1B3D;background:#7FA8FF;padding:6px 11px;border-radius:7px;white-space:nowrap} </style> </head> <body> <div class="card"> <div class="hdr"> <span class="ic">&#128172;</span> <div class="t"> <div class="n">Open-Text Barriers · thematic analysis of open-ended survey responses</div> <div class="s">DEMO-04 · Sopact Sense · Workforce Program (Barriers Survey)</div> </div> <span class="pill"><span class="d"></span>Theming</span> </div> <div class="tabs" id="tabs"></div> <div class="divider"></div> <div class="body" id="body"></div> <div class="promptwrap"> <div class="promptlbl">The prompt behind this step</div> <div class="prompt"><span class="car">&gt;</span><span id="promptTxt"></span></div> </div> </div> <script> (function(){ var TABS = ["Setup", "Theme", "Fix", "Report"]; var CONTENT = [{"eyebrow": "Step 1 · Load the dataset", "headline": "Clean data, persistent contact IDs, one Decision Brief.", "lede": "The Assistant works only from the Open-Text Barriers dataset (DEMO-04) already loaded in Sopact Sense — nothing outside the record.", "prompt": "You are the Sopact Sense Assistant working over the [DEMO] dataset (clean data + persistent contact IDs). Load my Decision Brief (decision, audience, outcomes, indicators, evidence standard) first, then wait for my task."}, {"eyebrow": "Step 2 · Theme the open-ended question", "headline": "Do the counts reconcile to N?", "lede": "Every theme is named, counted, and graded <strong style=\"color:#15884F\">green</strong>, <strong style=\"color:#C9820A\">amber</strong>, <strong style=\"color:#C63E2B\">red</strong>.", "prompt": "Analyse all responses to [QUESTION]: themes, definitions, counts, 2 quotes each, 'other', prevalence reconciled to N. Hold single mentions. Grade green/amber/red."}, {"eyebrow": "Step 3 · One-fix follow-up", "headline": "Handle the 'other'.", "lede": "The Assistant takes the lowest-graded element — Other bucket, 1 ambiguous — and names the single indicator that would move it to green.", "prompt": "Take the lowest-graded element above and fix it using only what the program could realistically measure. Show the before → after grade and the single indicator/edit that moves it to green."}, {"eyebrow": "Step 4 · Missing & incomplete report", "headline": "Survey — Thematic Analysis Report", "lede": "Every amber or red item becomes one named ask — the report leads with the decision it informs.", "prompt": "Create a 'missing & incomplete' report from this analysis in Sopact branding [or paste your website URL / brand guideline to apply your own]. List every element graded amber or red, what is missing, and the one input that fixes each. Lead with the decision this report informs."}]; var BODIES = { setup: "<div class=\"row\"><span class=\"emoji\">&#128221;</span><div class=\"rt\"><div class=\"n\">Representative response</div><div class=\"s\">&quot;No reliable transport — the bus doesn&#x27;t run early enough.&quot; (transport + childcare recur across responses)</div></div><span class=\"pillsm\">Pasted</span></div><div class=\"row\"><span class=\"emoji\">&#128203;</span><div class=\"rt\"><div class=\"n\">DEMO-04 · Open-Text Barriers</div><div class=\"s\">1 Question · 18 Responses · Active</div></div><span class=\"pillsm gray\">Structure</span></div><div class=\"row\"><span class=\"emoji\">&#128274;</span><div class=\"rt\"><div class=\"n\">No hallucination rule</div><div class=\"s\">Only what the program states; infer = flagged</div></div><span class=\"pillsm gray\">Required</span></div>", build: "<div class=\"mxwrap\"><div class=\"mxhead\"><div></div><div>Count</div><div>% · quote</div></div><div class=\"mxrow\"><div class=\"mxlabel\">Transport</div><div>5<span class=\"mxchip\" style=\"background:#15884F;color:#fff\">GREEN</span></div><div>28% · &quot;bus doesn&#x27;t run early&quot;</div></div><div class=\"mxrow\"><div class=\"mxlabel\">Childcare</div><div>4</div><div>22% · &quot;no childcare&quot;</div></div><div class=\"mxrow\"><div class=\"mxlabel\">Other</div><div>1<span class=\"mxchip\" style=\"background:#C9820A;color:#fff\">AMBER</span></div><div>6% · ambiguous</div></div><div class=\"mxrow\"><div class=\"mxlabel\">Language</div><div>1<span class=\"mxchip\" style=\"background:#C63E2B;color:#fff\">RED</span></div><div>6% · hold</div></div></div>", fix: "<div class=\"mxwrap\"><div class=\"mxhead\"><div></div><div>Count</div><div>% · quote</div></div><div class=\"mxrow\"><div class=\"mxlabel\">Transport</div><div>5<span class=\"mxchip\" style=\"background:#15884F;color:#fff\">GREEN</span></div><div>28% · &quot;bus doesn&#x27;t run early&quot;</div></div><div class=\"mxrow\"><div class=\"mxlabel\">Childcare</div><div>4</div><div>22% · &quot;no childcare&quot;</div></div><div class=\"mxrow\"><div class=\"mxlabel\">Other</div><div>GREEN · classifiable<span class=\"mxchip\" style=\"background:#15884F;color:#fff\">GREEN</span></div><div>6% · ambiguous</div></div><div class=\"mxrow\"><div class=\"mxlabel\">Language</div><div>1<span class=\"mxchip\" style=\"background:#C63E2B;color:#fff\">RED</span></div><div>6% · hold</div></div></div><div class=\"cnote\" style=\"color:#15884F;margin-left:0\"><span class=\"plus\">+</span><strong>Fix applied:</strong> Add a clarifying sub-question next round</div>", reportFlags: "<div class=\"flagrow\" style=\"border-left-color:#C9820A\"><span class=\"flagtag\" style=\"color:#C9820A;background:#FBF0DA\">AMBER</span><div class=\"flagtxt\">The &#x27;Other&#x27; bucket holds one ambiguous response that can&#x27;t be classified as written.</div></div><div class=\"flagrow\" style=\"border-left-color:#C63E2B\"><span class=\"flagtag\" style=\"color:#C63E2B;background:#FCE9E7\">RED</span><div class=\"flagtxt\">&#x27;Language&#x27; is a single-mention theme — held rather than reported until it recurs.</div></div>", lpTitle: "Missing & incomplete report · shareable link", lpSub: "Decision-first · Sopact branding · Open-Text Barriers · thematic analysis of open-ended survey responses", summaryBefore: "Grade: 1 green &middot; 1 amber &middot; 1 red. Counts reconcile to N=18 — transport and childcare dominate, one ambiguous, one single mention", summaryAfter: "Grade: 2 green &middot; 0 amber &middot; 1 red. The 'Other' bucket moves from ambiguous to classifiable with one clarifying sub-question" }; var phase = 1; var tabsEl = document.getElementById('tabs'); var bodyEl = document.getElementById('body'); var promptEl = document.getElementById('promptTxt'); function renderTabs(){ tabsEl.innerHTML = ''; TABS.forEach(function(label, i){ var b = document.createElement('button'); b.type = 'button'; b.className = 'tab' + (i === phase ? ' on' : ''); b.innerHTML = '<span class="num">' + (i+1) + '</span><span>' + label + '</span>'; b.addEventListener('click', function(){ phase = i; render(); }); tabsEl.appendChild(b); }); } function renderBody(){ var c = CONTENT[phase]; var eyebrowClass = (phase===2) ? 'eyebrow g' : 'eyebrow'; var out = '<div class="' + eyebrowClass + '">' + c.eyebrow + '</div><div class="headline">' + c.headline + '</div><p class="lede">' + c.lede + '</p>'; if (phase === 0) { out += '<div class="rows">' + BODIES.setup + '</div>'; } else if (phase === 1) { out += BODIES.build; out += '<div class="csummary"><div class="t"><b>' + BODIES.summaryBefore + '</b></div></div>'; } else if (phase === 2) { out += BODIES.fix; out += '<div class="csummary"><div class="t"><b>' + BODIES.summaryAfter + '</b></div></div>'; } else { out += BODIES.reportFlags; out += '<div class="lp"><span class="ic">&#128196;</span><div class="rt"><div class="n">' + BODIES.lpTitle + '</div><div class="s">' + BODIES.lpSub + '</div></div><span class="open">Open &rarr;</span></div>'; } bodyEl.innerHTML = out; promptEl.textContent = c.prompt; } function render(){ renderTabs(); renderBody(); } render(); })(); </script> </body> </html>

In short: Most thematic analysis quietly loses track of how many people said what. Done well, it names each theme, defines it, counts it, gives two quotes, parks the leftovers in a defined 'other', and reconciles every count back to N — holding single mentions out of the headline until a second response confirms them. One prompt does all of this and grades the result green, amber, or red.

1 · Set up the assistant over your data

Point the Sopact Sense Assistant at your dataset so it works from clean records with persistent contact IDs, then have it load your Decision Brief first. Clean text and a known N are what let the assistant reconcile theme counts back to the total rather than inventing prevalence.

You are the Sopact Sense Assistant working over the DEMO-04 · Open-Text Barriers dataset (clean data + persistent contact IDs). Load my Decision Brief (decision, audience, outcomes, indicators, evidence standard) first, then wait for my task.

2 · Write the prompt

Analyse all responses to the barriers question: themes, definitions, counts, 2 quotes each, 'other', prevalence reconciled to N. Hold single mentions. Grade green/amber/red.

Five elements make this prompt work: it runs over your dataset; it produces themes + counts with definitions and quotes; it reconciles to N so percentages add up; it holds single mentions out of the main pattern; and it ends with a grade of green, amber, or red.

3 · Read what Sense produces

Run on the Open-Text Barriers dataset (DEMO-04) already loaded in Sopact Sense.

GRADE: green | Recurring theme | counted to N; amber | Ambiguous other | undefined; red | Single mention | not a pattern

A green theme recurs, is defined, and reconciles to N. An amber 'other' bucket holds responses that were never defined — usable but unfinished. A red single-mention theme is one response dressed up as a pattern, and should be held back until a second response confirms it.

4 · Turn a weak link green

Take the lowest-graded element above and fix it using only what the program could realistically measure. Show the before → after grade and the single indicator/edit that moves it to green.

5 · Make the report and share it

Create a 'missing & incomplete' report from this analysis in Sopact branding [or paste your website URL / brand guideline to apply your own]. List every element graded amber or red, what is missing, and the one input that fixes each. Lead with the decision this report informs.
Create a shareable link for this report and open it in a new tab.

Tricks, tips, and troubleshooting

Reconcile every count to N. If your theme counts add up to more than the number of responses without a clear note about multi-coding, the prevalence is wrong. Always state the denominator.

Define the 'other' bucket. A catch-all 'other' is where rigor goes to hide. Either define what belongs in it or split it into named themes until it is small and explainable.

Hold single mentions back. One person raising an issue is worth recording, but it is not yet a theme. Keep single mentions in a watch-list, not the headline counts.

Give two quotes per theme. One quote can be a fluke of phrasing; two show the theme is a shared idea, not a single voice.

Show me the 'other' bucket verbatim and propose the smallest set of named themes that would empty it while still reconciling to N.

Frequently asked questions

How do you analyze open-ended survey responses?

Read every response, group them into named and defined themes, count how many people fall in each, give two quotes per theme, and reconcile the totals back to N. Park genuinely one-off responses as single mentions rather than inflating them into themes.

What does 'reconcile to N' mean?

It means the theme counts can be traced back to the total number of responses, so prevalence figures are honest. If counts exceed N, you must say so and explain that some responses were coded to more than one theme.

Why hold single mentions out of the main analysis?

Because one response is not evidence of a pattern. Recording it keeps it visible for next cycle, but counting it as a theme overstates how common the issue is. Promote it only when a second response confirms it.

Ready to try it for yourself?

Open Sopact Sense, paste your program description, and put it to work.

Try in Sopact