Additional course links
The aim is to learn something useful from people’s words without flattening different experiences into a word count. This lesson produces a simple codebook, a reviewed response table and one evidence-based finding.
In the membership course, use this method to separate a benefit someone values from a concern they still need resolved. Keep an individual response distinct from an organization’s report. Bring the reviewed themes and supporting excerpts back to the lesson 3 exercise, using the practice path you chose.
Start with a decision, not a list of popular words
A fictional service team asks customers what made onboarding difficult. The team needs to choose what to improve next. A word cloud may highlight “contact” or “setup,” but it cannot establish whether a person praised the contact, could not find the contact or had no setup problem.
Write the question the analysis must address. For example: “What difficulties did respondents describe, and which of those difficulties does the service team need to investigate?” This gives the review a purpose while leaving space for an issue the team did not anticipate.
Preserve the response and its context
Keep each original response with the survey question, collection date and appropriate record identifier. Where identified collection is permitted, this can connect to the relevant customer, participant or member record. Anonymous responses should remain compatible with the promise made to respondents.
Keep corrections and derived fields separate from the original. If a response is translated, retain both versions and record uncertainty about meaning. Removing names for a shared view should not silently rewrite the source wording in the analysis record.
Classify response status before themes. A blank answer, “not applicable,” “no problems” and a substantive complaint are different states. An empty response is not evidence that the person had no difficulty.
Choose what you are counting
You might count responses, people, accounts or distinct incidents. These are not interchangeable. One person can submit several check-ins, and one response can describe several difficulties.
For this example, the unit is a response to the onboarding question. Each response may receive more than one theme, but a theme is counted at most once per response. This avoids counting a repeated phrase as several people experiencing an issue.
If the decision instead concerns unique customers over a quarter, specify how multiple responses from the same customer are treated. A persistent record can help you make that calculation where linkage is appropriate; it does not choose the analytical rule for you.
Build a codebook that another reviewer can use
| Code | Include | Do not assume |
|---|---|---|
| Unclear next step | The respondent says they did not know what to do or who would act next | A delayed action does not necessarily mean the instructions were unclear |
| Setup difficulty | The respondent describes difficulty completing the initial setup | A mention of setup alone is not evidence of difficulty |
| Delayed response | The respondent reports waiting for a reply or action | The comment alone may not establish the actual elapsed time |
| Positive support experience | The respondent describes helpful support | Positive support does not mean there were no other difficulties |
| Other or uncertain | A relevant issue does not fit the current definitions, or meaning is unclear | Do not force it into the nearest existing category |
These are example codes, not a universal customer service taxonomy. Begin with the analysis question, read a varied sample and revise the definitions as you encounter evidence. Keep a version history so later readers know which codebook produced the results.
Work through three responses
| Original response | Supported interpretation | Limit or follow-up |
|---|---|---|
| “Setup worked, but I did not know who would call me next.” | Unclear next step | Do not also mark setup difficulty |
| “The adviser was helpful. I waited several days before hearing back.” | Positive support experience; delayed response | Keep reported duration distinct from a calculation using dated records |
| “No problems.” | Explicit report of no problems | Do not treat it as a missing answer or as evidence about every part of the service |
Review disagreements against the words and context. If two reviewers interpret a phrase differently, improve the definition or retain the uncertainty. Agreement between reviewers is useful evidence about the coding process; it does not by itself prove the interpretation is correct.
Use AI to assist review, with evidence attached
An AI-assisted workflow can propose codes, highlight relevant passages and flag ambiguous cases. Give it the question, code definitions, inclusion rules and examples. Ask for the supporting text rather than only a label.
Check a varied sample, including unflagged responses, mixed positive and negative comments, different languages and uncommon issues. Review consequential interpretations with an appropriate person. Reusing the same prompt does not guarantee the same judgment or make every coding decision valid.
In Sopact, the useful connection is between the source response, its continuing context, the shared definitions and the reviewed analysis. Preserve approved coding results and the calculation used for a report. A fresh AI summary should not silently replace the evidence behind an earlier finding.
Plan the recurring coding and reporting work
The reviewed response table is the starting point. Now plan how your team will apply the codebook across a growing dataset, revise it and ask questions that connect comments with ratings.
A workflow with repeated manual work
- Define from an initial sampleRead material and agree on the codebook.
- Apply it across the datasetCode responses and check the result.
- Revise a definitionReturn to affected material and recode it.
- Reconnect the numbersReconcile coded results with ratings and context, then rebuild the view.
The Sopact workflow
- Your team owns the definitionsDecide what each code means and improve it as you learn.
- Apply coding across the eligible dataAutomate application; people review quality and exceptions.
- Reprocess after a definition changesReapply the revised definition across the configured scope instead of recoding each response by hand.
- Ask across coded text and numbersKeep the response, rating and relevant record context connected; inspect the evidence behind the result.
This compares workflow patterns, not a claim that every research tool requires manual coding or separate files. Some already automate parts of this work; compare the complete cycle.
For this codebook-based workflow, the main saving is repeated application and reconnection—not the removal of human judgment. A changed definition can be reapplied across the configured data while reviewers concentrate on quality, exceptions and interpretation. Coded text stays connected to the relevant ratings and context.
Count the recurring work in ownership cost. Include setup, coding, recoding after revisions, source reconciliation, review and reporting, plus your actual platform and processing expenses. A worked scenario of four cycles of 4,000 responses illustrates 272 fewer annual staff hours; it is an assumption-based example, not a customer benchmark. Existing automation, review needs and implementation effort can substantially change the result.
Adjust the workload assumptions and compare total effort →
A reliable assistant should calculate from the selected records and let a reviewer open the supporting evidence. Check the data scope, definition, denominator and access permissions. Reproducible arithmetic does not make every AI interpretation correct.
Watch: Why Qualitative Analysis Stays Small — And How to Scale It
Watch this 2-minute 58-second explanation of repeated coding, revised definitions and connected analysis. Then estimate the work for your own collection cycle.
Report patterns without overstating them
Suppose 50 people were invited, 30 submitted a response and 24 provided substantive comments. Eight of those 24 mentioned an unclear next step. Report the base explicitly: eight of 24 substantive comments, alongside the response coverage. Do not describe that as eight of every 24 customers in the entire customer base.
If themes overlap, say so. Their percentages need not total 100%. A low-frequency issue may still require attention because of its consequences, but frequency alone does not establish severity. Distinguish how often something was mentioned from how important a team judges it to be.
Use quotations to illustrate a finding, not to imply that one vivid comment represents everyone. Check whether a quotation could identify someone before sharing it. Where the evidence conflicts, keep that visible rather than selecting only comments that support the preferred explanation.
Turn the finding into a reviewable next step
A useful finding names the theme, reporting base, supporting evidence, uncertainty and proposed action. For example: “Eight of 24 substantive onboarding comments described uncertainty about the next step. Review the handoff instructions and check the relevant service records before deciding whether the issue is unclear wording, ownership or delayed action.”
Assign an owner and record the response. At the next collection point, ask an appropriate question to learn what changed. A theme in feedback is a starting point for investigation, not proof of the cause or proof that an intervention worked.
Your exercise
- Choose one question and the decision it should inform.
- Define the unit of analysis and response-status categories.
- Draft three codes with inclusion rules and non-examples.
- Review a varied set of fictional or appropriately permitted responses.
- Record disagreements and revise the codebook.
- Write one finding with its denominator, evidence and limits.
- Assign the next action and decide what follow-up would be useful.
Self-check: Could another reviewer reproduce the reported count from the approved codes and inspect the source behind each interpretation? If not, improve the record before presenting the result.
Frequently asked questions
Can one response have several themes?
Yes, when the text supports them. Define whether each theme is counted once per response and explain overlapping percentages. Theme counts are not automatically counts of distinct people.
How should blank answers be handled?
Keep blanks distinct from explicit no-problem responses, not-applicable answers and substantive comments. Show the appropriate response coverage and denominator for the finding.
Can AI analyze the responses without review?
AI can assist with suggested codes and supporting passages. Review the definitions, ambiguous and consequential cases, and a varied sample including unflagged responses before relying on results.
Return to Membership & networks · Lesson 3: Make sense of member comments →