Course progress and additional readings
Course lessons · 5 of 8
You will make: A short coding guide, reviewed comments and a finding with its evidence and limits attached.
Recorded walkthrough · from 9:05 · 2 min
The segment on the roughly eighty per cent of member intelligence that arrives as text: open answers, community transcripts, policy documents. It shows comments coded into themes as they land, with the story sitting beside the number rather than in a file nobody opens. Watch on YouTube ↗
How do you read a comment for the decision it points to?
“The certificate was useful, but I do not see enough value to justify another year of dues” is neither positive nor negative. It contains a benefit and an unresolved concern. A single sentiment score hides the reason a membership team might need to act.
The six fictional comments in the practice pack cover dues value, access and scheduling, and practical relevance. They are a small exercise, not a representative member survey. Start by reading them all. Preserve anonymous feedback as anonymous even when analyzing it alongside named comments at an appropriate aggregate level.
| Theme | Include when the comment says… | Do not infer… |
|---|---|---|
| Value for dues | A benefit or concern explicitly connected to membership cost or renewal value | That the member will leave |
| Access or scheduling | Timing, availability or access helped or prevented participation | Low motivation from a missed event |
| Practical relevance | Content did or did not help the respondent’s actual work | That praise means every need was met |
A comment may have several themes. Keep its stance toward each theme and the supporting words. Allow “uncertain” when context is insufficient. Do not hide an important concern only because one respondent mentioned it.
In this course the comments come from the survey in lesson 4, so each one already sits on a member’s record beside the chapter and member type that came in with your AMS list. That lets you ask which chapters raise which theme, and keep the answer tied to the people who wrote it.
How do you make the judgment visible?
Fictional coding example
Comment: “The certificate helped my work, but I still cannot justify next year’s dues.”
Reviewed interpretation: Practical relevance: positive. Value for dues: unresolved concern.
Next action: Ask what benefit is missing. Do not label this person a confirmed non-renewal.
The open-ended response method explains how to define codes, retain excerpts and count overlapping themes. Use its codebook structure to refine the three themes above, then return and code the six practice responses. Compare disagreements against the actual wording.
For independent member organizations, the same method can organize local reports and comments about delivery barriers. Keep the organization, reporting period, source and reviewer attached. A central team’s interpretation should not silently replace the member organization’s explanation.
How do you scale the method without making the review disappear?
AI can propose classifications across a larger set once the question and codebook are clear. First test a varied sample containing mixed views, short answers, different languages and ambiguous wording. Compare the proposed themes with human judgments; change unclear definitions and rerun the sample.
Ask for row IDs and exact supporting excerpts. Review uncertain cases, consequential concerns and a sample of routine classifications. Keep original-language text and arrange appropriate language review where needed. A confident-looking label is not evidence of accuracy.
Choose which fields the AI reads before you run it. Send the comment and the field the question needs, such as the chapter, and leave out name and email when the question does not need them.
| Question | Send to AI | Leave out |
|---|---|---|
| Why did satisfaction change in some chapters? | The comment and the chapter | Name, email |
| What did attendees say about the venue? | The comment | Name, email, member number |
| What do applicants mention most often? | The application text | Name, email |
Use this bounded analysis prompt
Use only the supplied, authorized comments and codebook. For each response return: response ID, proposed theme(s), stance for each theme, exact supporting words, uncertainty, and review needed. Allow more than one theme. Keep praise and criticism separate. Do not invent missing context, identities, renewal intentions or causes. Keep anonymous rows unlinked. List unclear cases for a person to review. Summarize only after showing the row-level decisions and the number of responses reviewed.
Sopact’s AI Assistant reads survey comments and uploaded documents, and each finding cites the comments behind it, so you can open them and check. A written question, a fixed codebook and cited comments make the review more consistent and auditable. They do not make it exact, and people still decide what a finding means.
ASME’s published story shows a membership organization turning previously underused stakeholder comments into categorized insight alongside quantitative data. The lesson is the move from selected anecdotes to systematic analysis.
The King Center’s story makes the related point across programs: feedback that was collected became usable during the work. It is not evidence of a membership-system replacement or a move away from Qualtrics.
How do you write a finding someone can question?
State which material you reviewed, what pattern you found and which source excerpts support it. Theme totals can exceed the number of responses because a response can contain several themes. Distinguish people, comments and mentions.
In this exercise, P101 and P102 both meet the event-and-certification condition and raise a dues-value concern. That is a finding about two linked respondents in a small synthetic extract. The anonymous comment can inform a separate theme summary; it cannot expand the named follow-up list.
Put it into practice
What reviewed finding will you write?
Use step 3 of the workbook. Complete only your chosen path.
Individual-member path: Code the six responses. Check the mixed view in R2 and keep R5 anonymous.
Member-organization path: Review NF1 to NF3 as organization summaries. Keep NF3 uncertain and write the clarification its source owner needs to answer.
For either path, write the material reviewed, the finding with source IDs or approved excerpts, and the limit on what it means.
Check your reasoning before moving on
A defensible answer retains the positive value of the certification and the concern about dues. It does not force overlapping themes to total six, discard the single anonymous concern, or treat critical feedback as a prediction of churn.
The practice pack includes a fuller answer key and source-row references.