Start with the evidence you actually have
Northstar receives M-12’s return: 60 people reached, marked as an estimate, and 140 attendance entries. A supporting document covers the previous quarter. A comment says that scheduling made participation difficult. These are different kinds of evidence, not a single answer waiting to be summarized.
Begin with completeness and consistency. Is the period clear? Are people and visits defined? Does the attached document support this return? Then ask the substantive question: what does the evidence suggest about participation and support needs?
On-arrival analysis means you have planned these questions and checks in advance. It does not mean every finding is automatically approved or that the system must issue real-time notifications. Analysis timing, routing and human review depend on the configured workflow.
Separate a flag from a conclusion
The two counts may both be valid: one person can attend more than once. The document-period mismatch is a concrete clarification need. Neither observation proves misconduct or poor performance. Record what is known, what is inferred and what remains unanswered.
A useful review record contains the source reference, proposed finding, relevant definition, uncertainty, reviewer and status. If the contributor supplies a current document, update the review while preserving the original evidence and reason for the change.
For comments, begin with a small codebook relevant to the question. A scheduling-barrier category needs a definition and an example. Allow multiple themes where appropriate, and retain an unclassified or uncertain state. Do not force every comment into a category merely to produce a neat chart.
Keep meaning through languages and formats
Northstar may receive feedback in several languages, translated notes or document extracts. Preserve the original wording, language and source. A translation or extraction is a working representation; it should not silently replace the original.
Use a competent reviewer to check consequential or ambiguous interpretations and a varied sample of routine ones. Record what was reviewed and what remains provisional. When review capacity is limited, report that limitation rather than treating unreviewed responses as having no issue.
A shared codebook helps organize analysis but does not make groups comparable by itself. Invitation coverage, wording, collection conditions and interpretation quality also matter. Counts of themes describe the material reviewed, not everyone in the network.
Use the assistant to prepare evidence for judgment
Instead of asking for a broad summary, specify the authorized records, question, definitions and evidence requirements. Request source references and explicit gaps. Uploaded content supplies evidence; it does not authorize new access or external actions.
For this exercise, ask for participation-related themes in the approved returns, with a source passage for each and a list of unresolved period or unit issues. The reviewer can then check the result rather than reconstruct the analysis from a polished paragraph.
The workflow differs from repeatedly downloading comments and reading attachments afresh because the criteria, sources and review history continue with the records. AI preparation still needs verification. A fluent response is not proof that the interpretation or calculation is correct.
Work through the Northstar example
This is a fictional teaching example. Adapt the fields and rules to the question your own network needs to answer.
| Incoming evidence | First interpretation | What the reviewer does |
|---|---|---|
| 60 people; 140 attendance entries | Potentially different units, not necessarily a contradiction | Confirm definitions; keep the measures separate |
| Supporting file covers previous quarter | Evidence does not establish the current period | Request current support or document the limitation |
| Scheduling concern in a comment | Possible participation barrier | Check original wording and codebook; record review status |
Build this part of your plan in more detail
These lessons address the next practical questions in this module. Choose the detail your workflow needs, then return to complete the exercise.
- How to Clean Open-Ended Survey Responses Without Losing Meaning
Keep blanks, uncertain answers and substantive comments distinct.
- How to Analyze Documents as Evidence: Sources, Context and Review
Review organizer reports and imported supporting documents.
More practical questions in this module (2)
- How to Analyze Open-Ended Survey Responses
Build and review a codebook, then report themes with a clear denominator.
- How to Analyze Multilingual Feedback and Evaluate Software
Review meaning before comparing feedback across languages.
Add this part to your plan
Fill workbook page 3 with two review records: the document-period issue and the scheduling theme. Include the source, interpretation, uncertainty, owner and next action.
Download workbook (fillable PDF)Compare with a suggested answer
Keep the old-period document as the original source and mark support for the current period unresolved. Ask M-12’s owner for clarification. Treat scheduling as a provisional theme until the passage is checked against the definition; retain language and review status. Do not combine people and visits or treat the theme as a cause of lower participation.
Self-check: Can another reviewer distinguish evidence from interpretation and reproduce the basis of your finding?
Watch: why qualitative evidence gets overlooked
A 2 minute 34 second companion overview. The lesson and exercise above stand on their own.
Questions you may have
Does analysis on arrival mean no human review?
No. It means the analysis questions and criteria are ready when information arrives. Human review remains necessary where interpretation, clarification or a consequential decision requires it.
Can an untranslated response be counted as no issue?
No. Record it as awaiting interpretation. State the analysis base and pending responses so incomplete review does not become a misleading result.
Apply the method to your work
Use the five-part plan to assess the sources, analysis and permissions your team needs. The solution page shows where a connected platform can support that workflow.
Explore the membership and networks solution →