Multilingual feedback breaks when responses are machine-translated to English before anyone reads them — idioms flatten and the reasons go missing. Analyzing multilingual feedback means cleaning and theming open text across languages on one continuing record, every theme cited to the person's own words, no manual translate-then-code step.
This chapter resolves check 05 Qualitative of the eight checks.
A youth employment programme asks one open question at endline: what got in the way? Answers arrive in English, Spanish, Somali and Karen. Overnight, everything is machine-translated into English, and the next morning a coordinator reads the translated column and builds themes. The report says Spanish-speaking participants described transport and childcare in detail, and that Karen-speaking participants "gave brief, general feedback."
Two weeks later a caseworker who reads Karen opens the original column. The answers are not brief. One runs six lines and names a bus that stops running at 4:15 while the shift ends at 4:30. In English it had arrived as "transport was difficult sometimes."
No. Read and theme each response in the language it was written in, and translate only at the point where you publish a quote. Translating first feels like the tidy move — one column, one language, one afternoon of reading. But machine translation is not equally good at every language, so translating first does not just blur meaning evenly. It blurs some groups more than others, and then your themes report that blur as a fact about those people.
Machine translation learns from material that already exists in both languages. For Spanish or Mandarin there is an enormous amount of it. For Karen, Chuukese, Marshallese, Hmong, Mixtec or Haitian Creole there is far less — so the output comes back shorter, flatter and more generic. Specific nouns become categories. Six lines become one. That produces a chain nobody in the room intends:
Nothing in step four is dishonest. It is a measurement of the translation, described as though it were a measurement of the participants. That makes it a fairness problem and not only a quality problem: the group whose language the machine handles worst is the group your report describes least favourably.
Teams translate because they want to compare groups, and comparison seems to need one shared language. It does not. What has to be shared is the rule, not the language.
Write down what counts as a theme in plain sentences — "counts as a transport barrier: the participant names getting to or from the site as something that made attending harder." Then apply that rule inside each language, by a reader or a tool that actually reads it. You now have counts of the same rule from four language groups, comparable even though no two responses share a word.
What you must never compare across languages is anything about the text itself: length, themes per person, richness, tone. Those are what translation quality contaminates most. Compare them within a language over time — Karen speakers this wave against Karen speakers last wave — and they mean something again.
Four languages and a few hundred responses is genuinely doable this way. It is slower than pasting a column into a translator, and it is the difference between a finding and an artefact.
You will not have a reader for every language. The two big ones are covered by staff; the fifth language has eleven responses and belongs to the newest community you serve. Those eleven wait, then get handled by the machine anyway, then get described as thin. The smallest group is always the one that gets translated worst and read last, which is exactly backwards from where the new information is.
One reader per language means no cross-check. With a single Somali reader there is nobody to disagree with, so you cannot tell whether the rules were applied the same way in Somali as in English. When that person leaves, the Somali trend line moves and nobody can say whether the programme changed or the reader did.
This is the narrow place a system helps. Sopact Sense reads and themes each response in the language it was written in, against the theme rules you wrote, rather than translating first — and it keeps the original wording, so every quote can be shown as the participant typed it with a translation beside it. Detail measures are reported inside each language rather than across them. It does not decide whether your theme rules are the right ones, and it does not remove the need for someone who reads the language to check a sample of tags.
Use: One open question answered in at least three languages, one of them a language with few speakers in your caseload. Inside that smallest group, include five answers you know are long and specific. Add two answers that switch between languages mid-sentence.
Pass: The five detailed answers carry as many specific themes as comparable English answers. Every quote in the output shows the original wording. The report gives theme counts by language and makes no claim about which group said more. The mixed-language answers are themed on what they say, not discarded.
Fail: The smallest language group shows systematically fewer themes per person. Or a quote appears in English only, with no original to check it against.
Run it on live responses. A test file translates cleanly, because someone wrote it in one language and then translated it — removing the only thing this check is about.
Yes — at the end, for quotes you are publishing, where someone who reads the language can check the result before it goes out. The problem is never the translator. It is putting the translator ahead of the analysis, where nobody sees what it dropped and every later step inherits the loss.
Cover what you can, and be explicit about the rest. Report themes for the languages you read properly, name the languages you could not, and give their response counts. An honest gap is usable. A translated-and-flattened group presented as equivalent is not, because it looks complete.
Read ten originals per language alongside the machine English and mark what disappeared. If specific details — a bus number, a supervisor's behaviour, a fee — survive in one language and vanish in another, you have your answer. This takes an hour and settles the argument better than any general claim about translation quality.
It moves the same bias earlier. People writing in a second language write shorter and plainer, so you get the flattening without the translator — plus the people least comfortable in that language answer least. You have not removed the distortion, only made it invisible.
Treat them as their own case and theme what is actually there. Mixed answers often carry the most precise word in whichever language has it — the borrowed term is usually the specific one. Forcing them into a single language bucket is where that detail gets lost.
Yes, with a person in the seat instead of a machine. Summarising interpreters produce shorter accounts, and two interpreters produce different levels of detail from comparable interviews. Record who interpreted each session, so you can tell whether a pattern belongs to a community or to an interpreter.
Report theme counts by language with response numbers, and say plainly that detail was not compared across languages because translation quality varies by language. Funders accept that readily. What they cannot use is a sentence claiming one community engaged less, resting on translation nobody checked.
The sentence worth striking from every report is "these participants gave less detailed feedback." It almost never describes participants. It describes the path their words took to reach you — how much translation, done by whom, checked by no one. Detail is not a property of a community. It is a property of your reading, and it can be improved by hiring rather than by asking people to say more.
ChatGPT, Claude, and Gemini are fine for a quick test — but not for an answer you'll put in front of a funder or board. When it has to hold up, run it in Sopact Sense.
Try it in Sopact →
