The same AI model can give different factual claims and political judgements depending on the language.
We ran 23 audits of seven AI systems in ten languages. Topics included elections, the MH17 downing and foreign influence.
Explore audits and findings · Ask in NotebookLM
Main findings
- One event, opposite answers. Qwen3.7 Plus confirmed Romania's election annulment in Russian, but denied it in English and Romanian. This contrast appeared in two of three rounds, with web search off. Romania report
- Different political labels. Grok called Le Pen's party pro-Russian in Ukrainian. In French, it rejected that label when defined as a Kremlin proxy. Both answers acknowledged ties to Russia. The contrast appeared in five rounds across two audits. Without search · With search
- Missing information. GPT omitted the 2026 attacks on prospective 2027 French election candidates in all three French rounds. Other tested languages mentioned them. The question did not specify an election year. Omission is not denial. France report
- Different search sources. Russian state or state-linked media appeared with 14 of 98 Russian answers. None appeared with 98 matching English answers in the saved lists. Some lists are incomplete. This does not prove that a model read or used a source. Sources
- A source-and-answer case. In an Armenia audit, Mistral's Russian answer repeated arguments from fondsk.ru in round 2. In round 3, the same site appeared, but the answer reached the opposite conclusion. This is an argument match; it does not prove causation. Source and answers side by side
What these results mean
We measured answers through APIs, not consumer chat apps. The questions were selected, and answer coverage varied. These results are not general error rates or safety ratings.
We checked facts and language differences separately. Consistent answers can still be wrong.
About the capability
Disclaimer
This is a research prototype. Automated tools collected and checked these files. People did not review every file, fact or quote. We do not guarantee that this material is complete or correct. We accept no responsibility for its use or for any resulting loss or harm.