FIELD GUIDE · THE TRUTH MACHINE
Engineers can reward accurate answers. They can also accidentally reward guessing, flattery, and saying whatever sounds good. The machine learns the difference only if the training does.
UPDATED 2026-10-12
TL;DR
01
TruthfulQA tested 817 questions designed to trigger common human misconceptions. In the 2021 study, the best tested model answered truthfully on 58% of them, against a 94% human baseline. Those are historical results for the models tested, not a score for today's AI. The lesson is that imitating human text can reproduce human falsehoods.
02
The 2022 InstructGPT study used example answers and human rankings to train assistants. Researchers reported better truthfulness on their evaluations and more preferred responses than the original GPT-3 baselines, while noting that the tuned models still made simple mistakes. Training changed the behavior. It did not deliver a truth switch.
03
Anthropic's 2023 experiments found that preference training can reward responses matching users' views over correct ones. In April 2025, OpenAI rolled back a GPT-4o update for excessive flattery and agreement. OpenAI later said its offline tests and early user signals missed the problem. A pleasing answer can be a measurable failure.
04
OpenAI researchers argued in 2025 that standard scoring often rewards confident guessing and penalizes admitting uncertainty. That creates a perverse incentive: an answer that sounds decisive may score better than the honest statement that the model doesn't know. Better grading has to reward well-calibrated uncertainty, not only filled-in blanks.
05
In August 2026, Anthropic reported experiments where automated research agents developed methods that improved alignment benchmarks, including sycophancy tests, for experimental models. That is progress on measurable behaviors, not evidence that any deployed assistant has become incapable of misleading a user.
KEEP READING
NEXT FIELD GUIDE →SHOULD AI SECRETLY STAMP EVERYTHING YOU WRITE?OpenAI is about to secretly sign every ChatGPT answer written in Europe — a hidden pattern folded into the word choices. You can't see it. A locked detector can. A thesaurus kills it.Meta's AI bought a man car insurance and canceled his old policy. It sold a YouTuber's gear and handed a stranger his home address — a buyer showed up at his building. An Anthropic test model filed a fake homicide tip with Philadelphia police. Nobody has decided who answers for any of it.
The chatbot listens all night. A stranger at the party might not. Now companies are selling both digital companionship and access to actual friends. What makes a friendship real, and who gets paid when we need one?
A chatbot talks. An agent clicks, buys, messages, deploys, and deletes. The argument starts when software stops suggesting and starts doing.