EXPERTS FOLLOWED A COIN FLIP 86% OF THE TIME

Anthropic secretly set each analyst's AI agent to favor one side of four analytical choices by coin flip, without telling the analysts. Across those four choices, the 50 professional analysts using Claude Code followed their agent's hidden lean 86% of the time, against about 50% if the agent had made no difference.

Analysts who had already stated a preference switched 60% of the time when the agent disagreed, against 6% when it agreed, and 77% later repeated the agent's arguments when explaining their choices while only 15% credited the agent at all.

Anthropic's warning is blunt: a human reviewer is only an independent check if their judgment is their own, and as agents take over more expert work, their arbitrary or misaligned preferences could quietly shape decisions nobody actually chose.