The fine print: the doctors were generalists, and much of the gain came from the underlying model

The fine print matters. Cases this complex are usually handled by specialists or physician teams, who were not part of the comparison. The physicians were also barred from using LLMs or search engines — tools most doctors use daily.

And much of the gain sits in the underlying model, not Microsoft's orchestration. The study's own data shows GPT-5 alone scoring 78.9% with a basic prompt; the orchestrator added about four points and cut per-case cost by roughly $3,000. It helped weaker models most, lifting Gemini 2.5 Flash from 46.4% to nearly 69%.