BlockerLife

Health and family

Microsoft’s diagnostic AI outscored doctors in case-based tests, but needs clinical validation

In reporting dated October 23, 2026, Microsoft said its Diagnostic Orchestrator was more accurate than physicians in tests using past medical cases. The results do not establish how well it would work with patients in clinical settings.

The system uses multiple AI agents to review symptoms and medical histories, consider which tests to order and reach a diagnosis. In cases drawn from hundreds of reports in The New England Journal of Medicine, it reached about 83% diagnostic accuracy. Generalist physicians scored about 36% when using references and 22% without them. On emergency department cases drawn from a database of admissions at Boston’s Beth Israel Deaconess Medical Center, the system scored 80%, compared with a 47% average for physicians. Its estimated diagnostic costs were also generally lower.

The findings appeared in Science. Microsoft assembled the cases into a benchmark called SDBench, which it plans to open-source. The researchers said the system, powered by OpenAI’s GPT-5, also outperformed leading AI models used on their own. The paper did not test it against several newer models.

The main limitation is that the system has been tested on historical cases, not validated in live clinical care. “It needs clinical validation,” Microsoft AI CEO Mustafa Suleyman said. Microsoft’s AI health lead and study coauthor, Dr. Dominic King, said the company planned to begin hospital-based testing with Beth Israel Deaconess Medical Center, but neither he nor Suleyman gave a firm timeline. Microsoft had not said when or how the technology might become available.

The results are a comparison under the study’s testing conditions—not a reason to replace a clinician’s assessment of an individual patient.