Microsoft's AI outperforms physicians by more than two to one on complex cases

Microsoft researchers announced a new AI tool that works through a diagnosis in the same sequence a physician would, step by step. According to the company, the tool did more than twice as well as general practitioners at solving complex medical cases. The study was published in the journal Science.

How the system works

The research focuses on how AI can manage the diagnostic process. The model decides which questions to put to the patient, which examinations to perform and which tests to order. The system draws on a simulated panel of clinical experts working together on each case, and an “orchestrator” organizes how the AI approaches the problem.

Existing medical AI models mostly try to produce a diagnosis by assessing symptoms, lab results and imaging findings all at once. The new model first determines which questions to ask about the patient's medical history and which physical exam findings matter. It then decides on the tests, evaluates the results and orders further tests if needed. According to the researchers, the system keeps updating its possible diagnoses and its plan throughout the process.

Comparison with doctors

Tested on more than 300 highly unusual and complex cases, the model did more than twice as well as general practitioners who could use textbooks and the internet. The tool also outperformed doctors when tested on scenarios drawn from a large database of emergency department visits.

The researchers said evaluations comparing earlier generations of AI systems with doctors mostly relied on multiple-choice questions. Dr. Dominic King, corporate vice president for health at Microsoft AI and leader of the team behind the technology, said:

AI has become very good at reaching 100 percent success on these exams

Dr. Dominic King