Harvard Study Finds OpenAI Models Outperform Physicians in Emergency Room Triage
A study from Harvard Medical School shows that OpenAI models outperformed human physicians in emergency room diagnoses. The research used data from 76 patients at Beth Israel Deaconess Medical Center. The o1 model achieved 67% accuracy in triage cases. Human physicians reached 55% and 50% accuracy in the same cases.
Researchers provided the AI with raw electronic medical record data. They did not pre-process the information. The study focused on text-based inputs only. The authors state that AI is not ready for independent clinical decisions. They call for more trials in real-world settings.
Key Takeaways
- OpenAI o1 model achieved 67% diagnostic accuracy in triage cases compared to 55% for human physicians.techcrunch.com
- Researchers used raw electronic medical record data without any pre-processing for the AI inputs.techcrunch.comhms.harvard.edu
- The study authors call for prospective real-world trials before AI is used for clinical decisions.techcrunch.comhms.harvard.edu
- Critics note the study compared AI to internal medicine doctors instead of emergency medicine specialists.theguardian.com
- Lead authors highlight a current lack of formal accountability frameworks for AI-based medical diagnoses.theguardian.com
Affected Companies
Sources
Study Suggests AI Is Good Enough at Diagnosing Complex Medical Cases To Warrant Clinical Testing
Harvard Medical School
“The university announced research suggesting that AI models are capable of diagnosing complex medical cases using raw electronic records.”
AI outperforms doctors in Harvard trial of emergency triage diagnoses
The Guardian
“Experts warned that the study compared AI to internal medicine doctors rather than emergency specialists and noted a lack of accountability frameworks.”
Large language models in emergency room triage
Science
“Researchers published a study evaluating the performance of large language models against attending physicians using real patient data.”