THE CLINICAL AI INDEX
We track how frontier models score on the benchmarks that matter in medicine, graded the way it counts: by physicians, on real clinical work. The gap between models and doctors is exactly what our pilot exists to close.
by Mercor
100 primary-care tasks written by physicians at Brigham and Women's, UPenn, and Northwestern. Models do the work. Expert rubrics grade it.
Mean rubric score, as published by Mercor. Full task set is private.
by OpenAI
5,000 realistic health conversations graded against 48,562 criteria written by 262 physicians.
The rubric itself is the physician standard. No separate physician score published.
by OpenAI
Blinded grading of medical communication quality, model vs human.
As reported at launch. The rare benchmark where the model outscored the doctors.
by Stanford
35 benchmarks across notes, patient communication, decision support, research help, and admin work. Built with 29 clinicians.
Public leaderboard, updated as models are evaluated.
by Hugging Face
Community-run medical evals on Hugging Face. Synced Oct 12, 2026 from the public dataset.
| # | Model | Accuracy | Tasks | Evaluated |
|---|---|---|---|---|
| 1 | ProbeMedicalYonseiMAILab/medllama3-v20 | 90.0% | 9 | 2024-05-24 |
| 2 | aaditya/OpenBioLLMLlama-70B | 86.1% | 9 | 2024-04-26 |
| 3 | Med-PaLM 2 (5 Shots) | 84.1% | 9 | 2024-03-14 |
| 4 | GPT-4 | 83.0% | 9 | 2024-03-14 |
| 5 | skumar9/Llama-medx_v3.2 | 75.4% | 9 | 2024-05-07 |
Scores come from public submissions through May 2024. Rankings move as the community submits new evaluations.
Our own benchmark, graded by our pilot physicians on real clinical reasoning tasks. In the works.
Scores republished with attribution and a link to each source. Benchmarks update on different schedules; every card shows its last evaluation date. ClinicalPearls does not run these benchmarks. Yet.
Standings from the AI Boards, where verified anesthesiologists grade AI models blind.
Loading board scores…
How board scores work: clinicians grade AI models that are labeled only AI Model 1, AI Model 2 and so on, and names are revealed after grading. Board scores reflect grades from verified clinicians in the matching specialty. Anyone can grade, but other grades are saved without moving the scores. Every model starts at a board score of 1500, using standard Elo with a K-factor of 32. A tie counts as half a win for each side. In Challenge AI Models, each higher-ranked answer beats every lower-ranked answer. Ranking five answers produces ten comparisons. Grade the Boards keeps its blind two-model votes. Scores update as grades land. Models with fewer than 30 grades are marked Provisional.
Join the ClinicalPearls pilot. Evaluate clinical AI on real cases, a few minutes a day, and get compensated for your expertise.
Apply to the pilot