32.6 %
Higher diagnostic accuracy with support from our Clinical Assistant
Medsys AI's preclinical evaluation, published in JMIR Formative Research, shows improved diagnostic accuracy with real-time support in highly challenging consultations.
+12.3
percentage points
with Medsys AI
Estimates adjusted for physician and case.
95% CI for the difference: +2.7 to +22.6 points; p = 0.01.
What does Top-3 mean? The correct diagnosis appeared among the up to three final diagnoses proposed by the physician. We focused on this because we wanted to measure the tool's ability to improve the physician's differential diagnosis, rather than provide a definitive one.
13family
physicians
physicians
260simulated
voice consultations
voice consultations
40diagnostically
challenging cases
challenging cases
3independent assessors
blinded to study arm
blinded to study arm
Supporting clinical reasoning
The consultation evolves.
So does the support.
Medsys AI supports physicians as they gather information and build their hypotheses. Based on the conversation, it suggests differential diagnoses, relevant questions and examinations that may help complete the assessment.
Physicians can use, ignore or dismiss each suggestion.
During the consultationReal-time support
1
The physician leads the conversation
They decide what to ask and examine. Information emerges as the dialogue progresses.
2
Medsys AI adds context
It generates suggestions based on the information available at that moment.
HypothesesQuestionsExaminations
3
The physician assesses and decides
They weigh the alternatives and retain responsibility for the diagnostic decision.
What the study found
An improvement in
differential diagnosis.
The primary outcome measures whether physicians included the correct diagnosis among their proposed diagnoses at the end of the consultation.
The study assessed physicians' performance with and without assistance during the interaction, rather than a model's ability to solve a case on its own.
4.38 / 5
Perceived usefulness
4.46 / 5 for overall satisfaction.
+35 s approx.
Longer assisted consultations
AI-assisted consultations lasted 10.7% longer on average.
How it was evaluated
Real physicians.
A controlled setting.
Version evaluated:
Medsys AI-Clinical Assistant V1.0.0
260 consultations20 per physician · 40 reference cases
Random assignment by consultation
120consultations
without assistance
without assistance
140consultations
with Medsys AI
with Medsys AI
01
Virtual patient
We used a simulator designed to reproduce the behaviour of a typical patient. A review of its performance found no quality deviations in 244 of the 260 consultations.
02
Physician-led voice consultations
The study involved 13 specialists in Family and Community Medicine from SERMAS.
03
Blinded assessment of diagnoses
Three independent physicians, blinded to study arm, assessed the diagnoses from each consultation.
Explore further
The details
Full details are available in the article and study materials.
Top-3, Top-2 and Top-1 results
Top-3 was the primary outcome. Top-2 and Top-1 were analysed post hoc as sensitivity analyses.
Swipe the table to see all columns →
| Measure | Without AI | With AI | Difference; 95% CI | p |
|---|---|---|---|---|
| Top-3 · primary | 62.3 % | 74.6 % | +12.3 points; +2.7 to +22.6 | 0.01 |
| Top-2 · post hoc | 57.9 % | 70.7 % | +12.8 points; +3.1 to +23.2 | 0.01 |
| Top-1 · post hoc | 47.8 % | 57.0 % | +9.2 points; −1.0 to +20.1 | 0.08 |
How to interpret the 32.6% and NNT-equivalent
Adjusted Top-3 errors decreased from 37.7% without assistance to 25.4% with AI: a relative reduction of 32.6% (95% CI: 8.9 to 54.4%).
The NNT-equivalent was 8.1 simulated consultations per additional correct Top-3 outcome (95% CI: 4.4 to 37.2).
What the safety analysis showed
The analysis examined consultations in which the AI's operational diagnostic suggestion was incorrect. Adjusted agreement between the physician's first diagnosis and that suggestion was 38.8% without assistance and 55.8% with AI.
The adjusted difference was +17.0 points (95% CI: −2.7 to +36.8); adjusted odds ratio 2.08 (95% CI: 0.89 to 5.19); p = 0.09. The comparison was not statistically conclusive, but indicates that we should continue to monitor this in future research.
Robustness and results by difficulty
When excluding one physician at a time, the adjusted Top-3 difference remained between +9.2 and +14.3 points. When excluding 16 consultations with simulator quality deviations, the difference was +13.2 points (95% CI: +3.3 to +23.6).
In an exploratory analysis, the highest-difficulty quartile showed a difference of +43.9 points (95% CI: +22.8 to +64.3). The easiest quartile favoured the unassisted condition: −7.7 points (95% CI: −16.3 to −0.6).
Authors and ethics
The Research Ethics Committee of Hospital Universitario La Paz determined that ethical approval was not required (11 August 2026).
Article and materials
JMIR Formative Research
Real-Time Artificial Intelligence Diagnostic Copilot in Simulated Primary Care Consultations: Randomized Simulation Study
2026;10:e104579 · DOI: 10.2196/104579
A question is also
a good place to start.
Get in touch. We would be happy to tell you more about this and future research.
Materials for journalists are available in our press room.