Preclinical trial · Randomized simulation study
Real-Time Artificial Intelligence Diagnostic Copilot in Simulated Primary Care Consultations: Randomized Simulation Study
JMIR Formative Research. 2026;10:e104579.
DOI: 10.2196/104579
Scientific evidence · Preclinical trial
Higher diagnostic accuracy with Medsys AI
In a study with family physicians in highly challenging simulated consultations, real-time support from the Medsys AI Clinical Assistant was associated with a greater ability to include the correct diagnosis among physicians' proposed diagnoses.
+12.3 percentage points
in physicians' diagnostic accuracy
A consultation counted as correct when the correct diagnosis appeared among the up to three diagnoses proposed at its end: the measure known as Top-3.
Adjusted estimates. 95% confidence interval for the difference: +2.7 to +22.6 points.
Complementary measures:
32.6%
Relative error reduction (RER)
Fewer Top-3 errors, relative to the unassisted condition.
8.1
NNT-equivalent
AI-assisted simulated consultations per additional correct Top-3 outcome.
The NNT (number needed to treat) equivalent refers to the highly challenging simulated consultations in this trial.
Published in JMIR Formative Research
Supporting clinical reasoning
Support as physicians build their differential diagnosis
The Medsys AI Clinical Assistant is designed to support physicians' reasoning during the conversation. It proposes diagnostic hypotheses, relevant questions and examinations that may help complete the assessment. Physicians can use, ignore or dismiss its suggestions.
Randomized preclinical trial
Trial details
- Family physicians
- 13
- Simulated consultations
- 260
- Highly challenging cases
- 40
- Independent assessors
- 3
In this preclinical trial, physicians spoke with virtual patients and decided what to ask and examine. Each consultation was randomly assigned to working with Medsys AI or without assistance, with no other external diagnostic aids allowed.
The AI received the conversation but had no access to the virtual patient's script.
Blinded, independent assessment of diagnoses
Three practising family physicians, independent of the participants, assessed diagnoses against each case's reference diagnosis. They received physicians' proposed diagnoses, AI suggestions and the reference diagnosis, with all information that could reveal the consultation's assignment or whether the physician had seen the suggestions removed.
They applied predefined criteria to assess matches with the gold-standard diagnosis.
Version evaluated in the preclinical trial: Medsys AI-Clinical Assistant V1.0.0.
Results by difficulty · Exploratory analysis
In the most challenging cases
The highest-difficulty group showed the largest absolute improvement in Top-3 diagnostic accuracy among the difficulty groups analysed.
≈44.8%
Relative reduction in Top-3 errors in the most challenging cases
Ratings and use
Rated positively by participating physicians
Mean ratings from the 13 participating physicians.
AI-assisted consultations lasted about 35 seconds longer on average, an increase of 10.7%.
Scientific article
Read the full study
Let's explore the potential of Medsys AI in your organisation
Let's talk about how you can integrate our Clinical Assistant into your organisation.