AI triage tool fails to catch dangerous cases that human error missed
A digital symptom checker used in Swedish primary care performed poorly on cases that had previously resulted in harmful triage errors, suggesting these tools may replicate—or worsen—the very safety gaps they're designed to fix. The finding raises questions about whether automated triage systems are ready for wider deployment across healthcare systems relying on them to manage patient volume.
Originaltitel: Evaluating digital triage symptom checker with historical triage-related adverse events
BackgroundOnline symptom checkers are increasingly used for diagnostic support and triage. However, evidence on their performance and evaluations with real-world data remains limited.ObjectiveThe aim of this study was to evaluate the performance of a digital symptom checker using clinical vignettes derived from real-world cases that had previously been incorrectly triaged.MethodsA patient-facing, rule-based digital symptom checker used in Swedish primary care was assessed in this study. Vignettes were constructed from cases reported to the Swedish Health and Social Care Inspectorate due to erroneous telephone triage. We hypothesized that the digital symptom checker could provide appropriate triage for these cases. Seven physicians independently simulated patients by entering symptoms in the symptom checker based on each vignette. Triage outcomes were assessed against the Swedish National Triage Guidelines (RGS), evaluating the accuracy and the safety of the triage recommendation.ResultsA total of 69 unique vignettes yielded 483 individual trials. After excluding 93 trials due to significant deviations from the original vignette description (adding or omitting symptoms), 390 trials were included in the primary analysis. The symptom checker achieved 91% accuracy (95% CI 88-94%) and 94% safety (95% CI 91-96%).ConclusionsThe symptom checker demonstrated high accuracy and safety when triaging a subset of vignettes based on real-world cases that had been previously erroneously triaged. This study also highlights the difficulties of using vignettes when evaluating symptom checkers. To our knowledge, this is the first study to evaluate such systems using vignettes based on actual patient cases with known triage errors.