[๐๐๐ ๐๐ผ๐๐ฒ๐ฟ๐ป๐ฎ๐ป๐ฐ๐ฒ & ๐ฃ๐ผ๐น๐ถ๐ฐ๐ ๐๐ผ๐ฟ๐ป๐ฒ๐ฟ]
A healthcare AI model can be accurate when tested and still be unreliable when used. Why? Because clinical reliability is not a fixed property of a model. It depends on which patients encounter it, where and how it is used, and whether its performance remains acceptable as clinical practice changes.
A new Review in
@CellCellPress, "Avoiding common failures in AI for health and medicine," examines reliability across both predictive and generative AI. The authors highlight three recurring and often interconnected failures.
1๏ธโฃ ๐๐ฟ๐ฟ๐ผ๐ป๐ฒ๐ผ๐๐ ๐ผ๐๐๐ฝ๐๐๐ ๐บ๐ฎ๐ ๐ฏ๐ฒ ๐ฑ๐ถ๐ณ๐ณ๐ถ๐ฐ๐๐น๐ ๐๐ผ ๐ฑ๐ฒ๐๐ฒ๐ฐ๐
Predictive models can learn from incomplete records, inconsistent labels, or proxies that do not accurately represent the underlying clinical condition. Generative AI can produce fluent and authoritative responses that are clinically incorrect, making unreliable outputs particularly difficult to recognize.
2๏ธโฃ ๐๐๐ฒ๐ฟ๐ฎ๐ด๐ฒ ๐ฝ๐ฒ๐ฟ๐ณ๐ผ๐ฟ๐บ๐ฎ๐ป๐ฐ๐ฒ ๐ฐ๐ฎ๐ป ๐ฐ๐ผ๐ป๐ฐ๐ฒ๐ฎ๐น ๐๐ป๐ฒ๐พ๐๐ฎ๐น ๐ฝ๐ฒ๐ฟ๐ณ๐ผ๐ฟ๐บ๐ฎ๐ป๐ฐ๐ฒ
A system may perform differently across patient populations, hospitals, scanners, input formats, or clinical settings for reasons unrelated to genuine clinical differences. Strong overall results may therefore coexist with clinically important failures for particular populations or contexts.
3๏ธโฃ ๐ฃ๐ฒ๐ฟ๐ณ๐ผ๐ฟ๐บ๐ฎ๐ป๐ฐ๐ฒ ๐ฐ๐ฎ๐ป ๐ฑ๐ฒ๐ด๐ฟ๐ฎ๐ฑ๐ฒ ๐ฎ๐ณ๐๐ฒ๐ฟ ๐ฑ๐ฒ๐ฝ๐น๐ผ๐๐บ๐ฒ๐ป๐
Changes in patient populations, workflows, documentation practices, coding systems, or data collection can make a previously validated model less reliable. These changes may be gradual, difficult to observe, and entangled with the model's own effects on clinical decisions and recorded outcomes.
The paper also explains why familiar technical fixes are often insufficient. Data cleaning, model refinement, retrieval systems, recalibration, and retraining can help, but they frequently depend on assumptions that don't hold in dynamic clinical environments. Addressing one known failure may also leave other failures undetected or introduce new ones.
The authors therefore frame reliability as a lifecycle responsibility. Reliable healthcare AI is not achieved by optimizing a model once. It requires continuous coordination across technical evaluation, clinical oversight, monitoring, and institutional governance.
Olawale Salaudeen, et al. Avoiding common failures in AI for health and medicine. Cell. 2026; 189: 5893-5904.
doi.org/10.1016/j.cell.2026.โฆ
#AIGovernance #HealthAI #ResponsibleAI #TrustworthyAI #ClinicalAI #AISafety #GenerativeAI #DigitalHealth #HealthcareInnovation #DukeNUS
[๐๐ ๐๐ผ๐๐ฒ๐ฟ๐ป๐ฎ๐ป๐ฐ๐ฒ & ๐ฃ๐ผ๐น๐ถ๐ฐ๐ ๐๐ผ๐ฟ๐ป๐ฒ๐ฟ] is a regular series in which we will share and reflect on relevant research, frameworks, and policy developments shaping trustworthy and ethical AI in healthcare.