An international cluster of 117 researchers from 50 countries has arrived at a consensus on six principles that, in the team’s considered view, ought to guide the use of AI across healthcare worldwide. The principles are fairness, universality, traceability, usability, robustness and explainability.
Basing its name on the acronym formed by the first letters in these words, the group is going by an easy-to-remember name: the FUTURE-AI Consortium.
In a paper published this month in The BMJ, a sizeable subgroup of the consortium breaks down the six principles into 30 detailed recommendations for building trustworthy and readily deployable AI systems in healthcare.
The resulting framework is “dynamic,” as put by lead author Karim Lekadir, PhD, of the University of Barcelona and co-authors. This means it will evolve over time as conditions change due to technological advancements and stakeholder feedback.
Here are excerpts from the introductions to each of the six principles.
1. Fairness.
AI tools in healthcare should maintain the same performance across individuals and groups of individuals, the authors explain. AI-driven medical care “should be provided equally for all citizens,” they write. The team acknowledges that, in practice, perfect fairness “might be impossible to achieve.” Therefore:
‘AI tools should be developed such that potential AI biases are identified, reported and minimized as much as possible to achieve ideally the same—but at least highly similar—performance across subgroups.’
2. Universality.
A healthcare AI tool should be generalizable outside the controlled environment in which it was built. Specifically, the AI tool “should be able to generalize to new patients and new users and, when applicable, to new clinical sites.” More:
‘[H]ealthcare AI tools should be as interoperable and as transferable as possible so they can benefit patients and clinicians at scale.’
3. Traceability.
Medical AI tools “should be developed together with mechanisms for documenting and monitoring the complete trajectory of the AI tool,” from development and validation to deployment and usage, the authors state.
‘This will increase transparency and accountability by providing detailed and continuous information on the AI tools during their lifetime to clinicians, healthcare organizations, citizens and patients, AI developers and relevant authorities.’
4. Usability.
End-users “should be able to use an AI tool to achieve a clinical goal efficiently and safely in their real world environment,” Lekadir and colleagues write. “On one hand, this means that end-users should be able to use the AI tool’s functionalities and interfaces easily and with minimal errors.” On the other hand:
‘The AI tool should be clinically useful and safe, improve the clinicians’ productivity and/or lead to better health outcomes for the patient and avoid harm.’
5. Robustness.
Research has shown that even small, imperceptible variations in input data might lead AI models into incorrect decisions, the authors note. Biomedical and health data “can be subject to major variations in the real world—both expected and unexpected—which can affect the performance of AI tools.”
‘It is important that healthcare AI tools are designed and developed to be robust against real world variations. They should be evaluated and optimized accordingly.’
6. Explainability.
Medicine is a high-stakes discipline—one that requires transparency, reliability and accountability. Yet machine learning techniques “often produce complex models that are ‘black box’ in nature,” the authors write.
‘Explainability enables end-users to interpret the AI model and outputs, understand the capacities and limitations of the AI tool, and intervene when necessary, such as to decide to use it or not.’
Expounding on that last point, Lekadir et al. accept that explainability is a complex task. Its challenges “need to be carefully addressed during AI development and evaluation” to make sure AI explanations are “clinically meaningful and beneficial to end-users.”
The paper is available in full for free.
