Design: Descriptive study.
Purpose: To identify the main aspects that currently complicate the integration of artificial intelligence (AI) in ophthalmic settings.
Methods: Based on an extensive review of state-of-the-art literature of AI applied to Ophthalmology plus interviews with multidisciplinary, international experts, we identified the most relevant aspects to consider during AI design to generate trustworthy (i.e., transparent, robust, and sustainable) AI systems and, consequently, facilitate a subsequent successful integration in real-world ophthalmic settings.
Results: Several essential aspects to consider were identified:
1) The reliability of the human annotations that are used for establishing the reference standard an AI system learns from, or for setting robust observer studies that allow for fair human-AI performance comparison.
2) The ability of an AI system to generalize across populations, ophthalmic settings, and data acquisition protocols in order to avoid the negative consequences of algorithmic bias and lack of domain adaptation.
3)The integration of multimodal data for AI development to consider multiple contexts when available (phenotyping, genotyping, systemic variables, patient medical history...).
4) The importance of providing interpretable AI-based predictions to open the "black box" and increase trust and clinical usability.
5) A plan to monitor the impact of AI on the clinical workflow, i.e., the adaptation of healthcare providers and patients to the new technology, human-AI interaction, cost-benefit analyses...
6) The necessity to update current regulations to accelerate and control AI integration and all related aspects, such as patient privacy, systems' updates, and liability.
Conclusions: It is important that healthcare providers in Ophthalmology consider these aspects and their consequences when thinking of AI in practice. It is key that all involved stakeholders collaborate and interact from the beginning of the AI design process to ensure a good alignment with real-world clinical needs and settings. This way, it will be possible to generate trustworthy AI solutions and close the gap between development and deployment, so that the AI benefits currently shown on paper reach the final users.