ANTONI: Towards a virtual pathologist via multi-modal foundation models for computational pathology

Background

There is a worldwide shortage of pathologists while healthcare demands continue to grow. Our Computational Pathology Group has demonstrated that AI can match and even exceed human expert performance in diagnostic tasks like assessing breast and prostate cancer severity. However, current AI systems lack the ability to work across multiple organs and diseases, are not interactive and have limited explainability.

Aim

Supported by the Ammodo Science Award for groundbreaking research, the ANTONI project aims at developing a virtual assistant capable of supporting pathologists in their clinical diagnosis across multiple tissue types and diseases. ANTONI will combine advanced computer vision and natural language processing to analyze digitized tissue samples and support pathologists in their diagnosis across multiple tissue types and diseases in the form of an interactive virtual assistant. This will characterize ANTONI as an "explainable AI" model, which will provide pathologists with feedback and explanations behind each diagnosis in plain text, with potential for boosting clinician confidence and enabling broader AI adoption in healthcare.

The project aims to leverage over 100,000 whole-slide images from multiple European medical centers to build and validate multimodal foundation models and end-to-end learning techniques that can leverage entire pathology whole-slide images. All models, code, and data will be made freely available to accelerate global progress in AI-assisted pathology and directly address the critical shortage of pathologists worldwide.

Funding

People

Francesco Ciompi

Francesco Ciompi

Associate Professor

Computational Pathology Group

Jeroen van der Laak

Jeroen van der Laak

Professor

Computational Pathology Group

Geert Litjens

Geert Litjens

Professor

Computational Pathology Group

Sander Moonemans

Sander Moonemans

PhD Candidate

Sebastiaan Ram

Sebastiaan Ram

PhD Candidate

Computational Pathology Group

Meta Tax

Meta Tax

Project Manager

Computational Pathology Group

Thomas Mulder

Thomas Mulder

Technical Data Manager

Computational Pathology Group