Background: Currently, many hospitals are implementing AI software. However, clear clinical implementation procedures are not yet available. In order to exchange experiences, two interventional stroke centres (Radboudumc and Rijnstate) collaborated in the prospective evaluation of an AI tool for stroke diagnostics.
Methodology: Primary aim of StrokeViewer (Nicolab) implementation in both centres was diagnostic support in detecting large vessel occlusions (LVO) in anterior cerebral circulation. Additionally, in Rijnstate analysis of cerebral CT perfusion (CTP) was available. In Radboudumc, LVO results were available after log in to the StrokeViewer server. In Rijnstate, results were pushed to PACS as a pdf-report. Trial period in Radboudumc was 12 months, in Rijnstate 7 months. The performance of proximal LVO detection was compared with radiologists' assessments. Users filled in a questionnaire on user experience at several time points. In Radboudumc, the use was monitored by individual log-in information.
Results: Quantitative evaluation of ICA, M1 and proximal M2 occlusion detection (prevalence 18%) resulted in a case base sensitivity and specificity of 74% and 91% in Rijnstate (n=276) and 77% and 91% in Radboudumc (n=516). The use of the tool decreased over time. Radiologists unfamiliar with LVO assessment tended to value the AI report more than experienced radiologists. The net promoter scores were -56% in Radboudumc and -65% in Rijnstate. The tool was considered user friendly (7.2/10). CTP assessment in Rijnstate was used more frequently than LVO detection.
Conclusion: This evaluation aids to understand some of the challenges involved in clinical implementation and acceptance by users of AI tools. Findings are consistent for both centres. Success is not only dependent on the product and its performance, but also on clinical goal setting, expectations, context and implementation choices. Sharing experience within the NVvR AInetwork can help to gain insights into crucial factors for success ("Gluren-bij-de-buren").