Purpose: To investigate changes in peer-reviewed evidence on commercially available radiologic artificial intelligence (AI) products from 2020 to 2023.
Methods and materials: A comprehensive review of the literature published between January 2015 and March 2023 of CE certified radiological AI products (according to www.healthairegister.com) was performed. Complying with the previous systematic review, this follow-up study categorized the publications according to the hierarchical model of efficacy: from technical and diagnostic accuracy (levels 1 and 2) to impacts on clinical decision-making and patient outcomes (level 3-5) or socio-economic impact (level 6).
Results: By March 2023, 91 vendors were identified, offering a total of 179 products, with 120 of these (67%) having peer-reviewed evidence, compared to 36% in 2020. In 2023, there were 662 publications on these 120 products, compared to 237 publications on 36 products in 2020. An increase (22 to 25%) was found in publications focusing on technical or potential clinical efficacy. The majority of publications described the diagnostic accuracy of the product (level 2), although relatively showing a decrease (55 to 52%). For the higher levels of efficacy (level 3-6) the respective contribution to the total remained the same as 2020 (23%).
Conclusion: While there is an increase in the amount of publications validating AI products, the majority of publications continue to describe the lower levels of efficacy. This suggests that even though the field has been maturing, we still have limited knowledge and evidence of the clinical impact of AI products in radiology.
Limitations: Several products have a high number of publications, which may cause them to be overrepresented in the total.