Artificial intelligence (AI) technology is not yet good enough to replace radiologists when it comes to spotting breast cancer following screening, a review in The BMJ says today.
Although previous research has suggested that AI systems outperform humans and could be used instead of experienced radiologists, a review of 23 studies went on to highlight evidence gaps and concerns about the methods used.
It was why the UK National Screening Committee commissioned researchers at the University of Warwick, UK, to examine the accuracy of AI for the detection of breast cancer in mammography screening practice.
They reviewed 12 studies carried out since 2010 involving data for 131,822 screened women in Sweden, USA, Germany, the Netherlands and Spain.
They found the quality of the methods used in all of the studies was poor and their applicability to European or UK breast cancer screening programmes was low.
Three large studies involving 79,910 women compared AI systems with the clinical decisions of the original radiologist. Of these, 1,878 had screen detected cancer or interval cancer within 12 months of screening.
However, 34 out of 36 (94%) of AI systems evaluated in these three studies were found to be less accurate than a single radiologist, and all were less accurate than the consensus of two or more radiologists, which is the standard practice in Europe.
Five smaller studies involving 1,086 women claimed all of the AI systems evaluated were more accurate than a single radiologist, but the research team found these studies were at high risk of bias and their promising results are not replicated in larger studies.
In three studies, AI that was used as a pre-screen to triage which mammograms needed to be examined by a radiologist and those that did not screened out 53%, 45%, and 50% of women at low risk but also 10%, 4%, and 0% of cancers detected by radiologists.
“Current evidence on the use of AI systems in breast cancer screening is a long way from having the quality and quantity required for its implementation into clinical practice,” the research team write.
“Well designed comparative test accuracy studies, randomised controlled trials, and cohort studies in large screening populations are needed which evaluate commercially available AI systems in combination with radiologists in clinical practice.”
Freeman K, Geppert J, Stinton C et al. Use of artificial intelligence for image analysis in breast cancer screening programmes: systematic review of test accuracy. BMJ 2 September 2021. DOI: 10.1136/bmj.n1872
Leave a Reply