Smartphone apps used to diagnose skin cancer are poorly regulated and are often unreliable, British experts warn today.
The apps are designed to provide a risk assessment of a new or changing mole, using specialised algorithms to try to detect possible skin cancers – but UK researchers have analysed a series of studies produced to evaluate the accuracy of six different apps.
The results, published in latest edition of The BMJ, reveal a mixed picture, with only a small number of studies showing variable and unreliable test accuracy among the apps evaluated.
The researchers, from the University of Birmingham’s Institute of Applied Health Research and the Centre of Evidence-Based Dermatology at the University of Nottingham, UK, also voice concerns about the quality of the studies themselves, which evaluated apps using images taken by experts rather than by app users.
Many studies did not identify if lesions identified as ‘low risk’ by the apps were benign, which, they say, further compromises the conclusions.
Lead researcher Dr Jac Dinnes, from the University of Birmingham, UK, said: “This is a fast-moving field and it’s really disappointing that there is not better quality evidence available to judge the efficacy of these apps. It is vital that healthcare professionals are aware of the current limitations both in the technologies and in their evaluations.”
App manufacturers can currently apply CE marks to smartphone apps without being subject to independent inspection by bodies such as the UK Medicines and Healthcare Products Regulatory Agency (MHRA). The researchers note this may change with new Medical Device Regulations, which is due to come into force this year.
Co-author Professor Jon Deeks, Professor of Biostatistics in the Institute of Applied Health Research, adds: “Regulators need to become alert to the potential harm that poorly performing algorithm-based diagnostic or risk monitoring apps create. We rely on the CE mark as a sign of quality, but the current CE mark assessment processes are not fit for protecting the public against the risks that these apps present.”
Professor Hywel Williams, Professor of Dermatology at the University of Nottingham, added: “Although I was broad minded on the potential benefit of apps for diagnosing skin cancer, I am now worried given the results of our study and the overall poor quality of studies used to test these apps.”
The study recommends that any future studies of smartphone apps must be based on clinically relevant population of smartphone users who may have concerns about their risk of skin cancer; must include all skin lesions identified by smartphone users; and must include clinical follow-up of benign lesions.
Freeman H, Dinnes J, ChuChu N et al. Algorithm-based smartphone ‘apps’ for assessment of the risk of skin cancer in adults: a systematic review of diagnostic accuracy studies. BMJ 11 February 2020
Leave a Reply