Appraisal of AI-generated dermatology literature reviews
Passby, Lauren Madhwapathi, Vidya Tso, Simon Wernham, Aaron G H
Passby, Lauren
Madhwapathi, Vidya
Tso, Simon
Wernham, Aaron G H
Citations
Altmetric:
Affiliation
University Hospitals Birmingham NHS Foundation Trust; South Warwickshire NHS Foundation Trust; Walsall Healthcare NHS Trust
Other Contributors
Publication date
2024-12
Subject
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Background: Artificial intelligence (AI) tools have the potential to revolutionize many facets of medicine and medical sciences research. Numerous AI tools have been developed and are in continuous states of iterative improvement in their functionality.
Objectives: This study aimed to assess the performance of three AI tools: The Literature, Microsoft's Copilot and Google's Gemini in performing literature reviews on a range of dermatology topics.
Methods: Each tool was asked to write a literature review on five topics. The topics chosen have recently had peer-reviewed systematic reviews published. The outputs of each took were graded on their evidence and analysis, conclusions and references on a 5-point Likert scale by three dermatologists who are working in clinical practice, have completed the UK dermatology postgraduate training examination and are partaking in continued professional development.
Results: Across all five topics chosen, the literature reviews written by Gemini scored the highest. The mean score for Gemini for each review was 10.53, significantly higher than the mean scores achieved by The Literature (7.73) and Copilot (7.4) (p < 0.001).
Conclusions: This paper shows that AI-generated literature reviews can provide real-time summaries of medical literature across a range of dermatology topics, but limitations to their comprehensiveness and accuracy are apparent.
Citation
Passby L, Madhwapathi V, Tso S, Wernham A. Appraisal of AI-generated dermatology literature reviews. J Eur Acad Dermatol Venereol. 2024 Dec;38(12):2235-2239. doi: 10.1111/jdv.20237. Epub 2024 Jul 12.
Type
Article