Revealing transparency gaps in publicly available COVID-19 datasets used for medical artificial intelligence development-a systematic review
Alderman, Joseph E Charalambides, Maria Sachdeva, Gagandeep Laws, Elinor Palmer, Joanne Lee, Elsa Menon, Vaishnavi Malik, Qasim Vadera, Sonam Calvert, Melanie
Alderman, Joseph E
Charalambides, Maria
Sachdeva, Gagandeep
Laws, Elinor
Palmer, Joanne
Lee, Elsa
Menon, Vaishnavi
Malik, Qasim
Vadera, Sonam
Calvert, Melanie
Citations
Altmetric:
Affiliation
University Hospitals Birmingham NHS Foundation Trust; University of Birmingham; University Hospital Southampton NHS Foundation Trust; The Royal Wolverhampton NHS Trust; King's College London; Birmingham Women's and Children's NHS Foundation Trust; University Hospitals of Leicester NHS Trust; NIHR Blood and Transplant Research Unit (BTRU); Massachusetts Institute of Technology; The Hospital for Sick Children; SickKids Research Institute; University of Cambridge; Roche Diagnostics; University College London; PATH; Wellcome Trust; Independent Cancer Patients Voice; Oxford University Hospitals NHS Foundation Trust; Moorfields Eye Hospital; NIHR Biomedical Research Centre
Other Contributors
Publication date
2024-10-23
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
During the COVID-19 pandemic, artificial intelligence (AI) models were created to address health-care resource constraints. Previous research shows that health-care datasets often have limitations, leading to biased AI technologies. This systematic review assessed datasets used for AI development during the pandemic, identifying several deficiencies. Datasets were identified by screening articles from MEDLINE and using Google Dataset Search. 192 datasets were analysed for metadata completeness, composition, data accessibility, and ethical considerations. Findings revealed substantial gaps: only 48% of datasets documented individuals' country of origin, 43% reported age, and under 25% included sex, gender, race, or ethnicity. Information on data labelling, ethical review, or consent was frequently missing. Many datasets reused data with inadequate traceability. Notably, historical paediatric chest x-rays appeared in some datasets without acknowledgment. These deficiencies highlight the need for better data quality and transparent documentation to lessen the risk that biased AI models are developed in future health emergencies.
Citation
Alderman JE, Charalambides M, Sachdeva G, Laws E, Palmer J, Lee E, Menon V, Malik Q, Vadera S, Calvert M, Ghassemi M, McCradden MD, Ordish J, Mateen B, Summers C, Gath J, Matin RN, Denniston AK, Liu X. Revealing transparency gaps in publicly available COVID-19 datasets used for medical artificial intelligence development-a systematic review. Lancet Digit Health. 2024 Nov;6(11):e827-e847. doi: 10.1016/S2589-7500(24)00146-8.
Type
Article