The value of standards for health datasets in artificial intelligence-based applications
Author
Arora, AnmolAlderman, Joseph E
Palmer, Joanne
Ganapathi, Shaswath
Laws, Elinor
McCradden, Melissa D
Oakden-Rayner, Lauren
Pfohl, Stephen R
Ghassemi, Marzyeh
McKay, Francis
Treanor, Darren
Rostamzadeh, Negar
Mateen, Bilal
Gath, Jacqui
Adebajo, Adewole O
Kuku, Stephanie
Matin, Rubeta
Heller, Katherine
Sapey, Elizabeth
Sebire, Neil J
Cole-Lewis, Heather
Calvert, Melanie
Denniston, Alastair
Liu, Xiaoxuan
Publication date
2023-10-26Subject
Practice of medicine
Metadata
Show full item recordAbstract
Artificial intelligence as a medical device is increasingly being applied to healthcare for diagnosis, risk stratification and resource allocation. However, a growing body of evidence has highlighted the risk of algorithmic bias, which may perpetuate existing health inequity. This problem arises in part because of systemic inequalities in dataset curation, unequal opportunity to participate in research and inequalities of access. This study aims to explore existing standards, frameworks and best practices for ensuring adequate data diversity in health datasets. Exploring the body of existing literature and expert views is an important step towards the development of consensus-based guidelines. The study comprises two parts: a systematic review of existing standards, frameworks and best practices for healthcare datasets; and a survey and thematic analysis of stakeholder views of bias, health equity and best practices for artificial intelligence as a medical device. We found that the need for dataset diversity was well described in literature, and experts generally favored the development of a robust set of guidelines, but there were mixed views about how these could be implemented practically. The outputs of this study will be used to inform the development of standards for transparency of data diversity in health datasets (the STANDING Together initiative).Citation
Arora A, Alderman JE, Palmer J, Ganapathi S, Laws E, McCradden MD, Oakden-Rayner L, Pfohl SR, Ghassemi M, McKay F, Treanor D, Rostamzadeh N, Mateen B, Gath J, Adebajo AO, Kuku S, Matin R, Heller K, Sapey E, Sebire NJ, Cole-Lewis H, Calvert M, Denniston A, Liu X. The value of standards for health datasets in artificial intelligence-based applications. Nat Med. 2023 Oct 26. doi: 10.1038/s41591-023-02608-w. Epub ahead of print. PMID: 37884627.Type
ArticlePMID
37884627Journal
Nature MedicinePublisher
Nature Researchae974a485f413a2113503eed53cd6c53
10.1038/s41591-023-02608-w