Deep learning for extremity radiographs confounded by labels
Grad-CAM heatmaps for deep learning models trained on (A) original radiograph, shows emphasis on laterality and/or technologist initial labels; (B) radiograph with label covered by black box, shows emphasis on anatomic features, such as bones. (Colors toward red end of spectrum indicate greater emphasis, whereas colors toward blue end of spectrum indicate less importance.)
Credit: American Roentgen Ray Society (ARRS), American Journal of Roentgenology (AJR)
Convolutional neural networks trained to identify abnormalities on upper extremity radiographs are susceptible to a ubiquitous confounding image feature that could limit their clinical utility: radiograph labels.
According to an open-access Editor’s Choice article in ARRS’ American Journal of Roentgenology (AJR), convolutional neural networks (CNN) trained to identify abnormalities on upper extremity radiographs are susceptible to a ubiquitous confounding image feature that could limit their clinical utility: radiograph labels.
“We recommend that such potential image confounders be collected when possible during dataset curation, and that covering these labels be considered during CNN training,” wrote corresponding author Paul H. Yi from the University of Maryland’s Medical Intelligent Imaging Center in Baltimore.
Yi and team’s retrospective study evaluated 40,561 upper extremity musculoskeletal radiographs from Stanford’s MURA dataset that were used to train three DenseNet-121 CNN classifiers. Three inputs were used to distinguish normal from abnormal radiographs: original images with both anatomy and labels; images with laterality and/or technologist labels subsequently covered by a black box; images where anatomy had been removed and only labels remained.
For the original radiographs, AUC was 0.844, frequently emphasizing laterality and/or technologist labels for decision-making. Covering these labels increased AUC to 0.857 (p=.02) and redirected CNN attention from the labels to the bones. Using labels alone, AUC was 0.638, indicating that radiograph labels are associated with abnormal examinations.
“While we can infer that labels are associated with normal versus abnormal disease categories,” the authors of this AJR article added, “we cannot determine the specific aspect of the labels that resulted in their being confounding factors.”
An electronic supplement to this AJR article is available here.
Founded in 1900, the American Roentgen Ray Society (ARRS) is the first and oldest radiological society in North America, dedicated to the advancement of medicine through the profession of radiology and its allied sciences. An international forum for progress in medical imaging since the discovery of the x-ray, ARRS maintains its mission of improving health through a community committed to advancing knowledge and skills with an annual scientific meeting, monthly publication of the peer-reviewed American Journal of Roentgenology (AJR), quarterly issues of InPractice magazine, AJR Live Webinars and Podcasts, topical symposia, print and online educational materials, as well as awarding scholarships via The Roentgen Fund®.
Journal: American Journal of Roentgenology
DOI: 10.2214/AJR.21.26882
Method of Research: Observational study
Subject of Research: People
Article Title: Deep Learning Algorithms for Interpretation of Upper Extremity Radiographs: Laterality and Technologist Initial Labels As Confounding Factors
Article Publication Date: 10-Nov-2021
COI Statement: No disclosures relevant to this work.
MEDIA CONTACT:
Logan K. Young, PIO
44211 Slatestone Court
Leesburg, VA 20176
703-858-4332
lyoung@arrs.org
All latest news from the category: Medical Engineering
The development of medical equipment, products and technical procedures is characterized by high research and development costs in a variety of fields related to the study of human medicine.
innovations-report provides informative and stimulating reports and articles on topics ranging from imaging processes, cell and tissue techniques, optical techniques, implants, orthopedic aids, clinical and medical office equipment, dialysis systems and x-ray/radiation monitoring devices to endoscopy, ultrasound, surgical techniques, and dental materials.
Newest articles
‘Entirely unanticipated’ role of protein netrin1 in spinal cord development
Known for its axon guidance properties, new research suggests protein is critical in guiding neural development. Scientists at the Eli and Edythe Broad Center of Regenerative Medicine and Stem Cell Research…
AI to improve brain cancer diagnosis, monitoring, treatment
Recommendations published in The Lancet Oncology call for good clinical practice of new technologies to modernize decades-old standard of care for brain cancer patients. An international, multidisciplinary team of leading…
AI tool ‘sees’ cancer gene signatures in biopsy images
AI tool reads biopsy images… To determine the type and severity of a cancer, pathologists typically analyze thin slices of a tumor biopsy under a microscope. But to figure out…