Deep learning algorithm may streamline lung cancer radiotherapy treatment
“The biggest translation gap in AI applications to medicine is the failure to study how to use AI to improve human clinicians, and vice versa,” said corresponding author Raymond Mak, MD, of the Brigham’s Department of Radiation Oncology. “We’re studying how to make human-AI partnerships and collaborations that result in better outcomes for patients. The benefits of this approach for patients include greater consistency in segmenting tumors and accelerated times to treatment. The clinician benefits include a reduction in mundane but difficult computer work, which can reduce burnout and increase the time they can spend with patients.”
The researchers used CT images from 787 patients to train their model to distinguish tumors from other tissues. They tested the algorithm’s performance using scans from over 1,300 patients from increasingly external datasets. Developing and validating the algorithm involved close collaboration between data scientists and radiation oncologists. For example, when the researchers observed that the algorithm was incorrectly segmenting CT scans involving the lymph nodes, they retrained the model with more of these scans to improve its performance.
Finally, the researchers asked eight radiation oncologists to perform segmentation tasks as well as rate and edit segmentations produced by either another expert physician or the algorithm (they were not told which). There was no significant difference in performance between human-AI collaborations and human-produced (de novo) segmentations. Intriguingly, physicians worked 65 percent faster and with 32 percent less variation when editing an AI-produced segmentation compared to a manually produced one, even though they were unaware of which one they were editing. They also rated the quality of AI-drawn segmentations more highly than the human expert-drawn segmentations in this blinded study.
Going forward, the researchers plan to combine this work with AI models they designed previously that can identify “organs at risk” of receiving undesired radiation during cancer treatment (such as the heart) and thereby exclude them from radiotherapy. They are continuing to study how physicians interact with AI to ensure that AI-partnerships help, rather than harm, clinical practice, and are developing a second, independent segmentation algorithm that can verify both human and AI-drawn segmentations.
“This study presents a novel evaluation strategy for AI models that emphasizes the importance of human-AI collaboration,” said co-author Hugo Aerts, PhD, of the Department of Radiation Oncology. “This is especially necessary because in silico (computer-modeled) evaluations can give different results than clinical evaluations. Our approach can help pave the way towards clinical deployment.”
This study was funded by the National Institutes of Health (U24CA194354, U01CA190234, and U01CA209414).