Assessing the robustness of AI-generated lesion risk scores acquired under various imaging conditions

Sammanfattning

Background and Aim: Artificial intelligence (AI) in mammography screening can aid cancer detection. Commercially available AI systems can assign region and exam scores based on malignancy suspicion. Despite promising suggestions, clinical implementation is limited- partly due to a lack of trust among radiologist and prospective evaluation. Inevitably, this leads to the question of how an AI system intended to be used in mammography screening can be validated. This study aims at assessing the robustness of an AI system’s response to various image acquisition conditions using an anthropomorphic breast phantom. Precision of obtained risk scores and possible relationships between exposure parameters and risk scores will be investigated. Material and Methods: Digital mammography (DM) and digital breast tomosynthesis (DBT) images of a breast phantom containing one spiculated mass were acquired using Siemens MAMMOMAT Inspiration. Exposure parameters such as tube voltage (kV) and tube loading (mAs) were varied relative to those obtained using automatic exposure control (AEC). For DM, this was tested using the following anode/filter combinations: W/Rh, Mo/Mo and Mo/Rh, and W/Rh was used for DBT. Five repeated exposures were made for each combination of kV and mAs. AEC mode was used while varying other settings, including phantom position, compression plate release, exclusion of highly attenuating “chest wall” and combinations thereof. Most setups included 20 repeated exposures. The organ dose was extracted from the DICOM header and used as substitute for average glandular dose (AGD). Images were analyzed in an AI system for region and exam scores. Number of cases where the AI system presented a region score of the lesion (lesion risk score) was recorded. Linear regression analysis assessed possible associations between kV/mAs and lesion risk scores. Mean lesion risk scores from AEC data sets were compared pairwise. Results: The AI system provided lesion risk scores for all images acquired using AEC mode. When varying the exposure parameters for DM, scores were given in 93%, 94% and 92% of the images for W/Rh, Mo/Mo and Mo/Rh respectively. Generally, a wide range of risk scores were reported within each DM data set. The precision was better for DBT and the risk scores were higher, resulting in significant difference in mean lesion risk scores (-25.1, 95% CI (-28.9, -21.3), p < 0.001) between DM and DBT. No other significant differences between AEC data sets were found for DM. Moving the phantom (in contrast to centering) in DBT showed a significant difference in mean lesion risk scores (3.18, 95% CI (0.06, 6.30), p = 0.042). Weak to somewhat strong significant linear association between each exposure parameter and lesion risk scores were found in most DM imaging conditions, varying by anode/filter combination. However, DBT showed consistent moderate to strong significant positive linear relationships between kV and mAs respectively, and lesion risk scores. Conclusion: The unexpected wide range of lesion risk scores within data sets could be a sign of deficient precision of AI systems and the possible reason for this needs to be further investigated.