Software Quality Evaluation of AI/ML-Based Neuroimaging Tools - A Simulation Study Using BRAPH 2

Sammanfattning

The increasing integration of artificial intelligence (AI) and machine learning (ML)into neuroimaging research has amplified concerns regarding the quality, reproducibility,and trustworthiness of such software systems. Existing evaluation practicesoften lack standardized and systematic frameworks, limiting their ability to testsoftware quality, especially for AI/ML-based neuroimaging software system. Thisstudy addresses this gap by developing and applying a simulation-based frameworkto evaluate key quality attributes—transparency, functional correctness, and robustness—in AI/ML-based brain imaging analysis pipelines. The framework leveragesthe Watts–Strogatz network model to generate controlled, simulated brain connectivitydatasets, enabling rigorous and repeatable testing. Two analysis pipelineswithin BRAPH 2 (Brain Analysis using Graph Theory 2), an open-source MATLABbasedneuroimaging software for brain network analysis, are tested: a graph theory–based pipeline and a deep learning–based analysis pipeline. Both pipelines areevaluated against simulated datasets, successfully identifying the predefined salientbrain regions and maintaining stable performance across repeated runs and randomnoise situations. Transparency was further enhanced through a graphical user interfaceand visualization modules that allow inspection of intermediate and final outputs.These results demonstrate that the proposed framework can effectively verifycritical quality attributes in a controlled environment. The established methodologyprovides a robust and accessible foundation for extending validation to real-worldneuroimaging datasets and for guiding future st andards in the quality evaluationof AI/ML-based neuroscience software.