Explainability and Utility with Pose in Instance Segmentation Algorithms : Flow visualisations for interpreting segmentation in medical imaging

Sammanfattning

Usage of artificial intelligence techniques in medical contexts necessitates their explainability to achieve reliability and trust. Visual explanations for how computer vision models process an image often attempt localisation of critical features which the objective function of the model is most sensitive to. However, segmentation effectively performs this localisation by default. So this approach is not fruitful for achieving explainability for segmentation models. Since the utilisation of pose estimation has been shown to be fruitful in instance segmentation, this study aims to probe the efficacy of pose visualisations as a means to understand behaviour and diagnose problems in instance segmentation models. The Cellpose cellular segmentation framework generates pose predictions as pixel-level feature representations of input in its process for instance segmentation of cells in microscopic slides. We utilise this model to generate flow visualisations and test human performance on counting cells in images when aided with flow visualisations compared with baseline human performance and standard segmentation support. The results did not show significant evidence to support flow visualisations improving performance. We also perform sensitivity analysis on preliminary pose predictions in the Cellpose architecture using the RISE perturbation method for generating saliency maps to interpret how pose predictions affect the segmentation process, providing insight into stability of segmentation near cell boundaries and clusters.