Evaluating NR-IQA and NR-PCQA Methods on Weather-Distorted Data in Autonomous Driving

Sammanfattning

In autonomous driving, balancing the quality of training data is essential for developing safe and reliable self-driving models. A necessary prerequisite for striking this balance is having reliable methods for assessing data quality. This thesis investigates methods for evaluating the quality of images and point clouds without relying on reference data, known as no-reference image quality assessment (NR-IQA) and no-reference point cloud quality assessment (NRPCQA), respectively. Five NR-IQA methods (IL-NIQE, TOPIQ, DBCNN, QualiCLIP, and Q-Align) and two NR-PCQA methods (MM-PCQA and MSPCQE) were evaluated. The image data was obtained from two drives in authentic winter conditions, while the point clouds were obtained from a test track. Both the images and the point clouds were synthetically distorted using artificial fog and rain, and the NR-IQA and NR-PCQA methods were evaluated on their ability to rank versions of the images or point clouds by distortion level. Neither of the two NR-PCQA methods demonstrated reliable performance in ranking the distorted point clouds. While MM-PCQA outperformed MS-PCQE, it also failed to generate reliable rankings. This suggests that NR-PCQA methods are not yet mature enough to be employed in the autonomous driving domain. Of the evaluated NR-IQA methods, Q-Align and IL-NIQE achieved the best performance, significantly outperforming the other methods. These results suggest that large multimodal models and natural scene statistics are viable approaches for assessing the quality of weatherdistorted images.