Multi-view mid-fusion for object detection : Applied to detection of rock bolts on conveyor belt in mining

Sammanfattning

Multi-view video object detection on mining conveyor belts presents several challenges, one of which is occlusions across different viewpoints. As a first step of automating the removal of unwanted rockbolts, this thesis investigates the application of a state-of-the-art mid-fusion method, originally designed for multi-modal object detection, to a multi-view setting using the same type of sensor. Key challenges include aligning multiple views and generating a unified ground truth for occluded objects. Two approaches to address these challenges are evaluated: (1) aligning views using a homography matrix and (2) concatenating labels from all views into a single annotation set. The proposed methods are evaluated on a small, newly collected dataset featuring occluded bolts on a conveyor belt. Both approaches achieve improved detection accuracy compared to a single-view YOLOv5 baseline, with accuracies of 0.783 for the concatenated-label approach and 0.739 for aligning perspectives, compared to 0.696 for the baseline YOLOv5. The results demonstrate that adapting a mid-fusion model for multi-modal data to a multi-view setting can improve detection performance. However, further evaluation is required to assess generalization to larger datasets, public benchmarks, and comparisons with other state-of-the-art multi-view architectures.