Exploring Multi-Modal Fusion forIndoor Localisation

Sammanfattning

In today’s society, smart autonomous robots are becoming more common. A robotthat operates outdoors and whose mission is, e.g., to deliver food can be locatedusing GPS, but for indoor work that requires cm-precision and where GPS doesn’twork well, are other methods needed. This thesis has therefore investigated howto fuse radio and image data to increase accuracy, rather than relying on a singlesource of data.This thesis has implemented and evaluated three different fusion methods andcompared them with models based on radio data and image data. The three fusion methods are: two-stage MLP fusion, two-stage gated fusion, and end-to-endfusion. The radio model consists of two distinct 3-D CNN modules followed by aGRU layer, yielding a mean error of 137.09 mm. The model based on image dataconsists of two branches of MobileNetV2 models, one for RGB and one for depthimages, and resulted in a mean error of 73.47 mm.All fusion models performed better than the single modal models. The best fusion model was the two-stage gated fusion with a mean error of 57.12 mm andwith 53,602 parameters in the fusion head. This shows that feature-level fusionwith a gating mechanism outperforms an end-to-end fusion model on this dataset.The model was also evaluated on random trajectories to assess generalisation toout-of-distribution data. The models’ mean errors increase on these trajectories,indicating that they do not generalise to data outside the training distribution.