CCR : Classification-Constrained Retrieval for City-Scale Geolocalization of Images

Sammanfattning

In recent years, investigative journalists and digital forensics experts have manually found and verified the locations of image evidence to track human rights violations. Visual Geolocalization (VG) aims to automate this process, predicting where an image was taken using only pixels as input. This thesis focuses on improving fine-grained city-scale geolocalization, i.e. geolocalization within the confines of a city. Typically, this has been performed through image retrieval. This thesis hypothesizes that a hybrid model combining image classification and retrieval could improve the performance of geolocation prediction, and the robustness under domain shift. Earlier attempts at such a hybrid model of classification-constrained retrieval (CCR) have been made for global geolocalization (where it has shown promise), but never for city-scale geolocation prediction on moderately-sized datasets. To fill this knowledge gap, this thesis conducts two experiments, comparing the performance of CCR to a state-of-the-art retrieval model on a dataset of street view images from Stockholm. The first experiment uses query images from the same distribution as the database, and the best configuration of CCR correctly predicts the location within 100 meters for 48.5% of queries compared to 44.9% for the retrieval model. The second experiment uses query images from another distribution (social media images), again, CCR outperforms the retrieval model, correctly predicting 20.8% of the queries within 100 meters compared to 12.8%. These findings demonstrate the great potential for CCR in city-level geolocalization, particularly in handling challenging scenarios reminiscent of real-world conditions.