Querying Fragmented Geospatial Data : An Experimental Evaluation of LLM-Grounded Retrieval Architectures

Sammanfattning

Geospatial data is typically fragmented across many independent collections, each with its own schema and naming conventions. Such data is most often stored as GeoJSON, a text format in which each record, called a feature, pairs ordinary attributes with a geometry, meaning the coordinates that place it on the map as a point, line, or area. Answering a natural language question over such fragmented data requires two steps: routing the question to the collection that holds the answer, and translating it into the spatial or attribute filtering query that retrieves the relevant data. This thesis builds and compares four LLM-based retrieval pipelines, one embedding-based and three structured. The first, Vector RAG, creates vector embeddings of every feature, then retrieves by matching the semantic similarity between the query and the feature vectors, with no discrete selection step. The other three (PageIndex, Graph-Indexed, and LLM-connected PostGIS) are structured pipelines that first select the relevant collection and then query it. PageIndex prompts the LLM to choose from a compact summary of all collections. Graph-Indexed encodes the collections’ metadata as a knowledge graph the LLM traverses using Cypher, the query language for graph databases. LLM-connected PostGIS loads the collections into PostGIS, the spatial extension for PostgreSQL, and has the LLM generate SQLqueries against it. The retrieval pipelines were evaluated using Claude Sonnet 4.6 on a benchmark of 199 author-written questions spanning five types (spatial radius, nearest-feature, attribute filter, conversational, and unanswerable), each paired with a deterministically computed groundtruth answer, drawn from 40 GeoJSON collections across five Danish municipalities. The decisive factor was less how a query was executed than whether the correct collection was selected in the first place, and selecting it reliably depended far more on how the routingindex was structured than on any agentic refinement. The strongest structured pipeline, PageIndex, matched Vector RAG without surpassing it, scoring 1.71 against 1.76 on a scalefrom 0 (incorrect) to 3 (perfect) assigned by an LLM judge, a gap too small to be statistically significant. Enriching the routing index produced almost all of the improvement, whereas adding an agentic loop yielded no significant gain on any structured pipeline and reduced quality on two, Graph-Indexed and PostGIS. The structured pipelines were not meaningfully slower than Vector RAG, yet consumed roughly three times as many tokens per question, so the choice between paradigms comes down to operational cost rather than answer quality.