Using Large Language Models to Solve Spatial Problems : Assessing ChatGPT’s Ability to Understand Spatial Queries and Geometric Topological Problems

Sammanfattning

Since the introduction of Open AI’s Generative Pre-Trained Transformer (GPT) in 2022,research on the field of generative artificial intelligence (AI), and its applications has beenintense. Large Language Models (LLMs), with ChatGPT in the lead, have been subject to scrutinyto find their weaknesses, with researchers pinpointing their struggle with mathematical andgeometrical operations, and understanding of complex questions. This study further testsChatGPT version 3.5 on these fields, by letting it answer questions normally solved usingGeographic Information Systems (GIS). Previous studies have been made, combining the fieldsof LLMs and GIS, mostly with a focus on more advanced aspects of GIS, such as formulatingprompts in Structured Query Language (SQL). In contrast, the purpose of this study was to findwhether laymen, without much GIS knowledge, could use the free version of ChatGPT to solvespatial problems, and to assess GPT’s understanding of topology, a common data structurewithin GIS. ChatGPT proved to mostly give inconsistent and unreliable answers, even for thesimplest tasks, having problems with calculating distances and population and visualizingtopology. Despite this, it exhibited an understanding of GIS concepts and displayed use of logicalthinking when presenting its answers, proving it had potential to reliably solve spatial problemsif it can be trained on better-suited datasets.