Multi-Agent LLM System for Time-Series Forecasting via Code Generation and Context Retrieval

Sammanfattning

Time-series forecasting is a task commonly reoccurring in fields such as retail, finance, logistics and energy systems, where predictions of future events based on historical data can be crucial for planning and decision-making. Traditionally, making these forecasts required tedious and repetitive work by an experienced data-scientist, including data exploration, data analysis, model selection, preprocessing, validation and interpretation. However, due to recent advances in artificial intelligence (AI) and agentic systems, automating larger and more time consuming tasks, such as this one, is now possible. This thesis suggests an alternative solution to the manual approach, utilizing multi-agent LLM systems and code generation in order to automate the process of time-series forecasting. The agentic system proposed in this report includes six specialized agents, the Clarifier, Diagnostician, Planner, Coder, Verifier and Reporter, orchestrated by a deterministic workflow. The structure of the agents is intended to imitate the manual procedure of a forecasting task, separating the different parts of the work into agent-suitable sub-tasks. The system was evaluated using a diverse set of synthetic datasets, including multiple different challenges in forecasting difficulties and data-handling aspects, as well as the real-world Walmart dataset. In addition to the forecast accuracy, metrics regarding reliability and reproducibility were computed in order to get a better overview of the system's performance. Additionally, ablation studies analyzing the potential gains and drawbacks of the Verifier agent, as well as utilizing a RAG pipeline in the planning stage, were conducted. The results show that the system can reliably and accurately propose and implement a forecasting pipeline with only a simple initial user prompt stating the objective of the forecast. Adaptability to different signal compositions and data structure is proven to be high and the reliability of the system is robust. The ablation studies furthermore confirm the need for the Verifier agent, and suggest that utilizing RAG-induced planning improves both reproducibility and accuracy. However, due to occasional failed runs and outlier forecasts, the system is better suited as a tool for assisting and streamlining forecasting workflows rather than full automation and human replacement.