LLM-based engineering for networked systems

Sammanfattning

Recent advances in Artificial Intelligence (AI) and Large Language Models (LLMs) have demonstrated the potential for LLM-based engineering. Until now, the work in the area has been focused almost exclusively on software engineering. This thesis aims to broaden the field to include other aspects of engineering for networked systems. The objective is to investigate LLMs, specifically ChatGPT, as tools for engineering tasks within the domain, to evaluate the efficiency of existing prompting strategies, and to develop taskspecific reusable prompt patterns. The evaluation is performed in three iterations and in a final evaluation where different prompting strategies are tested and new prompt patterns are developed. The prompting strategies that are tested are Zero-shot, Chain-of-Thought and the Persona strategies, as well as ChatGPT’s functionality to include PDFs in a prompt and to use web search. The test data is collected from three KTH course projects which cover a variety of engineering tasks in the field. The tasks are categorised by output type into Analytical, Descriptive, Operational, and Visual tasks. The Analytical, Operational, and Visual categories are evaluated in the thesis. The results show that ChatGPT is highly effective for Analytical and Visual tasks, while the performance on Operational tasks rely on the provided details and the preciseness of the prompting. The developed prompt patterns significantly improve the performance for all investigated categories compared to the simple prompting strategies. The findings of the thesis suggest that LLMs can serve as powerful support tools for engineering networked systems if used correctly.