Adaptive Shields : An investigation of Long-Short Term Memory applications against Advanced Persistent Threats

Sammanfattning

The ever-evolving nature of cyber-adversaries pose serious threats to society at large. The rise of a new powerful adversary, the Advanced Persistent Threat, further emphasizes the critical need for robust cyber-defenses. This thesis explores the use of Reinforcement Learning (RL) for the purpose of training an autonomous defender against multi-stage attacks. The work focuses on developing an RL-environment featuring a network digital-twin using Graphical Network Simulator 3 (GNS3) and Docker containers, and an RL-agent tasked with defending the digital-twin from a multi-stage attack. The attack inspired by tactics used by known adversary Advanced Persistent Threat (APT) 29 is simulated, rooted in frameworks such as MITRE ATT&CK and the Cyber Kill chain. Defender actions are mapped to real-world techniques using the MITRE D3FEND framework. The thesis investigates whether recurrent policies using Long Short-Term Memory neural networks offer tangible advantages in strategic decisionmaking under partial observability. A comparison between Proximal Policy Optimization (PPO) and Recurrent-PPO (RPPO) is performed to answer the research questions. Experimental results show that due to the time-intensive nature of high-fidelity network digital-twins, the sample efficiency of regular PPO outperforms its recurrent counterpart. The study concludes with a discussion on trade-offs between realism and efficiency, along with proposal for future works involving stochastic attackers, multi-agent frameworks, and further exploration of attack types for a more general and nuanced autonomous defender.