Congesting Distributed AI Within the Host Network

Sammanfattning

As artificial intelligence models scale, they rely on distributed, multi-tenant hardware.In these environments, the internal host network of e.g. CPUs, GPUs, memorycontrollers and inter-socket links is critical to transferring data efficiently. Theinternal host network is often viewed as a trusted environment, but congestion ofthe shared paths can have substantial effect on performance.Consequently, this thesis addresses the following research question: Can a co-locatedadversary intentionally utilize host network congestion to execute an attack againstdistributed AI workloads?To evaluate this potential threat, we developed adversarial workloads mainly targetingthe memory and the inter-socket link. These attacks were executed concurrently withTransformer and Graph Neural Network (GNN) models while capturing performanceand hardware statistics.The results show that an adversary can weaponize the host network and causesubstantial end-to-end performance degradation for the AI models. The magnitudeof the performance degradation depends on how the models use the hardware. Modelsthat continuously rely on the CPU memory for data suffered the largest performancedrops, when an attacker saturated CPU memory. In contrast, the model that keptit’s data in the GPUs’ VRAM and synchronized peer-to-peer was immune to thiskind of attack. It was however vulnerable to an attack against the inter-socket path.Essentially, this thesis demonstrates a vulnerability in multi-tenant high-performancecomputing. An adversary does not need to break virtual machine boundaries or haveelevated privileges to cause harm. Exploiting the host network can cause damage interms of performance degradation. Expensive GPUs being underutilized translatesto a substantial indirect financial cost.