Investigating Energy Consumption and Performance Trade-offs in Large Language Model Inference Using Quantization and Pruning

Sammanfattning

large language models (LLMs) are increasingly adopted across industries due to their strong performance on a variety of natural language processing (NLP) tasks. However, these models require significant computational resources during inference, resulting in high energy consumption and increased environmental Impact. As the demand for sustainable AI grows, optimizing the runtime efficiency of LLMs without compromising performance has become a key concern. In this work, we investigate the Impact of optimization techniques, such as Quantization and Pruning, on the energy efficiency and performance of LLMs during inference. We evaluate four models: GPT-2 1.5B, DeepSeek- R1 1.5B, DeepSeek-R1 7B, and Mistral 7B, across three NLP tasks: text completion (LAMBADA), sentiment classification, and binary question answering (BoolQ). We compare performance across four precision formats (FP32, FP16, INT8, and INT4) and apply structured Pruning to the 7B models. Our results show that FP16 achieves more than four times the energy saving of full-precision without compromising accuracy. INT8 and INT4 experienced small performance drops, though implementation limitations likely prevented full utilization of their efficiency potential. Pruning improved efficiency in Mistral 7B but degraded DeepSeek 7B, due to architectural differences. We also performed statistical tests to verify the significance of observed differences in accuracy and energy consumption. Our findings showcased how effective model optimization strategies can significantly reduce energy consumption without compromising performance, and suggest future work to further explore additional methods like fine-tuning or prompt design for further gains.