An Efficiency Comparision of NPU, CPU, and GPU When Exceuting an Object Detection Model YOLOv5

Sammanfattning

In recent years, rapid advancements in machine learning and artificial intelligence have spurred the development of specialized hardware designed to optimize performance and efficiency. Among these, Neural Processing Units (NPUs) have emerged as a promising technology for accelerating neural network tasks. This study compares the efficiency of an NPU with a Central Processing Unit (CPU) and Graphical Processing Unit (GPU) in executing an object detection model, YOLOv5. The comparison focuses on inference time, power, and energy consumption. Pre-trained YOLOv5 models of various sizes, precisions, and formats are used to fairly showcase the capabilities of each hardware platform. The YOLOv5 models used in this study come in nano, small, medium, and large sizes. The models on the CPU and GPU are available in single and low precisions, in both PyTorch and ONNX formats. The models on the NPU are in half and low precisions, each in RKNN format. The results highlight the NPU’s superiority in power and energy efficiency, though it is slower than the PyTorch models on the GPU in terms of inference time. PyTorch models on the GPU are the fastest and the second most energy-efficient, while being the most power-intensive. The CPU serves as a middle-ground option in terms of inference time and power consumption, while being the most energy-intensive.