Interconnecting Multiple FPGAs for Distributed Machine Learning Inference

Sammanfattning

Machine learning has been growing in popularity for many years now and is currently popular among both businesses and private individuals. Running machine learning models takes a lot of computational resources, which means a large amount of energy is consumed. The aim of this thesis is to investigate the usage of machine learning model specific hardware on Field Programmable Gate Arrays (FPGAs) as a potential solution to low-latency machine learning inference. However, as FPGAs have limited resources, not all models can fit on one FPGA. Therefore, the model has to be partitioned to be run over two or more FPGAs to distribute the need for resources. It was found that partitioning the machine learning model over multiple FPGAs introduces a few challenges, such as an unbalanced resource utilization over the FPGAs and a possible bottleneck of inter-FPGA communication when the goal is high throughput. The main difficulty when designing the hardware accelerator is how to store all the millions of parameters. On the other hand, the introduced latency of the inter-FPGA communication is negligible, which means focus can be placed on reducing the computational latency. Top-1 inference accuracy is also close to the fully floating-point model, while top-5 accuracy is slightly higher when implementing the convolutional neural network in hardware.