Benchmarking a machine learning model in the transformation from PyTorch to CoreML

Sammanfattning

Due to rapid development in the field of machine learning and large increases in the capabilitiesof mobile devices, utilizing machine learning on these is becoming increasingly popular. Onemethod of deployment is to develop a machine learning model in well-established deep learningframeworks like PyTorch. However, to be able to run these models on mobile devices, specificframeworks are usually needed. In this thesis, we investigate the performance of a popular objectdetection model, YOLOv5, while being converted from PyTorch to CoreML. This includesmeasuring the performance of the model while running on different hardware. To accomplishthis, we put forward several common benchmarking metrics and compare the different stages.Our results show that CoreML greatly reduces the latency of a machine learning model and hascomparable detection accuracy of within a few percent in the metrics chosen. For iOS deviceswith the ANE-chipset, we also found that ANE (Apple Neural Engine) has significantly fasterlatency compared to running the model on GPU and CPU, while detection accuracy ismaintained. We discuss what could be the root cause for the small loss of accuracy in the modeland a foundation is laid for future work.