An Empirical Analysis of Heterogeneous Execution for YOLO-Based Object Detection in Continuous Mobile Inference

Sammanfattning

Deep-learning inference is increasingly deployed directly on smartphones for computer-vision tasks such as object detection. This can reduce latency, limit dependence on cloud services, and keep visual data on the device. However, running these models continuously on a phone is difficult. Smartphones have limited power and cooling capacity, so performance that appears strong at the beginning may decrease as the device heats up and thermal throttling starts. This issue is relevant to the proof-of-concept deployment context of Supersight Oy, where Android smartphones are considered for long-running YOLO-based analytics. This thesis studies sustained YOLO-based object detection on Android smartphones, with a focus on heterogeneous CPU--GPU execution. The main question is whether splitting inference across the CPU and GPU remains useful during long-running operation, once thermal effects, cut-point feasibility, data-transfer overhead, and runtime limitations are taken into account. To answer this, the study compares heterogeneous execution with CPU-only and GPU-only baselines under controlled live-camera workloads. The evaluation follows an empirical systems approach. Controlled long-run experiments were conducted on representative Android smartphones using sequential and heterogeneous execution configurations. The study also examines camera-side workload pacing through frame-rate control, and re-evaluates selected configurations under capped input rates. The analysis focuses mainly on sustained throughput and thermal behaviour, while timing and profiling signals are used to interpret selected execution bottlenecks. The results show that simple heterogeneous CPU--GPU execution does not provide a universal sustained advantage over strong sequential baselines. In several cases, it remains only approximately competitive. In selected device--model--cut combinations, it provides a modest benefit. The results also show that camera-side input-rate control can make selected heterogeneous configurations more competitive by reducing uncontrolled workload pressure. Overall, the findings provide practical guidance for evaluating long-running mobile object-detection systems within the studied NCNN-based Android pipeline, and suggest broader considerations for deployment under real device constraints.