An FPGA-based On-device Fine-tuning Accelerator for Edge CNN Adaptation

Sammanfattning

With the widespread deployment of convolutional neural networks (CNNs) at the edge, on-device fine-tuning has become essential for real-time model adaptation to address environmental accuracy degradation. However, on-device training is constrained by the "memory wall" energy bottleneck and numerical instability from low-precision fixed-point arithmetic. To tackle these challenges, this thesis presents an energy-efficient, FPGA-based on-device fine-tuning accelerator featuring a deep hardware-algorithm co-design. At the algorithmic level, a partial fine-tuning strategy is adopted to freeze shallow feature extractors while only updating deeper task-specific layers, reducing the activation-storage overhead of full backpropagation. Additionally, a hardware- aware quantization-aware training (HA-QAT) pipeline incorporates LFSR-based Stochastic Rounding and Stochastic Approximate Pruning (SAP), achieving over 70% backward gradient sparsity with negligible accuracy loss. At the architec- tural level, forward and backward computations are modeled as a unified seven- dimensional nested loop mapped onto a custom 2D Processing Element (PE) array. By employing a Row-Stationary (RS) dataflow and adaptive spatial unrolling, the design maximizes data reuse and minimizes memory access. The system is prototyped and verified on the Xilinx Zynq UltraScale+ ZCU104 platform. Evaluations using a lightweight VGG-style network on a perturbed CIFAR-10 dataset show that the accelerator recovers model accuracy from a de- graded 69.02% to 84.85% with stochastic pruning enabled (a mere 0.30% drop versus the unpruned baseline). Operating at 166.67 MHz, it delivers an effective throughput of 21.93 GOPS with a power consumption of only 2.085 W, yielding an energy efficiency of 10.52 GOPS/W. This work demonstrates the feasibility of practical CNN on-device fine-tuning on resource-constrained edge FPGAs, provid- ing an efficient hardware-software co-design solution for local continuous learning in edge intelligence.