Efficient Pipeline Design for Real-Time RNN-based Anomaly Detection in Embedded Edge AI Systems

Sammanfattning

This master's thesis explores and evaluates how to design efficient edge AI pipelines for anomaly detection. The deep learning architecture used was a recurrent neural network (RNN) autoencoder, capable of reconstructing and detecting abnormal data points, suitable for resource-constrained embedded systems. It investigates how such models can be executed on the ESP32-P4 MCU using Zig as the primary programming language. The main goals are to evaluate model accuracy, latency (throughput), and memory footprint for real-time use cases. The proposed solution uses a compact Recurrent Neural Network autoencoder trained on time-series data from the SMAP and MSL datasets to detect anomalies in univariate signals. The system performs inference without any dynamic memory allocation, using compile-time Matrix and Layer types built from scratch, and compiles them into the same trained model used in PyTorch. Benchmarking on the ESP32-P4 showed a latency way below 2 ms per window, with a peak throughput (latency) of approximately 786 predictions per second for the final optimized build. The model generalization achieved an F1-score of 0.6075 and an AUROC of 0.667 on the E-3 channel (dataset), while maintaining a total memory footprint of ≈3.875 KB and an average energy consumption of 0.772 mJ per inference. The result demonstrates that deep learning models can run deterministically on low-powered hardware using Zig's safety-aware programming model. Moreover, this thesis contributes to a reproducible toolchain, a Zig runtime for RNN inference, and a benchmark methodology that links accuracy, latency, and memory usage in edge AI embedded anomaly detection.