Unsupervised Clustering of Core Dumps to Support Debugging in IoT Device Fleets : Semantic Preprocessing and Density-Based Clustering of Firmware Failure Artifacts

Sammanfattning

As IoT device fleets grow in scale, manual inspection of core dumps becomes increasingly impractical, outpacing the engineers who perform it. While core dumps, memory snapshots captured at the time of device failure, are a standard debugging mechanism in embedded systems, no established approach exists for analyzing them at scale from large, unlabeled, real world IoT deployments. This thesis investigates how core dumps from a fleet of deployed hearing aids at Demant A/S can be grouped and analyzed to support failure analysis and defect debugging. A preprocessing pipeline was developed to transform core dumps into a representation suitable for unsupervised machine learning. Key contributions include source-grounded semantic feature annotation, type-aware encoding, and a noise handling strategy that combines firmware sentinel matching with a data-driven cliff detection. UMAP was then applied for dimensionality reduction, followed by HDBSCAN to identify dense failure groups in the resulting embedding. Applied to 41,685 core dumps from a single firmware release, the pipeline produced 73 clusters with a Density Based Clustering Validation (DBCV) score of 0.641. When evaluated against domain-defined crash tags assigned at two levels of granularity, 99.4% of clustered samples shared the same crash type as the majority of their cluster, and 87.5% shared the same detailed crash tag. Preprocessing reduced noise from 30.3% to 7.1% and improved tag-level agreement from 57.1% to 87.5% compared to clustering on non-preprocessed data. Expert evaluation suggested that the pipeline output was interpretable and useful for debugging within the studied setting. Applied to a single firmware release, this work shows that semantically guided preprocessing enables the formation of meaningful clusters, and that unsupervised machine learning can distill tens of thousands of failure artifacts into structured, analyzable groups without the need for labels. Extending this validation across multiple firmware releases remains an important direction for future research.