Bridging the Lab-to-Industry External Validity Gap in Deep Learning-Based Fault Diagnosis: A Comparative Evaluation of Multi-Source Multi-Target Domain Adaptation on Cross-Machine Domain Shifts in Real Industrial CNC Machine Data

Sammanfattning

Introduction: Machine fault diagnosis is critical to operational reliability and economic sustainability. Undetected faults cost the world’s 500 leading firms an estimated $1.4 trillion in 2024. Deep learning (DL) has become the state-of-the-art approach to data-driven fault diagnosis, but its performance depends on training and test data sharing the same probability distribution. This assumption is frequently violated in real industrial settings, so domain adaptation (DA) has emerged as an effective solution. However, existing DA research relies almost exclusively on controlled laboratory datasets, and the most practically relevant scope, which is multi-source multi-target DA, remains severely underexplored. Research Question: The primary research question is: "To what extent does multi-source multi-target domain adaptation improve deep learning-based bearing fault diagnosis on real industrial CNC machine data compared to no adaptation?" Method: A quantitative comparative framework was designed to evaluate the 3 existing multi-source multi-target DA methods specialised for fault diagnosis known as IDANN, TTMN, and WJMMD-MDA. The proprietary dataset used to test these methods consists of 2,672 spectrograms derived via Short-Time Fourier Transform from the vibration signals of 9 real CNC machines operating in factory environments. Performance was assessed across 3 cross-machine diagnostic tasks. A frozen ViT-B/16 transformer was evaluated on the same dataset as an upper-bound reference for source-only learning. Results: All 3 DA methods achieved only marginal accuracy improvements over their non-adapted baselines, and none outperformed the frozen ViT-B/16 baseline, which achieved the highest mean accuracy of 57.3%. TTMN and WJMMD-MDA failed to exceed random-chance performance at their best configurations. All DA methods exhibited high run-to-run variance, frequent class collapse, and an inability to translate separable feature representations into accurate classification due to miscalibrated decision boundaries caused by the unresolved domain shift. Discussion: These findings indicate that state-of-the-art multi-source multi-target DA methods for fault diagnosis may not fully generalise to real industrial cross-machine conditions characterised by both statistical and structural domain shifts. Consequently, this study highlights that the external validity of existing DA-based methods, which achieve near-perfect accuracy on controlled benchmarks, warrants further critical evaluation in applied settings.