An Experimental Study on Model Complexity and Uncertainty Quantification in Transformer Based Time Series Prediction

Sammanfattning

RUL prediction is a key in industrial maintenance with reducing costs and boosting reliability.Transformer models work good in this task where most studies look into accuracy and less into the uncertainty quality. This study investigates how model size and architectural complexity with transformer-based models affect prediction accuracy and uncertainty quantification quality in RUL prediction. Three transformer models with different complexity levels, small, medium and large is evaluated in to alongside a LSTM model as a benchmark on C-MAPSS dataset FD001 and FD004. Two uncertainty quantification methods, Stochastic Weight Averaging-Gaussian (SWAG) and deep ensembles is applied and evaluated using metrics for both prediction accuracy and uncertainty quality. The results show that increasing model size does not lead to consistently improved performance on either dataset. Deep Ensembles outperform SWAG when it comes to calibration and stability, giving coverage closer to the nominal level across different model sizes and datasets. The findings indicate a simpler model might be better in practice and what method to use for uncertainty quantification matters.