Cost-Aware Learning for Failure Prediction in Industrial Predictive Maintenance

Sammanfattning

Introduction: Industrial predictive maintenance systems use sensor data to detect equipment failures before they occur. This reduces unplanned downtime and reduces repair costs. Building effective predictive models for this task is difficult because two practical problems almost always arise together. First, failure events are rare compared to normal operation. A model can appear to perform well simply by predicting that nothing will ever fail, while missing all real failures. Second, the cost of a missed failure is much higher than the cost of a false alarm. Missed failure can lead to emergency repairs, safety risks, and production losses. A false alarm only causes an unnecessary inspection. Standard machine learning training methods do not account for this difference in cost. They treat all prediction errors as equally important. This creates a gap between how a model is trained and what actually matters in industrial practice. Research Question: This thesis asks whether incorporating the real cost difference between missed failures and false alarms directly into the model training process leads to a lower total maintenance cost in practice. Three specific questions guide the investigation. Research question 1(RQ1) is whether cost-aware training reduces the total maintenance cost compared to standard training. Research question 2(RQ2) is how changing the weight assigned to missed failures in the training process affects the balance between missing failures and raising false alarms. Research question 3(RQ3) is how sensitive the results are to the choice of this weight. Method: The study uses a real-world industrial dataset that contains sensor measurements collected from heavy-duty trucks, provided by SCANIA. The data set includes 23,550 training vehicles with a failure rate of 9.6%. The validation and test sets each contain approximately 5,000 vehicles, of which 136 are failure vehicles. The sensor data consists of 14 anonymised variables. Performance is evaluated using a five-class cost matrix that reflects the industrial cost of each type of prediction error. Missed failures cost between 200 and 500 units, depending on how close the vehicle is to the actual failure. False alarms cost between 7 and 10 units. Four experiments are conducted. All four use the same gradient boosting model architecture, which is an ensemble method that builds decision trees sequentially, each correcting the errors of the previous one. Each experiment focuses only on the training objective, the loss function, and changes between experiments. The first experiment uses standard binary cross-entropy loss as the baseline. The second uses focal loss, which is a modified cross-entropy loss function that reduces the weight assigned to easily classified examples and concentrates training on difficult or misclassified ones. The third and fourth experiments use the proposed cost-aware focal loss(CAFL), which derives the class weight directly from the cost matrix. The weight is calculated as the mean missed-failure cost divided by the sum of the mean missed-failure cost and the mean false-alarm cost, giving a value of approximately 0.976. In the third experiment, the cost matrix is used as published. In the fourth experiment, the missed-failure costs are halved before computing the weight. For all experiments, hyperparameters are selected by minimizing the total validation cost across 100 automated trials. Decision thresholds that map the model’s output probability to one of five prediction classes are also optimized in the validation set. Results: The baseline experiment achieves a test cost of 46,393 and correctly identifies 79.6% of failure vehicles. Focal loss produces almost identical results, cost 46,345, failure detection rate 81.0%, confirming that focal loss does not provide practical benefit over the standard approach on this data set. The CAFL with the original cost matrix achieves a higher failure detection rate of 100%, meaning that every failure vehicle is correctly identified. However, total cost increases to 49,099 - 5.8% higher than the baseline. This happens because the optimized threshold is very low, which causes almost all healthy vehicles to be incorrectly flagged as failures. The combined false-alarm cost of more than 4,700 healthy vehicles exceeds the savings from detecting every failure. The CAFL with halved missed-failure costs achieves the lowest measured total cost of 28,050, which is a 39.5% reduction from the baseline. however, this result is not operationally useful because the model detects no failures at all. It predicts every vehicle as healthy. The cost appears lower only because the evaluation matrix assigns reduced penalties to missed failures in this configuration, not because failure detection has improved. A sensitivity analysis varies a scaling parameter for false-negative costs (fn_weight) across seven values. All values below the cost-matrix-derived setting of fn_weight = 1.0 produce random-chance discriminative ability and zero failure detection. Values above this setting produce no meaningful cost improvement. The setting derived directly from the published cost matrix produces the strongest discriminative ability in the entire study, with a ranking ability score of 0.710 on the test set. Discussion: The results show that embedding industrial cost information in the training objective changes the behavior of the model in measurable ways. However, in this study, a genuine cost reduction achieved through improved failure detection is not demonstrated. The core issue is not the design of the loss function, but the way the threshold optimizer searches for decision boundaries. Without a minimum failure detection constraint, the optimizer finds solutions that minimize the cost metric by flagging everyone as a failure or flagging no one as a failure. Neither solution is operationally acceptable. This study contributes to a CAFL formulation in which the class weight is derived analytically and directly from the industrial cost matrix. It also identifies the specific weight value below which the model degenerates into random-chance performance, and provides empirical evidence that a minimum failure detection constraint is necessary for cost metric optimization to produce operationally valid results. The main limitation of this study is the absence of such a constraint during threshold optimization. A second limitation is the low discriminative ceiling imposed by the 14 anonymised sensor variables, which removes the most informative signals from the raw data. The test set contains only 136 failure vehicles, which introduces uncertainty in the cost and classification metric estimates. Future work has two main directions. The first direction is to add a minimum failure detection requirement to the threshold search. For example, the model could be required to detect at least 50% of failures. This would prevent the model from taking the shortcut of predicting everyone to be healthy. It would also allow the optimizer to find a more useful balance between missed failures and false alarms. The second direction is to use model architectures that process raw sensor data directly over time. Currently, the sensor readings are summarized in a single row per vehicle before training. Removing this step may help the model learn better patterns from the data. This could improve failure detection ability and make cost-aware training more effective.