Privacy Preserving Knowledge Distillation Architecture for Credit Scoring : A study on machine learning for automating bank loan approvals

Sammanfattning

Financial institutions operating under the European Union's General Data Protection Regulation face a fundamental tension in automated credit scoring: black-box machine learning models offer superior predictive accuracy but cannot satisfy the regulation's right to explanation, while interpretable white-box models lack the predictive capacity required for reliable automated decisions. This tension represents an unresolved research gap, as no established architecture simultaneously satisfies accuracy, data privacy, and GDPR compliance in credit scoring contexts. This study designs and evaluates a Privacy Preserving Knowledge Distillation Architecture in which an XGBoost teacher model transfers predictive knowledge to a single decision tree student model through Response Based Knowledge Distillation on soft probability targets. The architecture is evaluated using the Home Credit Default Risk dataset, with predictive performance measured through AUC-ROC and F1-score, and semantic alignment between teacher and student reasoning measured through Rank-Biased Overlap. The Student Model retained approximately 95.8% of the Teacher Model's AUC and 92.9% of its F1-score, while achieving a 9.5% relative improvement in semantic alignment compared to a control model trained on hard labels. The resulting model size of 44.8 KB satisfies the defined on-device deployable threshold of under 1 MB. These results demonstrate that the tension between predictive performance and regulatory interpretability requirements can be partially reconciled through knowledge distillation.