Towards a Comprehensive Bias Mitigation Framework in Machine Learning for Group Fairness and Accuracy-Fairness Trade-Off

Sammanfattning

Bias mitigation in machine learning models poses a real challenge, as biases often remain undetected, thereby reinforcing existing societal inequalities within systems and sometimes exacerbating structural issues in society. As machine learning systems are increasingly being used in critical decision-making, it is of high importance that these systems are effective and not compromised due to replication or amplification of preexisting biases from historical data or the internal logic of the models, leading to undesirable results. Bias mitigation frameworks have been developed previously, however, mostly in regards to specific domains, fields, or machine learning models. In this Design Science Research, we aim to contribute a step toward a comprehensive bias mitigation in the form of a framework. Consideration has been made for the user to abide by regulatory or legal requirements for appropriate metrics; although not completely agnostic towards the machine learning model, data type, and field of application, this framework provides both default and adjustable values with the goal of creating room for customizability. The proposed approach includes five phases of bias mitigation: Bias Detection, Pre-Processing, In-Processing, Post-Processing, and Maintenance. The framework aims to encompass the bias mitigation life-cycle of a machine learning system by including these five steps. Unique for our framework is the consideration of both individual fairness and group fairness at different decision points, while attempting to maintain acceptable levels for the accuracy-fairness trade-off; research shows that improvement in fairness often leads to a decrease in predictive accuracy. The framework is presented as a research-backed interactive decision tree, guiding the user through each step, with the goal of supporting a more fair and accurate machine learning system at the end of the process. Because this study is solely a step towards comprehensiveness, the user is encouraged to apply the framework in a responsible manner, through application of best practices documented in previous research, supporting responsible development and maintenance practices. The evaluation process is theoretical and descriptive; the framework has yet to be empirically validated across different scenarios, leaving this as a step for future work.