AI Agent-based Job Matching : A comparative study of agent-based vs traditional andsemantic models for CV-Job Matching

Sammanfattning

In this age of digital transformation and a rapidly evolving landscape, the process of matching candidates to job roles has changed significantly as digitalization has impacted all domains across all industries. From the traditional, manual resume-job posting matching to the emergence of multiple systems, to the use of automation and artificial intelligence (AI) in the recruitment. As the use of AI is accelerating, the importance of knowing the right model to use for performance and tackling concerns like fairness is becoming extremely crucial. This research study focuses on analyzing how OpenAI agent-based systems, particularly, OpenAI GPT-4.1 mini, perform as compared to the traditional and semantic models, specifically TF-IDF and SBERT, for the task of CV-job matching based on the limited dataset for this study. Both the resume and job datasets, sourced from Kaggle, were released under the CC0 1.0 Universal (Public Domain Dedication) license. Another aspect considered in this research is fairness across the abovementioned models, and to do that, the attribute available in the dataset that allowed us to conduct that analysis was seniority. To perform the research for this study, a literature review was conducted to understand the related work done across multiple scientific databases. The related work was studied and used across all the topics regarding recruitment, the usage of traditional and semantic models, the current landscape of usage of agent-based models, and the fairness lens in job matching. The metrics were chosen based on performance metrics, ranking quality metrics, and fairness metrics for observing seniority bias, across all three models. For implementation, a thorough exploratory analysis was done to understand the dataset, followed by data preparation and model input preparation. Given the scope and timeframe of the study, ground truth was formed with the help of the OpenAI GPT-5.5 model. In addition, human validation and cross AI-model validation was done to validate the ground truth accuracy. From the performance standpoint, it was seen that the OpenAI agent performed best on all core and ranking metrics; from the fairness standpoint, it was seen that while the OpenAI-based agents did have the lowest accuracy parity gap, it did not fully outperform the traditional and semantic baselines in TPR for the different seniority buckets. However, the OpenAI agent achieved a more balanced result across the different buckets than either TF-IDF or SBERT individually under the adopted methodology. The results were followed by discussions related to the findings, including the interpretations, surprises, and implications found throughout the research process. The study also concludes the answers for both research questions and discusses related future work that can be taken up further to enhance research in this domain within academia.