Is That Polite? How Linguistic Features Shape Large Language Model Judgments in Chinese Politeness Classification

Sammanfattning

Politeness, as an important pragmatic phenomenon, plays a key role in human communication.However, its automated recognition is still challenging in the field of NaturalLanguage Processing (NLP), especially in the Chinese context, which is characterizedby high context-dependency and distinct cultural features. This study takes the ChineseMandarin politeness classification as its research focus, and systematically comparesthe performance of several methodologies, including traditional machine learning,handcrafted feature-based methods, pre-trained language models (BERT), and LargeLanguage Models (LLMs). The research centers on two primary questions: first, theperformance of LLMs in the Mandarin politeness classification task; and second, thespecific linguistic features that drive the models’ politeness judgements.Methodologically, this thesis provides a comparative analysis of classification performanceacross various models. From an interpretability perspective, it employs featureimportance analysis and feature ablation experiments to explore the role of handcraftedfeatures in traditional models. Furthermore, feature grouping analysis and perturbationexperiments are designed to examine the sensitivity of LLMs to different linguistic feature.Experimental results show that BERT model achieves the best overall performance,while LLMs show significant performance gain under few-shot settings. Feature analysisfurther found that LLMs rely more on surface-level formal features, such as exclamationmarks and strong-tone expressions in politeness judgments, while showing limitedunderstanding of pragmatic strategies such as hedging and indirectness. The study suggeststhat current models still have shortcomings in Chinese pragmatic understanding,particularly when handling deep pragmatic strategies. This thesis systematically analyzesthe Chinese Mandarin politeness classification task from the perspective of modelperformance and feature interpretability, providing a new perspective for understandingthe behavioral patterns of LLMs in pragmatic tasks.