Presenting ENBYS The Europarl Non-BinarY Sentences: an English-Polish-French Dataset for Machine Translation Evaluation of Non-Binary Gender Inclusion

Sammanfattning

Investigations of gender bias within natural language processing (NLP), and more specificallymachine translation (MT), tend to adopt a binary view of gender, upholding amasculine/feminine dichotomy. The present thesis aims to widen that research area byshedding light on how machine translation handles non-binary genders and the nonstandardlinguistic forms that express them.We focus here on singular forms in three languages that each have a different grammaticalsystem regarding gender: English, in which grammatical gender is almost nonexistent;French, which has a binary masculine/feminine grammatical gender; and Polish,which, in the singular, has three main grammatical genders: masculine (with furthersubdivisions), feminine and neuter. In all three languages, non-binary communities havedeveloped different strategies to express themselves in ways that reflect their identities,through the use of existing forms (such as using third-person plural pronouns, neuterpronouns, neuter agreements), the practice of splitting (including both masculine andfeminine word endings separated by typographical symbols), or the creation of completelynew words forms.We thus present the Europarl Non-BinarY Sentences (ENBYS), an English-Polish-French dataset of parallel sentences varying on the gender of a main actor within thesentence, on which we evaluate three widely used machine translation tools (DeepL Translator,Google Translate and Microsoft Translator) on their ability to recognize and generatecorrectly gendered forms between each language pair (EN ↔ FR, EN ↔ PL,FR ↔ PL).We find that while the translation models transfer global semantic information adequately,they struggle to generate non-binary forms when expected, and instead revertto binary forms.