Neural network approaches to research grant proposal classification : A comparative study of RNN, CNN, and RoBERTa with environmental impact analysis

Sammanfattning

Automated classification of research grant proposals into review panels is a task with tangible benefits for research funding bodies. This thesis trains and evaluates three neural network architectures, an attention-based long short-term memory (LSTM) recurrent neural network (RNN), a convolutional neural network (CNN), and a robustly optimised BERT approach (RoBERTa). The thesis endeavoured for a balance between a performance-oriented and sustainability-oriented perspective. The RoBERTa model outperformed both the CNN and RNN on all the datasets, but the difference was significant only on 3 out of 4 datasets for the CNN, and 1 out of 4 datasets for the RNN. The RNN only significantly outperformed the CNN model on 1 out of 4 datasets. In terms of sustainability; the CNN produced the lowest estimated carbon emissions, compared to both RoBERTa and RNN. However, these estimates do not account for RoBERTa’s pre-training cost. Were this cost included, the environmental gap between RoBERTa and the simpler architectures would widen considerably in their favour. This thesis thus demonstrates that simpler models constitute a viable alternative to large pre-trained language models for this task, whilst carrying a markedly lower total environmental footprint.