Artificial Neural Networks and Inductive Biases for Multi-Instance Multi-Modal Tabular Data : A Case Study for Default Probability Estimation in Small-to-Medium Enterprise Lending

Sammanfattning

The success of artificial neural networks in homogeneous data domains such as images, textual data, and audio and other signals has had considerable impact on Machine Learning and science in general. The domain of heterogeneous tabular data, while arguably much more common, remains much less explored with regards to artificial neural networks and deep learning. Furthermore, work on tabular data tends to focus on the single-instance case. Much tabular data in industry is multi-instance; where single samples are composed of multiple entities (rows, records, etc). Tabular data is also often multi-modal; samples are composed of mixtures of e.g. series, sets, graphs, or vectorial data. This thesis is concerned with designing artificial neural networks to directly operate on these complex data structures, through a case study on a specific risk estimation problem in the financial domain. The task is re-framed as a graph-modelling problem, enabling the use of the flexible Graph Network formalism. Several different artificial neural networks are designed within this formalism, utilizing different inductive biases. Through experiments, it is found that on the available dataset weaker inductive biases result in a stronger model at a high statistical certainty, and even a linear model designed to operate on the complex data structure performs competitively. Some more complex models proposed are super-sets of the best performing model; this is surprising, and suggests that the dataset size is a limitation, and that the specific learning procedure requires further exploration. The space of possible model designs within the formulation of the problem proposed in this thesis is much greater in scope than can be thoroughly investigated in this project, and some suggested next steps for further work are discussed.