Representation Learning for Natural Language and Biological Data : Using contrastive learning to find a joint representation for single-cell RNA data and gene information text data

Sammanfattning

Recent advances in sequencing technologies such as Single-Cell RNA or Spatial transcriptomics have provided scientists with an abundant amount of data. These data contain valuable information on the changes and developments of a cell. However, analyzing this massive amount of data and extracting useful information is not a trivial task. Machine Learning methods and in particular Deep Learning, have shown a spectacular performance in a wide variety of tasks. Inspired by their undeniable success, many researchers have started using these methods to analyze biological data. In this work, we aim to learn a joint representation space for single-cell RNA data and gene text data using contrastive learning. Having access to such representations can facilitate the exploration of a single-cell dataset by enabling the researchers to use text data to retrieve biological samples or zero-shot classification of biological samples. Furthermore, the trained model can act as a backbone for downstream tasks where text information can be utilized to either enhance the performance or bring interpretability. We will show that by using the CLIP loss function, our model has successfully learned a common space, enabling the model to correctly classify cell types and genes and also retrieve marker genes for each cell type with significant recall performance.