Generative Handwritten Text Separation and Recognition for Palimpsests Modeling

Sammanfattning

This project aims to uncover obscured texts hidden within damaged palimpsests. Generative methods for palimpsest text separation using deep neural networks are investigated together with handwritten text recognition. The Generative Adversarial Network (GAN) Pix2pix as well as a custom Denoising Diffusion Model (cDDM) modified to learn structured noise, is trained to identify partially hidden under-texts and generate restorations of them. The output from the generative networks are recognized and transcribed with AttentionHTR, a pre-trained network for handwritten text recognition which is fine-tuned for each of the generated datasets. The experimental setup consists of multiple datasets, two synthetically created in this work, where MNIST and parts of the historical handwritten texts from Saint Galls dataset are extracted and used as undertexts. Additionally, a third public synthetic dataset created to closely simulate the multi-spectral parts of the real Georgian palimpsests is included. Pix2pix and cDDM both perform well for the task of text separation, where different strengths become apparent in relation to the datasets. Additionally, the impact multi-spectral imaging has for revealing previously occluded or unseen texts is highlighted with three samples of multi-spectral data captured with the first Multi-spectral Imaging System for Historical Artifacts (MISHA) ever built in Sweden. The ultimate goal of this work is to render the once barely legible remnants of our medieval past accessible once again.