Combining automatic speechrecognition with largelanguage models : Designing a secure speech-to-text assistant

Sammanfattning

The rise of artificial intelligence in recent years has led to the development of a variety ofuser-targeted tools incorporating neural networks. These neural networks areimplemented in applications such as chatbots, used for data analytics, and, more recently,speech recognition. The goal of this thesis was to develop a speech-to-text assistant thatcan aid the employees of a company in transcribing meetings or phone calls, possiblytranslating them, or removing the nuances of spoken language for further usage indocuments. To the best of our knowledge, this is one of the first implementations of aSwedish speech-to-text system combining automatic speech recognition and largelanguage models, while also controlling for data leakage due to the application in anenterprise environment. In achieving that aim, a literature review was conducted to beable to select the best possible components for the subsequent design and implementationof a software artifact. Wav2vec2 2.0 VoxRex was selected, a neural network that cantranscribe the Swedish language alongside the large language model Falcon 3-7B, whichis used for the text manipulation. The raw transcription rate lies at 20.2429 words/second.As the company facilitating this work has confidentiality requirements, it was ensuredthat the program does not send telemetry data to third-party actors.