Transformer Based Next Generation Wireless Communication Receiver

Sammanfattning

The Integration of Machine Learning in OFDM Receivers The integration of Machine Learning (ML) into various fields has prompted exploration into the effectiveness of advanced ML architectures within Orthogonal Frequency Division Multiplexing (OFDM) receivers. Prior work by Nvidia demonstrated the promise of Graph Neural Networks (GNNs) in replacing key components of a traditional receiver, surpassing conventional methods such as Least Squares (LS) channel estimation and Linear Minimum Mean Square Error (LMMSE) combined with K-Best detection. This thesis extends that line of inquiry by investigating the transformer architecture as an alternative base for an AI-based OFDM receiver. Various transformer configurations are designed and optimized, and their performance is compared against classical baselines including LMMSE + Maximum Likelihood Detection (MLD), perfect Channel State Information (CSI) + MLD, and the Nvidia GNN model. The study explores the following main topics: Comparison of transformer, GNN, and baselines. Results show that the transformer achieves a gain of up to 0.3 dB at 10% BLER compared to the GNN, and a gain of approximately 1 dB at 10% BLER compared to the LMMSE + MLD baseline. The effect of different channel estimation strategies as initial input to the system, where tested methods include LS estimation, LMMSE estimation, and a separately trained transformer model solely for channel estimation. Results show no major performance difference in BLER between methods, allowing the simplest strategy, LS channel estimation, to be used without loss of performance. The effect of Positional Encoding (PE) as input to the system. Results show a slight gain for certain SNR values, e.g., 0.4 dB at 10% BLER for both transformer and GNN. The impact of varied and fixed Signal-To-Noise Ratio (SNR) during training. Results show similar performance for the specified SNR, but a significant loss for lower SNR values, reaching upwards of 2 dB at 40% BLER. The effect of different channel evaluation methods. Comparisons of Mean Square Error (MSE) loss, Squared Generalized Cosine Similarity (SGCS) loss, and no channel estimation loss showed no major difference in BLER performance. Channel estimation loss was still retained in models, however, due to stability advantages presented by Nvidia. The impact of different Demodulation Reference Signal (DMRS) patterns. Results show that different DMRS patterns can impact the BLER performance of the models. A DMRS pattern with high temporal density gave a larger separation of the GNN and transformer models’ BLER performance than a DMRS pattern with higher frequency density, showcasing how the different models can leverage the DMRS pattern in different ways. The potential of Iterative Channel Estimation Detection (ICED). Results show that ICED can be used to improve channel estimation, but improvement in BLER performance was not observed. The effect of power boosting of DMRS symbols. Results show that power boosting of DMRS symbols can improve the Uncoded Bit Error Rate (UcBER) of the transformer to the point where it approaches perfect CSI + MLD. The same was, however, not observed for BLER or Channel Estimation error (CE error), where it reached saturation. The generalization capability of the model across different channel types. Results show that the transformer generalizes well to channels of similar characteristics but can incur significant loss when channels differ greatly. This is especially evident for models trained on a channel with low Doppler and frequency selectivity, such as EPA5, and tested on a channel with high Doppler and frequency selectivity, such as ETU100. In this case, the model saw a loss of up to 3 dB at 20% BLER. Experimental results show that the proposed transformer-based receivers outperform both the LMMSE + MLD baseline and the NVIDIA GNN in the SISO case, demonstrating the potential of attention-based models in the future of wireless signal processing.