Transformer-Based Real-Time Terrain Recognition from Proprioceptive Signals in Cross-Domain and Heterogeneous Sensor Settings : Edge-Deployed Off-Road Terrain Recognition for Vehicle Navigation

Sammanfattning

This thesis develops a real-time terrain classification system for off-road vehicle navigation that operates on embedded platforms while maintaining robust performance across different vehicle platforms. The system classifies terrain based on proprioceptive inputs such as acceleration, wheel speeds, and suspension travel, identifying four main types: Gravel, Mixgravel, Roughtarmac, and Goodtarmac as well as a separate Lowspeed class. Two essential techniques were developed to handle sensor limitation and vehicle domain variations through a conditional vector quantized variational autoencoder that generates suspension signals from inertial data and a domain adversarial neural network that matches feature distributions between vehicles. The final model reaches 92% accuracy in in-domain testing and improves cross-domain performance by 21% when the two techniques are combined. The system achieves an inference time of approximately 16ms on a Raspberry Pi 3B, demonstrating its suitability for real-time edge deployment. The research demonstrates a functional and expandable method to implement terrain-based autonomous systems in upcoming off-road vehicles despite limited sensor capabilities and processing power. This approach is particularly valuable in safety-critical autonomous or assisted driving systems where visual sensors can fail under adverse conditions.