PRIX: Plan from Raw Pixels : A vision-only End-to-End Autonomous Driving Architecture

Sammanfattning

End-to-end autonomous driving models show promise but face deployment challenges due to large model sizes, reliance on costly LiDAR, and heavy BEV feature representations. This thesis project proposes PRIX (Plan from Raw Pixels), an efficient vision-only driving architecture that directly predicts trajectories from raw pixels, removing the need for LiDAR and explicit BEV features. PRIX introduces the Context-aware Recalibration Transformer (CaRT) to enhance multi-level visual features via self-attention and employs a conditional diffusion planner that refines trajectories through iterative denoising. Experiments on NavSim-v1 and NavSim-v2, which are large-scale simulation datasets for evaluating autonomous driving performance, demonstrate state-of-the-art performance, with PRIX achieving 57 FPS and a PDMS score of 87.8, surpassing methods relying on sensor fusion. Ablation studies further validate design choices in CaRT and diffusion steps. These results establish PRIX as a practical, vision-only solution for autonomous driving. By eliminating reliance on expensive sensors and complex BEV representations, this thesis makes driving technology more accessible and scalable. Future work includes extending PRIX to real-world datasets and diverse weather conditions to further improve robustness.