Real-Time Rendering for AR/VR with Novel View Synthesis

Sammanfattning

Real-time 3D reconstruction is crucial for immersive AR/VR but is hard to achieve with traditional photogrammetry and mesh-based pipelines due to latency and limited scalability. Neural Radiance Fields (NeRF) enable high-quality novel view synthesis but suffer from long training and slow rendering, while Gaussian Splatting (GS) achieves real-time rendering at the cost of heavy preprocessing and memory usage. This thesis presents a comparative study of NeRF and GS in terms of visual quality, efficiency, and scalability, and proposes an optimized GS-based pipeline for real-time AR/VR. The system introduces three main techniques: (1) clustering-based reduction of COLMAP points to cut preprocessing to under two minutes; (2) gradient-aware clustering of Spherical Harmonic coefficients to shrink storage; and (3) quantization-aware training to reduce precision and memory requirements without harming visual quality. The optimized pipeline lowers total processing time to as little as 5 minutes, reaches up to 180 FPS at 1080p, reduces model size by 6–7×, and maintains competitive fidelity (PSNR > 26 dB, SSIM ≈ 0.80), bringing practical real-time novel view synthesis closer to deployable AR/VR systems.