Synthetic Data Generation for Object Detection with NeRF and Gaussian Splatting : Enhancing Synthetic Drone Detection with Neural and Point-Based Rendering Techniques in Unreal Engine 5

Sammanfattning

The advancement of computer vision and object detection relies heavily on high-quality training data. However, acquiring and annotating real-world datasets is often expensive, time-consuming, and constrained by privacy concerns. Synthetic data generation offers a scalable and flexible solution for training AI models, reducing reliance on manually annotated real-world datasets. This thesis explores the use of Neural Radiance Fields (NeRFs) and Gaussian Splatting for generating synthetic datasets, focusing on their application in drone detection within Unreal Engine 5. To investigate the effectiveness of these neural rendering techniques, a dataset was created using a combination of Gaussian Splatting for realistic environment reconstruction and NeRF-generated 3D objects. The synthetic data was integrated into a simulation framework in Unreal Engine 5 and used to train object detection models based on YOLOv8. The models were tested against both synthetic validation sets and real-world datasets to evaluate generalizability. Fine-tuning synthetic datasets with NeRF-generated objects led to measurable improvements in drone detection accuracy, especially in scenarios with complex occlusions and varied lighting conditions. While baseline models trained exclusively on synthetic CAD models showed moderate accuracy, incorporating NeRF-generated data led to higher detection precision and better domain adaptation to real-world datasets. However, computational costs and the complexity of rendering high-quality NeRFs remain challenges for large-scale dataset generation. This study contributes to the field of synthetic data for object detection by demonstrating the advantages and limitations of NeRFs and Gaussian Splatting for training computer vision models. Future work should focus on optimizing data generation efficiency and expanding the dataset diversity to improve real-world applicability further.