Data Augmentation with NeRF-based models

Sammanfattning

Driver monitoring systems constitute a key tool for ensuring road safety, with the ongoing automatization and extended assistance tools for the control of cars. In that sense, a great effort has been put in ensuring these systems are paired with reliable sensors that avoid distractions of the driver. These distractions are indicated by the attention patterns of the user, which can be measured based on the gaze position and head pose. However, extracting these features efficiently and with confidence becomes a challenging task for the most advanced system. Machine learning has been one of the solutions to this problem. To keep the necessary reliability and resilience to hard cases, it is key to use good quality and extensive datasets. To alleviate the load on the collection of data and to reduce the resources and cost of the same, we explore the effects of generating synthetic samples from real users by means of generative NeRF networks. We perform a series of experiments where we train a regression machine learning model with a dataset of real samples that have been extended with new generated images. Using architectures such as GazeNeRF allows us to control the position of the samples output, so that we can create a corpus of uniform samples covering a uniform data-space. We compare the performance of the model against a model trained using only real samples using an evaluation dataset. The results of our study show that the use of the extended dataset lead to an increase in the accuracy of the network. This effect can be noted also in the case of the most extreme camera views, where the network trained using the larger expanded dataset proves a reduction of the prediction error metrics for the outlying cases. These results prove that the use of generated data in the training of machine learning models can lead to noticeable performance increases as well as reduce the reliance on the data collection process to ensure the coverage of the most outlying samples