Mesh2Splat : Gaussian Splatting from 3D Geometry and Materials

Sammanfattning

The field of Computer Graphics continually seeks methods to enhance the realism and efficiency of rendering 3D scenes. This thesis presents a method for directly converting 3D meshes and materials into a 3D Gaussian Splatting (3DGS) format, with the goal of producing an output suitable for real-time rendering with dynamic lighting. 3DGS, a novel view synthesis and point-based rendering technique, serves as both the final representation and the rendering method for visualizing the converted data. The primary problem addressed is the current absence of a direct conversion method from 3D meshes to a 3DGS format without relying on time-consuming optimization processes. Current optimization-based methods are not tailored for synthetic data, requiring extensive computations to estimate 3D geometry from image-based inputs, which hampers their practicality for production applications. The significance of this problem lies in the necessity for more efficient, flexible, and high-fidelity rendering techniques in real-time applications, particularly within the gaming industry. Existing options are either computationally prohibitive or fail to reproduce the target ground truth visual fidelity, thus impeding their adoption. The novelty and rapidly evolving nature of Gaussian Splatting mean that standards and methods are still under development, making this a timely and relevant topic for a Master's thesis. This project introduces Mesh2Splat, a method that directly converts 3D meshes into Gaussian Splatting representations without the need for intermediate optimization steps. By leveraging the detailed information contained within the 3D meshes, Mesh2Splat aims to improve visual fidelity and computational efficiency compared to existing methods. The method involves developing C++ code to create Gaussian Splatting files from static 3D models, evaluating visual accuracy using metrics like SSIM and PSNR, and comparing performance against classical Gaussian Splatting pipelines. Key results demonstrate that Mesh2Splat significantly reduces computation time bringing the conversion time from several minutes to milliseconds, while maintaining and improving visual accuracy for most non-volumetric input 3D meshes with explicit surfaces. This efficiency supports the seamless conversion of 3D mesh formats into a customized Gaussian Splatting representations. Although a complete, user-validated tool was not developed, Mesh2Splat not only advances the understanding and application of 3D Gaussian Splatting but also provides a foundation for further research and development in the field.