Mixed-Precision Strategies for GPU-Based Electron Repulsion Integrals

Sammanfattning

Modern GPUs provide significantly higher throughput for single-precision arithmetic than for double-precision arithmetic, creating opportunities for performance optimization in computational chemistry. In quantum chemistry, the evaluation of electron repulsion integrals (ERIs) is one of the main computational bottlenecks in Hartree–Fock and density functional theory (DFT) calculations. For sufficiently small density-weighted ERI contributions, FP32 evaluation can be used with little loss of accuracy in the Coulomb matrix and the final SCF energy, because the associated absolute round-off error remains small. This thesis introduces and evaluates a mixed-precision scheme for ERI-based Coulomb matrix (J) contribution in the VeloxChem quantum chemistry software, within a GPU framework that supports both NVIDIA CUDA and AMD HIP platforms, although the present work is optimized and evaluated on NVIDIA GPUs. A threshold-based precision-selection strategy was designed to partition the Coulomb-build workload into FP64, FP32, and screened-out regions. Density-weighted ERI contributions above the precision threshold are retained in double precision, contributions below this threshold but above the screening threshold are evaluated in single precision, and contributions below the screening threshold are skipped. The switching criteria were formulated to improve performance while maintaining numerical stability and overall accuracy. Custom GPU kernels were developed and optimized to support efficient mixed-precision execution. Performance and accuracy were evaluated on representative molecular test cases. The results demonstrate that the proposed strategy achieves significant reductions in execution time while maintaining numerical stability and numerical accuracy within acceptable tolerances. The approach provides a practical method for accelerating quantum chemistry calculations on modern GPU architectures.