Latent Space Interpolation in the Variational Autoencoder and the Denoising Diffusion Probabilistic Model

Sammanfattning

This work investigates image synthesis of two generative models: the Variational Autoencoder (VAE) and the Denoising Diffusion Probabilistic Model (DDPM). One approach for synthesis is to interpolate between real samples within the models' latent spaces. To understand how we can interpolate in the models' latent space, we derive the loss functions of the models. We trained the models on MNIST and Fashion-MNIST dataset. With that knowledge, we explain why continuous linear interpolation in the latent space often finds the shortest interpolation path in the VAE: because of a the geometric structuredness and continuity of the latent space. In the DDPM, linear interpolation in the latent space tends to fail for samples with different features because the model does not learn a smooth geometric structure for the latent representation of the data. We show how increasing the dimensionality of the latent space from two to three in the VAE results in more clarity of the synthesized samples. In the two dimensional latent space we train a degree two polynomial by the difference between each image in the pixel space and by encouraging proximity to high density regions in the latent space. This resulted in shorter interpolation path compared to linear interpolation in the latent space.