Consistent Video Generation through Reinforcement Learning

Sammanfattning

Diffusion models are modern generative models that build on the theory of non-equilibrium statistical mechanics and are capable of producing highly realistic images and videos. Oftentimes, generating videos can take several minutes and require a lot of computational power just to generate samples during inference. Certain distilled models can generate high quality samples in fewer sampling steps, thus reducing the sampling time to seconds instead. However, these models can sometimes display deterioration in consistency, with objects deforming unrealistically being one example of quality reduction. Another issue is the production of static videos. Inspired by the alignment techniques using reinforcement learning for large language models and recently also for diffusion models, this thesis adapts a recent state-of-the-art algorithm to the context video generation. The optimization goal is to improve consistency and increase the distance traveled in the videos while only taking a few sampling steps with a distilled model. While the results are only partially successful, different parameter configurations showcase clearly different results. In particular, one configuration collapses to fully consistent but static videos while another actually improves the estimated distance traveled. The latter does deteriorate slightly in consistency but as improving distance seems a necessary first step, this configuration is highly promising for future work. Overall, the results showcase that the algorithm indeed works but is dependent on the reward model and the parameters of the configuration. Future work should therefore explore the latter of the mentioned configurations in particular, as it seems to learn in the desired direction, as well as increasing the robustness of the reward structure.