Optimal Transport for Video Frame Interpolation

Sammanfattning

Video frame interpolation aims to increase the framerate of a video by generating intermediate images between each frame. In this thesis, we propose an approach to this problem based on optimal transport. By viewing each frame as brightness values assigned to an integer grid, the interpolation problem is formulated as transporting brightness in the first frame such that it becomes the next frame, while minimizing the total transport cost. This leads to a Kantorovich optimal transport problem which is solved using Sinkhorn's algorithm. The method is then tested in various scenarios with different pre- and post- processing strategies. The results show that the method yields visually plausible results, with limitations imposed by the computational cost, visual artifacts and limited object persistence across frames.