Unpaired Image-to-Image Translation for Domain Adaptation with Diffusion Model : a Novel Approach based on Masking and Inpainting Techniques

Sammanfattning

Diffusion models have proven highly effective in generative tasks, particularly image generation, and are now integral to state-of-the-art image generator frameworks. These models operate by denoising images and exhibit remarkable adaptability, enabling their use in tasks like image editing without needing to train additional networks. This research focuses on image-to-image translation for domain adapta- tion. The task involves transforming an input image from one category (e.g., a cat) into another (e.g., a dog) while preserving the essential features of the original. This task is challenging as it requires minimal alterations to retain recognizability of the source image, while ensuring it conforms to the target category. While image-to-image translation has been previously explored, our in- vestigation aims to enhance existing methods without resorting to specialized models. This approach ensures the applicability of our methods to various transfers without the need for model re-training. By reinterpreting this task as an inpainting problem, we first conduct experiments to improve the efficiency of inpainting, including new techniques and theoretical results. Then, we explore sampling techniques from inpainting and novel masking methods to isolate regions for modification while preserving others for image- to-image translation. This approach led to the development of the Progressive Inpainting Mask (PIM), a new framework for diffusion models. PIM applies varying noise levels to the same input, enabling a more refined exploration beyond traditional single-level noise application. Experimental results confirm the statistical significance of adding a mask and using inpainting techniques. Our experimental findings demonstrate superior performance compared to state-of-the-art methods for the tasks of Cat-to-Dog translation and outperforming all diffusion models for the Male-to-Female translation, regardless of the inclusion of assistance from specialized networks. We have achieved more faithful images (−15% with the L2-distance), by deteriorating the realism by only 1.5%.