Liu, Ming-Yu, Thomas Breuel, and Jan Kautz. “Unsupervised image-to-image translation networks.” Advances in neural information processing systems. 2017.

Paper

• arXiv:1703.00848

Code

• Author’s repo
• PyTorch-GAN

Related posts

• Two Minute Papers
• UNIT 介紹 - Unsupervised Image-to-Image Translation
• https://blog.csdn.net/leviopku/article/details/83653527

---

Introduction

CV problems such as super-resolution and colorization can be viewed as image-to-image translation problems, mapping an image in one domain to a corresponding image in another domain.

This problem can be studied in supervised and unsupervised learning settings. Getting data used for supervised learning is burdensome, while collecting data for unsupervised learning is easier.

In the unsupervised setting, the coupling theory¹ shows that inferring the joint distribution from the two marginal distributions is a highly ill-posed problem. To address this problem, the author makes a **shared-latent space** assumption, which assumes a pair of corresponding images in a different domains can be mapped to a same latent representation in a shared-latent space. Then it comes to the model.

We model each image domain using a VAE-GAN. The training objective interacts with a weight-sharing constraint, which enforces a shared-latent space, to generate corresponding images in two domains, while the variational autoencoders relate translated images with input images in the respective domains.

The proposed framework yielded high quality image translation results and achieved SOTA accuracies on domain adaptation problems.

The shared-latent space assumption was used in Coupled GAN² for joint distribution learning. (Here’s my notes of Coupled GAN)

They also show that the shared-latent space constraint implies the cycle-consistency³⁴ constraint.