High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
Wang, Ting-Chun
and
Liu, Ming-Yu
and
Zhu, Jun-Yan
and
Tao, Andrew
and
Kautz, Jan
and
Catanzaro, Bryan
arXiv e-Print archive - 2017 via Local Bibsonomy
Keywords:
dblp
https://i.imgur.com/lM3EjK9.png
Problem
=============
Label map (semantic segmentation) to realistic image using GANs.
Contributions
=========
1. Coarse-to-fine generator
2. Multi-scale discriminator
3. Robust adversarial learning objective function
Coarse-to-fine Generator
=================
https://i.imgur.com/osEyGOj.png
G1 - Global generator
G2 - Local enhancer
Global Generator:
1. convolutional front-end
2. set of residual blocks
3. transposed convolutional back-end
A semantic label map is passed through the 3 components sequentially
Local Enhancer:
1. convolutional front-end
2. set of residual blocks
3. transposed convolutional back-end
Training scheme:
1. Train standalone global generator
2. Freeze global generator weights, train local enhancer
3. Fine-tune all weights together
Multi scale Discriminator
===================
https://i.imgur.com/hNP1cni.png
To allow for global context but work at higher resolution as well, several discriminators are applied at different image scales.
Robust adversarial learning objective function
===============================
https://i.imgur.com/j7CIbV3.png
* Compare original and generated images in feature space at different scales.
* This is done to ensure more abstract resemblance, not just pixel-space resemblance.
* For feature extraction the discriminator is used.