Adversarially Learned Inference
Vincent Dumoulin and Ishmael Belghazi and Ben Poole and Alex Lamb and Martin Arjovsky and Olivier Mastropietro and Aaron Courville
arXiv e-Print archive - 2016 via Local arXiv
Keywords: stat.ML, cs.LG
First published: 2016/06/02 (6 years ago)
Abstract: We introduce the adversarially learned inference (ALI) model, which jointly learns a generation network and an inference network using an adversarial process. The generation network maps samples from stochastic latent variables to the data space while the inference network maps training examples in data space to the space of latent variables. An adversarial game is cast between these two networks and a discriminative network that is trained to distinguish between joint latent/data-space samples from the generative network and joint samples from the inference network. We illustrate the ability of the model to learn mutually coherent inference and generation networks through the inspections of model samples and reconstructions and confirm the usefulness of the learned representations by obtaining a performance competitive with other recent approaches on the semi-supervised SVHN task.
more
less
Vincent Dumoulin and Ishmael Belghazi and Ben Poole and Alex Lamb and Martin Arjovsky and Olivier Mastropietro and Aaron Courville
arXiv e-Print archive - 2016 via Local arXiv
Keywords: stat.ML, cs.LG
First published: 2016/06/02 (6 years ago)
Abstract: We introduce the adversarially learned inference (ALI) model, which jointly learns a generation network and an inference network using an adversarial process. The generation network maps samples from stochastic latent variables to the data space while the inference network maps training examples in data space to the space of latent variables. An adversarial game is cast between these two networks and a discriminative network that is trained to distinguish between joint latent/data-space samples from the generative network and joint samples from the inference network. We illustrate the ability of the model to learn mutually coherent inference and generation networks through the inspections of model samples and reconstructions and confirm the usefulness of the learned representations by obtaining a performance competitive with other recent approaches on the semi-supervised SVHN task.
[link]
* They suggest a new architecture for GANs.
* Their architecture adds another Generator for a reverse branch (from images to noise vector `z`).
* Their architecture takes some ideas from VAEs/variational neural nets.
* Overall they can improve on the previous state of the art (DCGAN).
### How
* Architecture
* Usually, in GANs one feeds a noise vector `z` into a Generator (G), which then generates an image (`x`) from that noise.
* They add a reverse branch (G2), in which another Generator takes a real image (`x`) and generates a noise vector `z` from that.
* The noise vector can now be viewed as a latent space vector.
* Instead of letting G2 generate *discrete* values for `z` (as it is usually done), they instead take the approach commonly used VAEs and use *continuous* variables instead.
* That is, if `z` represents `N` latent variables, they let G2 generate `N` means and `N` variances of gaussian distributions, with each distribution representing one value of `z`.
* So the model could e.g. represent something along the lines of "this face looks a lot like a female, but with very low probability could also be male".
* Training
* The Discriminator (D) is now trained on pairs of either `(real image, generated latent space vector)` or `(generated image, randomly sampled latent space vector)` and has to tell them apart from each other.
* Both Generators are trained to maximally confuse D.
* G1 (from `z` to `x`) confuses D maximally, if it generates new images that (a) look real and (b) fit well to the latent variables in `z` (e.g. if `z` says "image contains a cat", then the image should contain a cat).
* G2 (from `x` to `z`) confuses D maximally, if it generates good latent variables `z` that fit to the image `x`.
* Continuous variables
* The variables in `z` follow gaussian distributions, which makes the training more complicated, as you can't trivially backpropagate through gaussians.
* When training G1 (from `z` to `x`) the situation is easy: You draw a random `z`-vector following a gaussian distribution (`N(0, I)`). (This is basically the same as in "normal" GANs. They just often use uniform distributions instead.)
* When training G2 (from `x` to `z`) the situation is a bit harder.
* Here we need to use the reparameterization trick here.
* That roughly means, that G2 predicts the means and variances of the gaussian variables in `z` and then we draw a sample of `z` according to exactly these means and variances.
* That sample gives us discrete values for our backpropagation.
* If we do that sampling often enough, we get a good approximation of the true gradient (of the continuous variables). (Monte Carlo approximation.)
* Results
* Images generated based on Celeb-A dataset:
* 
* Left column per pair: Real image, right column per pair: reconstruction (`x -> z` via G2, then `z -> x` via G1)
* 
* Reconstructions of SVHN, notice how the digits often stay the same, while the font changes:
* 
* CIFAR-10 samples, still lots of errors, but some quite correct:
* 
Your comment:
|
You must log in before you can submit this summary! Your draft will not be saved!
Preview:
