They describe a CNN architecture that can be used to identify a person given an image of their face.

### How
  * The expected input is the image of a face (i.e. it does not search for faces in images, the faces already have to be extracted by a different method).
  * *Face alignment / Frontalization*
    * Target of this step: Get rid of variations within the face images, so that every face seems to look straight into the camera ("frontalized").
    * 2D alignment
      * They search for landmarks (fiducial points) on the face.
        * They use SVRs (features: LBPs) for that.
        * After every application of the SVR, the localized landmarks are used to transform/normalize the face. Then the SVR is applied again. By doing this, the locations of the landmarks are gradually refined.
      * They use the detected landmarks to normalize the face images (via scaling, rotation and translation).
    * 3D alignment
      * The 2D alignment allows to normalize variations within the 2D-plane, not out-of-plane variations (e.g. seeing that face from its left/right side). To normalize out-of-plane variations they need a 3D transformation.
      * They detect an additional 67 landmarks on the faces (again via SVRs).
      * They construct a human face mesh from a dataset (USF Human-ID).
      * They map the 67 landmarks to that mesh.
      * They then use some more complicated steps to recover the frontalized face image.
  * *CNN architecture*
    * The CNN receives the frontalized face images (152x152, RGB).
    * It then applies the following steps:
      * Convolution, 32 filters, 11x11, ReLU (-> 32x142x142, CxHxW)
      * Max pooling over 3x3, stride 2 (-> 32x71x71)
      * Convolution, 16 filters, 9x9, ReLU (-> 16x63x63)
      * Local Convolution, 16 filters, 9x9, ReLU (-> 16x55x55)
      * Local Convolution, 16 filters, 7x7, ReLU (-> 16x25x25)
      * Local Convolution, 16 filters, 5x5, ReLU (-> 16x21x21)
      * Fully Connected, 4096, ReLU
      * Fully Connected, 4030, Softmax
    * Local Convolutions use a different set of learned weights at every "pixel" (while a normal convolution uses the same set of weights at all locations).
    * They can afford to use local convolutions because of their frontalization, which roughly forces specific landmarks to be at specific locations.
    * They use dropout (apparently only after the first fully connected layer).
    * They normalize "the features" (probably the 4096 fully connected layer). Each component is divided by its maximum value across a training set. Additionally, the whole vector is L2-normalized. The goal of this step is to make the network less sensitive to illumination changes.
    * The whole network has about 120 million parameters.
    * Visualization of the architecture:
      * ![Architecture](https://raw.githubusercontent.com/aleju/papers/master/neural-nets/images/DeepFace__architecture.jpg?raw=true "Architecture")
  * *Training*
    * The network receives images, each showing a face, and is trained to classify the identity of the face (e.g. gets image of Obama, has to return "that's Obama").
    * They use cross-entropy as their loss.
  * *Face verification*
    * In order to tell whether two images of faces show the same person they try three different methods.
    * Each of these relies on the vector extracted by the first fully connected layer in the network (4096d).
    * Let these vectors be `f1` (image 1) and `f2` (image 2). The methods are then:
      1. Inner product between `f1` and `f2`. The classification (same person/not same person) is then done by a simple threshold.
      2. Weighted X^2 (chi-squared) distance. Equation, per vector component i: `weight_i (f1[i] - f2[i])^2 / (f1[i] + f2[i])`. The vector is then fed into an SVM.
      3. Siamese network. Means here simply that the absolute distance between `f1` and `f2` is calculated (`|f1-f2|`), each component is weighted by a learned weight and then the sum of the components is calculated. If the result is above a threshold, the faces are considered to show the same person.

### Results
  * They train their network on the Social Face Classification (SFC) dataset. That seems to be a Facebook-internal dataset (i.e. not public) with 4.4 million faces of 4k people.
  * When applied to the LFW dataset:
    * Face recognition ("which person is shown in the image") (apparently they retrained the whole model on LFW for this task?):
      * Simple SVM with LBP (i.e. not their network): 91.4% mean accuracy.
      * Their model, with frontalization, with 2d alignment: ??? no value.
      * Their model, no frontalization (only 2d alignment): 94.3% mean accuracy.
      * Their model, no frontalization, no 2d alignment: 87.9% mean accuracy.
    * Face verification (two images -> same/not same person) (apparently also trained on LFW? unclear):
      * Method 1 (inner product + threshold): 95.92% mean accuracy.
      * Method 2 (X^2 vector + SVM): 97.00% mean accurracy.
      * Method 3 (siamese): Apparently 96.17% accuracy alone, and 97.25% when used in an ensemble with other methods (under special training schedule using SFC dataset).
  * When applied to the YTF dataset (YouTube video frames):
    * 92.5% accuracy via X^2-method.