#### Problem addressed: 
Estimation of the position of facial keypoints

#### Summary: 
The authors propose an effective convolutional network cascade for facial point detection. Deep convolutional networks at the first level provide highly robust initial estimations, while shallower convolutional networks at the following two levels finely tune the initial prediction to achieve high accuracy. The method therefore can avoid local minimum caused by ambiguity and data corruption. Networks at the first level are deep convolutional networks with four convolutional stages, absolute value rectification, and locally shared weights. Networks at the second and third levels share a common shallower structure. Since they are designed to extract local features, deep structures and locally sharing weights are unnecessary.

#### Novelty:
Three-level carefully designed convolution networks

#### Drawbacks:
High computational cost because of deep model

#### Datasets:
BioID, LFPW

#### Resources:
http://www.ee.cuhk.edu.hk/~xgwang/papers/sunWTcvpr13.pdf

#### Presenter:
Neeti Narayan

#### Problem addressed: 
Strategy for training deep neural networks

#### Summary: 
The input distribution (to every layer) undergoes constant changes while training a deep network. The authors call this internal covariate shift in the input distribution. The authors claim this leads to slow learning of optimal model parameters. In order to overcome this, they introduce the idea of normalizing the input of every layer a part of the optimization strategy. Specifically, they reparameterize the input to each layer so that it is whitened and thus has non-changing distribution at every iteration.
 
They apply 2 approximation in their strategy: 

1. this normalization is done for every mini-batch of training data, 

2. the input dimensions are assumed to be uncorrelated. 

Finally, the output of last layer is mean subtracted and variance normalized (these can be back-propagated while training). Additionally, the authors also introduce 2 learnable scalar parameters $(r,b)$ per dimension such that the final input to a layer is $y=rg(BN(x))+b$ where g is the activation function.
 
 The advantage of BN apart from the intuition mentioned above is that it allows higher learning rate and network behavior remains unaffected by the scale of the parameters W and bias. The authors also empirically show that BN acts as a regularizer since optimization without dropout yields at par performance.

#### Novelty:
Previous work only focused on whitening in 1st layer input. This work extends this idea to all layers and suggests a practical approach for applying this idea to real world data.

#### Datasets:
Imagenet

#### Resources:
presentation video available on cedar server

#### Presenter:
Devansh Arpit

#### Problem addressed: 
A new type of activation function

#### Summary: 
This paper propose a new activation function that computes a Lp norm from multiple projections on an input vector. The p value can be learned from training example, and can also be different for each hidden unit. The intuition is that 1) for different datasets there may exist different optimal p-values, so it make more sense to make p tunable; 2) allowing different unit take different p-values can potentially make the approximation of decision boundaries more efficient and more flexible. The empirical results support these two intuitions, and achieved comparable results on three datasets.

#### Novelty:
A generalization of pooling but applied through channels, when the data and weight vector dot product plus bias is constrained to non-negative case, the $L_\infty$ is equivalent to maxout unit.

#### Drawbacks:
Empirical performance is not very impressive, although evidence of supporting the intuition occurs.

#### Datasets:
MNIST, TFD, Pentomino

#### Resources:
http://arxiv.org/abs/1311.1780

#### Presenter:
Yingbo Zhou

#### Problem addressed: 
Bayesian approximation of neural netowrks

#### Summary: 
This paper gives an alternative view of dropout as Bayesian approximation, which allow one to obtain uncertainty from the predictions. The result is surprisingly simple, both the predictive mean and variance can be obtained by calculating the mean and variance (with some minor adjustment) of multiple passes through the network with dropout.

#### Novelty:
A new interpretation of dropout as a Bayesian approximation.

#### Drawbacks:
Some computational overhead, since calculating the predictive mean and variance need multiple passes through the network.

#### Datasets:
MNIST, solar irradiance, Maunua Loa Co2,

#### Resources:
Paper: http://arxiv.org/pdf/1506.02142v1.pdf

Blog post: http://mlg.eng.cam.ac.uk/yarin/blog_3d801aa532c1ce.html

#### Presenter:
Yingbo Zhou"

#### Problem addressed: 
A fast way of finding adversarial examples, and a hypothesis for the adversarial examples

#### Summary: 
This paper tries to explain why adversarial examples exists, the adversarial example is defined in another paper \cite{arxiv.org/abs/1312.6199}. The adversarial example is kind of counter intuitive because they normally are visually indistinguishable from the original example, but leads to very different predictions for the classifier. For example, let sample $x$ be associated with the true class $t$. A classifier (in particular a well trained dnn) can correctly predict $x$ with high confidence, but with a small perturbation $r$, the same network will predict $x+r$ to a different incorrect class also with high confidence.
 
 This paper explains that the exsistence of such adversarial examples is more because of low model capacity in high dimensional spaces rather than overfitting, and got some empirical support on that. It also shows a new method that can reliably generate adversarial examples really fast using `fast sign' method. Basically, one can generate an adversarial example by taking a small step toward the sign direction of the objective. They also showed that training along with adversarial examples helps the classifier to generalize.

#### Novelty:
A fast method to generate adversarial examples reliably, and a linear hypothesis for those examples.

#### Datasets:
MNIST

#### Resources:
Talk of the paper https://www.youtube.com/watch?v=Pq4A2mPCB0Y

#### Presenter:
Yingbo Zhou