There are 2 implementations for the paper:

 1. [Reference Implementation of Deep Linear Discriminant Analysis (DeepLDA)](https://github.com/CPJKU/deep_lda).
 2. [VahidooX/DeepLDA](https://github.com/VahidooX/DeepLDA).

[It seems something is wrong with the cost function implemented](https://github.com/VahidooX/DeepLDA/issues/1#issuecomment-392261355).
Also, while they derive the Gradient they didn't verify it and in the implementation use Theano's Auto Grad (While other Auto Grad can't work it out).

Great paper! Summarizes unsupervised reinforcement learning techniques, both with a model and model free. Include TD learning, Q learning, exploration vs. exploitation tradeoff, and other details. Not difficult to read for a technical audience. Explanations are clear while avoiding unnecessary detail and the paper has copious references. Granted I'm biased since I took one of the author's courses

Ba et al. propose layer normalization, normalizing the activations of a layer by its mean and standard deviation. In contrast to batch normalization, this scheme does not depend on the current batch; thus, it performs the same computation at training and test time. The general scheme, however, is very similar. Given the $l$-th layer of a multi-layer perceptron,

$a_i^l = (w_i^l)^T h^l$ and $h_i^{l + 1} = f(a_i^l + b_i^l)$

with $W^l$ being the weight matrix, the activations $a_i^l$ are normalized by mean $\mu_i^l$ and standard deviation $\sigma_i^l$. For batch normalization these are estimated over the current mini batch:

$\mu_i^l = \mathbb{E}_{p(x)} [a_i^l]$ and $\sigma_i^l = \sqrt{\mathbb{E}_{p(x)} [(a_i^l - \mu_i^l)^2}$.

However, this estimation depends heavily on the batch size; additionally, models change during training and test time (at test time, these statistics are estimated over the training set). For layer normalization, instead, these statistics are evaluated over the activations in the same layer:

$\mu^l = \frac{1}{H}\sum_{i = 1}^H a_i^l$ and $\sigma^l = \sqrt{\frac{1}{H}\sum_{i = 1}^H (a_i^l - \mu^l)^2}$.

Thus, the normalization is not depending on the batch size anymore. Additionally, layer normalization is invariant to scaling and shifts of the weight matrix (for batch normalization, this only holds for the columns of the matrix). In experiments, this approach is shown to work well for a variety of tasks including models with attention mechanisms and recurrent neural networks. For convolutional neural networks, the authors state that layer normalization does not outperform batch normalization, but performs better than using no normalization at all.

Also find this summary at [davidstutz.de](https://davidstutz.de/category/reading/).

Deep convolutional neural networks (DCNN) has been a popular model for image classification over the last few years. This paper proposes a DCNN structure, also known as AlexNet, for the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC). To train AlexNet, which has 60 million parameters, this paper uses Rectified Linear Units (ReLU) and multiple GPU to accelerate training. This paper also report that using local response normalization and overlapping pooling can reduce error rate. To prevent over fitting, they suggest data augmentation and apply dropout in the fully connected layer. 
Technical details

The following figure shows the architecture of AlexNet. It contains five convolutional and three fully connected layers. Response-normalization layers follow the first and second convolutional layers. Max-pooling layers follow the first and second response-normalization layers and the fifth convolutional layer.

![](http://i.imgur.com/2iqwCq1.png)

This paper discusses an approach to integrated task and motion planning. They argue that the primary requirement for the task planner is the ability generate task plans based on the motion planning results. To this end, they employ Satisfiability Modulo Theories solver to solve for a task plan, and dynamically add or remove task constraints based on the motion planning results.

More specifically, their task and motion planner, IDTMP, begins with only initial and goal state constraints, as well as the preconditions and effects at time step 0. It also begins with a time step of 1, which means it will try to find a task within one time step. If it determines that it is unable to produce a task of length 1 that abides by the constraints, it pops the goal constraint and then add additional preconditions and effects at time step h+1, and then add the goal constraint back in. They repeat this until a feasible task plan is found. 

Once the task plan is found, it then tries to find a set of motions plans that is associated with each task operators. They have a special operator, $\lambda_\rho$, which computes the goal configuration for motion planner; however, note that for complex task and motion planning tasks, this function is not easy to define. If a feasible motion plan is not found in any one of the operators, they add the constraint that says you should not try this task operator in this state, and re-plan the task plan using this constraint.

They prove the probabilistic completeness by noting the fact that the incremental SMT solver considers all previous task plans and RRT is probabilistic complete. The experiment is shown for a rather simple task of stacking box objects on top of another; comparison to only one other method is shown.

While this paper has some interesting ideas, such as  incrementally incorporating motion planning results as constraints for finding a task plan, it lacks some motivations as to why they had to use SMT solver. Would simple CSP solver suffice? A random sampling planner? Also, their task is rather too simple, and they get away with the difficulty of sampling continuous parameters for task operators without any explanation. Their comparison with only one benchmark makes this approach not very convincing.