Wu and He propose group normalization as alternative to batch normalization. Instead of computing the statistics used for normalization based on the current mini-batch, group normalization computes these statistics per instance but in groups of channels (for convolutional layers). Specifically, given activations $x_i$ with $i = (i_N, i_C, i_H, i_W)$ indexing along batch size, channels, height and width, batch normalization computes

$\mu_i = \frac{1}{|S|}\sum_{k \in S} x_k$ and $\sigma_i = \sqrt{\frac{1}{|S|} \sum_{k \in S} (x_k - \mu_i)^2 + \epsilon}$

with the set $S$ holds all indices for a specific channel (i.e. across samples, height and width). For group normalization, in contrast, $S$ holds all indices of the current instance and group of channels. Meaning the statistics are computed across height, width and the current group of channels. Here, all channels can be divided into groups arbitrarily. In the paper, on ImageNet, groups of $32$ channels are used. Then, Figure 1 shows that for a batch size of 32, group normalization performs en-par with batch normalization – although the validation error is slightly larger. This is attributed to the stochastic element of batch normalization that leads to regularization. Figure 2 additionally shows the influence of the batch size of batch normalization and group normalization.

https://i.imgur.com/lwP5ycw.jpg
Figure 1: Training and validation error for different normalization schemes on ImageNet.

https://i.imgur.com/0c3CnEX.jpg
Figure 2: Validation error for different batch sizes.

Also find this summary at [davidstutz.de](https://davidstutz.de/category/reading/).

This paper introduces a Stacked Attention Network (SAN) for visual question answering.
SAN uses a multiple layer attention mechanism that uses the semantic question representation
to query the image and locate relevant visual regions, and to infer the answer.
Details of the SAN model:

- Image features are extracted from the last pooling layer of a deep CNN (like VGG-net).
    - Input images are first scaled to 448 x 448, so at the last pooling layer, features
    have the dimension 14 x 14 x 512 i.e. 512-dimensional vectors at each image location
    with a receptive field of 32 x 32 in input pixel space.

- Question features are the last hidden state of the LSTM.
    - Words are one-hot encoded, transferred to a vector space by passing through an
    embedding matrix and these word vectors are fed into the LSTM at each time step.

- Image and question features are combined into a query vector to locate relevant visual regions.
    - Both the LSTM hidden state and 512-d image feature vector at each location are transferred
    to the same dimensionality (say k) by a fully connected layer, and added and passed through
    a non-linearity (tanh).
    - Each k-dimensional feature vector is then transformed down to a single scalar and
    a softmax is taken over all image regions to get the attention distribution (say p\_{I}).
    - This attention distribution is used to weight the pooling layer visual features (\sum_{i}p\_{i}v\_{i})
    and added to the LSTM vector to get a new query vector.
    - In subsequent attention layers, this updated query vector is used to repeat the same process
    of getting an attention distribution.
    - The final query vector is used to compute a softmax over the answers.

## Strengths

- The multi-layer attention mechanism makes sense intuitively and the qualitative results
somewhat indicate that going from the first attention layer to subsequent attention layers,
the network is able to focus on fine-grained visual regions as it discovers relationships
among multiple objects ('what are sitting in the basket on a bicycle').

- SAN benefits VQA, they demonstrate state-of-the-art accuracies on multiple datasets, with
question-type breakdown as well.

## Weaknesses / Notes

- Right now, the attention distribution is learnt in an unsupervised manner by the network.
It would be interesting to think about adding supervisory attention signal. Another way to
improve accuracies would be to use deeper LSTMs.

What the paper is about:
KeypointNet learns the optimal set of 3D keypoints and their 2D detectors for a specified downstream task. The authors demonstrate this by extracting 3D keypoints and their 2D detectors for the task of relative pose estimation across views. They show that, using keypoints extracted by KeypointNet, relative pose estimates are superior to ones that are obtained from a supervised set of keypoints.


Approach:
Training samples for KeypointNet comprise two views (images) of an object. The task is to then produce an ordered list of 3D keypoints that, upon orthogonal procrustes alignment, produce the true relative 3D pose across those views. The network has N heads, each of which extracts one (3D) keypoint (from a 2D image). There are two primary loss terms. A multi-view consistency loss measures the discrepancy between the two sets of extracted keypoints under the ground-truth transform. A relative-pose estimation loss penalizes the angular discrepency (under orthogonal procrustes) of the estimated transform using the extracted keypoints vs the GT transform. Additionally, they require keypoints to be distant from each other, and to lie within the object silhouette.

Catastrophic forgetting is the tendency of an neural network to forget previously learned information when learning new information. This paper combats that by keeping a buffer of experience and applying meta-learning to it. They call their new module Meta Experience Replay or MER.

How does this work? At each update they compute multiple possible updates to the model weights. One for the new batch of information and some more updates for batches of previous experience. Then they apply  meta-learning using the REPTILE algorithm, here the meta-model sees each possible update and has to predict the output which combines them with the least interference. This is done by predicting an update vector that maximizes the dot product between the new and old update vectors, that way it transfers as much learning as possible from the new update without interfering with the old updates. https://i.imgur.com/TG4mZOn.png

Does it work? Yes, while it may take longer to train, the results show that it generalizes better and needs a much smaller buffer of experience than the popular approach of using replay buffers.

## TL;DR

The paper presents a model-agnostic strategy to perform few-shot learning taking advantage of prior knowledge acquired during in multitask learning. Such prior knowledge derives from priors acquired about generalized model parameters (e.g. weights or hyperparameters) during the Model Agnostic Meta-Learning (MAML) algorithm. The strategy can be applied to any algorithm trained with gradient descent (not only neural networks) being more general and perhaps effective than transfer learning. It can loosely be referred to as "learning to learn".

## Why this is interesting

* Suitable in combination with any technique that uses gradient descent (supervised learning, reinforcement learning)
* Interesting idea: instead of further optimize existing models for performances, search a representation that can be subsequently tuned
* when only a few and diverse data is available, multiple tasks can be defined to harness the ability of the meta-model to learn while preserving generalization (see Experiments)

## Details

The key idea is performing the meta-learner update on a different data batch with respect to the parameters update(s) done for a single task. This leads   (formally) the same update procedure both for the learning and meta-learning phases of the algorithm (see Figure below) and provides a general framework for MAML. 

https://i.imgur.com/xq1wCai.png
image from [lilianweng post](https://lilianweng.github.io/lil-log/2018/11/30/meta-learning.html)

As [clearly worked out here](https://lilianweng.github.io/lil-log/2018/11/30/meta-learning.html), the method requires to compute the second derivatives for the outer-loop update. Surprisingly enough, omitting them and performing a first-order MAML does not sensibly affect the results according to the reported experiment. It is hypothesized that this is because ReLU networks are almost linear (and hence depends upon the actual network structure). 

## Experiments
### Supervised learning
1. regression from input to output of a sine wave. Amplitude and phase are varied among tasks. It is shown that MAML leads to good results and can generalize better than fine-tuning in the experiment conditions ("due to the often contradictory outputs on pre-training tasks". See Figure 2 in the paper)
2. few-shot image classification on Omniglot and MiniImageNet datasets (N, unseen, multiclass trained with K different instances). SOTA performance on the first dataset, and better-than SOTA on the second one where first-order MAML is also tested. 

### Reinforcement learning
1. 2D navigation: a single agent point must move to different positions (tasks). The model trained with MAML shows better performances, for the same number of gradient steps (Figure 4)
2. Locomotion: two simulated robots are provided with a set of tasks. MAML can learn a model that adapts much faster to new tasks (this is a case where pretraining is detrimental)
 
## Related work and resources
* official [gthub repo](https://github.com/cbfinn/maml)
* [videos](https://sites.google.com/view/maml) of the learned policies in MAML paper
* paper appendix: the part on the multi-task baseline is interesting
* [how to train your MAML](https://arxiv.org/abs/1810.09502): discusses various modifications to MAML to stabilize and improve performances
* [Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML](https://arxiv.org/abs/1909.09157)