Ji et al. propose a model-reuse, or trojaning, attack against neural networks by deliberately manipulating specific weights. In particular, given a specific input, the attacker intends to manipulate the model into mis-classifying this input. This is achieved by first generating semantic neighbors of the input, e.g. through transformations or noise, and then identifying salient features for these inputs. These features are correlated to the classifiers output, i.e. some of them have positive impact on classification, some of them have negative impact. The model is fine-tuned by actively adapting the identifying features until the target input is mis-classified.

Also find this summary at [davidstutz.de](https://davidstutz.de/category/reading/).

Pérolat et al. propose a game-theoretic variant of adversarial training on universal adversarial perturbations. In particular, in each training iteration, the model is trained for a specific number of iterations on the current training set. Afterwards, a universal perturbation is found (and the corresponding test images) that fools the network. The found adversarial examples are added to the training set. In the next iteration, the network is trained on the new training set which includes adversarial examples. Overall, this leads to a network being trained on a sequence of universal adversarial perturbations corresponding to earlier versions of that network.

Also find this summary at [davidstutz.de](https://davidstutz.de/category/reading/).

Russu et al. discuss robustness of linear and non-linear kernel machines through regularization. In particular, they show that linear classifiers can easily be regularized to be robust. In fact, robustness against $L_\infty$-bounded adversarial examples can be achieved through $L_1$ regularization on the weights. More generally, robustness against $L_p$ attacks are countered by $L_q$ regularization of the weights, with $\frac{1}{p} + \frac{1}{q} = 1$. These insights are generalized to the case of non-linear kernel machines; I refer to the paper for details.

Also find this summary at [davidstutz.de](https://davidstutz.de/category/reading/).

Rusu et al. Propose progressive networks, sets of networks allowing transfer learning over multiple tasks without forgetting. The key idea of progressive networks is very simple. Instead of fine-tuning a model (for transfer learning), the pre-trained model is taken and its weights fixed. Another network is then trained from scratch while receiving features from the pre-trained network as additional input.

Specifically, the authors consider a sequence of tasks. For the first task, a deep neural network (e.g. multi-layer perceptron) is trained. Assuming $L$ layers with hidden activations $h_i^{(1)}$ for $i \leq L$,  each layer computes

$h_i^{(1)} = f(W_i^{(1)} h_{i-1}^{(1)})$

where $f$ is an activation function and for $i = 1$, the network input is used. After training until convergence, a second network is trained – now on a different task. The parameters of the first network is fixed, but the second network can use the features of the first one:

$h_i^{(2)} = f(W_i^{(2)} h_{i-1}^{(2)} + U_i^{(2:1)}h_{i-1}^{(1)})$.

This idea can be generalized to the $k$-the network, which can use the activations from all the previous networks:

$h_i^{(k)} = f(W_i^{(k)} h_{i-1}^{(k)} + \sum_{j < k} U_i^{(k:j)} h_{i-1}^{(j)})$.

For three networks, this is illustrated in Figure 1.

https://i.imgur.com/ndyymxY.png
Figure 1: An illustration of the feature transfer between networks.

In practice, however, this approach results in an explosion of parameters and computation. Therefore, the authors apply a dimensionality reduction to the $h_{i – 1}^{(j)}$ for $j < k$. Additionally, an individual scaling factor is used to account for different ranges used in the different networks (also depending on the input data). Then, the above equation can be rewritten as

$h_i^{(k)} = f(W_i^{(k)} h_{i-1}^{(k)} + \sum_{j < k} U_i^{(k)} f(V_i^{(k)} \alpha_i^{(:k)} h_{i-1}^{(:k)})$.

(Note that notation has been adapted slightly, as I found the original notation misleading.) Here, $h_{i – 1}^{(:k)}$ denotes the concatenated features from all networks $j < k$. Similarly, for each network, one $\alpha_i^{(j)}$ is learned to scale the features (note that the notation above would imply a element-wise multiplication of the $\alpha_i^{(j)}$'s repeated in a vector, or equivalently a matrix-vector product). $V_i^{(k)}$ then describes a dimensionality reduction; overall, a one-layer perceptron is used to “transfer” features from networks $j < k$ to the current network. The same approach can also be applied to convolutional layers (e.g. a $1 \times 1$ convolution can be used for dimensionality reduction).

In experiments, the authors show that progressive networks allow efficient transfer learning (efficient in terms of faster training). Additionally, they study which features are actually transferred.

Also find this summary at [davidstutz.de](https://davidstutz.de/category/reading/).

Shafahi et al. discuss fundamental limits of adversarial robustness, showing that adversarial examples are – to some extent – inevitable. Specifically, for the unit sphere, the unit cube as well as for different attacks (e.g., sparse attacks and dense attacks), the authors show that adversarial examples likely exist. The provided theoretical arguments also provide some insights on which problems are more (or less) robust. For example, more concentrated class distributions seem to be more robust by construction. Overall, these insights lead the authors to several interesting conclusions:  First, the results are likely to extent to datasets which actually live on low-dimensional manifolds of the unit sphere/cube. Second, it needs to be differentiated between the existence adversarial examples and our ability to compute them efficiently. Making it harder to compute adversarial examples might, thus, be a valid defense mechanism. And third, the results suggest that lower-dimensional data might be less susceptible to adversarial examples.

Also find this summary at [davidstutz.de](https://davidstutz.de/category/reading/).