This paper proposes a neural architecture that allows to backpropagate gradients though a procedure that can go through a variable and adaptive number of iterations. These "iterations" for instance could be the number of times computations are passed through the same recurrent layer (connected to the same input) before producing an output, which is the case considered in this paper. 

This is essentially achieved by pooling the recurrent states and respective outputs computed by each iteration. The pooling mechanism is essentially the same as that used in the really cool Neural Stack architecture of Edward Grefenstette, Karl Moritz Hermann, Mustafa Suleyman and Phil Blunsom \cite{conf/nips/GrefenstetteHSB15}. It relies on the introduction of halting units, which are sigmoidal units computed at each iteration and which gives a soft weight on whether the computation should stop at the current iteration.

Crucially, the paper introduces a new ponder cost $P(x)$, which is a regularization cost that penalizes what is meant to be a smooth upper bound on the number of iterations $N(t)$ (more on that below).

The paper presents experiment on RNNs applied on sequences where, at each time step t (not to be confused with what I'm calling computation iterations, which are indexed by n) in the sequence the RNN can produce a variable number $N(t)$ of intermediate states and outputs. These are the states and outputs that are pooled, to produce a single recurrent state and output for the time step t. During each of the $N(t)$ iterations at time step t, the intermediate states are connected to the same time-step-t input. After the $N(t)$ iterations, the RNN pools the $N(t)$ intermediate states and outputs, and then moves to the next time step $t+1$. To mark the transitions between time steps, an extra binary input is appended, which is 1 only for the first intermediate computation iteration. 

Results are presented on a variety of synthetic problems and a character prediction problem.

TLDR; The authors introduce CopyNet, a variation on the seq2seq that incorporates a "copying mechanism". With this mechanism, the effective vocabulary is the union of the standard vocab and the words in the current source sentence. CopyNet predicts words based on a mixed probability of the standard attention mechanism and a new copy mechanism. The authors show empirically that on toy and summarization tasks CopNet behaves as expected: The decoder is dominated by copy mode when it tries to replicate something from the source.

#### Motivation:

+ Take advantage of the fact that missing values can be very informative about the label.
+ Sampling a time series generates many missing values.

![Sampling](https://raw.githubusercontent.com/tiagotvv/ml-papers/master/clinical-data/images/Lipton2016_motivation.png?raw=true)


#### Model (indicator flag):

+ Indicator of occurrence of missing value.

![Indicator](https://raw.githubusercontent.com/tiagotvv/ml-papers/master/clinical-data/images/Lipton2016_indicator.png?raw=true)

+ An RNN can learn about missing values and their importance only by using the indicator function. The nonlinearity from this type of model helps capturing these dependencies.
+ If one wants to use a linear model, feature engineering is needed to overcome its limitations.
  + indicator for whether a variable was measured at all
  + mean and standard deviation of the indicator 
  + frequency with which a variable switches from measured to missing and vice-versa. 

#### Architecture:

+ RNN with target replication following the work "Learning to Diagnose with LSTM Recurrent Neural Networks" by the same authors.

![Architecture](https://raw.githubusercontent.com/tiagotvv/ml-papers/master/clinical-data/images/Lipton2016_architecture.png?raw=true)


#### Dataset:

+ Children's Hospital LA
+ Episode is a multivariate time series that describes the stay of one patient in the intensive care unit

Dataset properties  | Value 
---------|----------
Number of episodes | 10,401
Duration of episodes | From 12h to several months
Time series variables | Systolic blood pressure, Diastolic blood pressure,  Peripheral capillary refill rate, End tidal CO2, Fraction of inspired O2,  Glasgow coma scale,  Blood glucose,  Heart rate,  pH,  Respiratory rate, Blood O2 Saturation,  Body temperature,  Urine output.



#### Experiments and Results:

**Goal**
+ Predict 128 diagnoses.
+ Multilabel: patients can have more than one diagnose.

**Methodology**
+ Split: 80% training, 10% validation, 10% test.
+ Normalized data to be in the range [0,1].

+ LSTM RNN:
  + 2 hidden layers with 128 cells. Dropout = 0.5, L2-regularization: 1e-6
  + Training for 100 epochs. Parameters chosen correspond to the time that generated the smallest error in the validation dataset.
+ Baselines:
  + Logistic Regression (L2 regularization)
  + MLP with 3 hidden layers and 500 hidden neurons / layer (parameters chosen via validation set)
  + Tested with raw-features and hand-engineered features.
+ Strategies for missing values:
  + Zeroing
  + Impute via forward / backfilling
  + Impute with zeros and use indicator function
  + Impute via forward / backfilling and use indicator function
  + Use indicator function only

#### Results

+ Metrics:
  + Micro AUC, Micro F1: calculated by adding the TPs, FPs, TNs and FNs for the entire dataset and for all classes.
  + Macro AUC, Macro F1: Arithmetic mean of AUCs and F1 scores for each of the classes.
  + Precision at 10: Fraction of correct diagnostics among the top 10 predictions of the model.
    + The upper bound for precision at 10 is 0.2281 since in the test set there are on average 2.281 diagnoses per patient.

![Results](https://raw.githubusercontent.com/tiagotvv/ml-papers/master/clinical-data/images/Lipton2016_results.png?raw=true)


#### Discussion:

+ Predictive model based on data collected following a given routine. This routine can change if the model is put into practice. Will the model predictions in this new routine remain valid?

+ Missing values in a way give an indication of the type of treatment being followed.

+ Trade-off between complex models operating on raw features and very complex features operating on more interpretable models.

Rakin et al. introduce the bit-flip attack aimed to degrade a network’s performance by flipping a few weight bits. On Cifar10 and ImageNet, common architectures such as ResNets or AlexNet are quantized into 8 bits per weight value (or fewer). Then, on a subset of the validation set, gradients with respect to the training loss are computed and in each layer, bits are selected based on their gradient value. Afterwards, the layer which incurs the maximum increase in training loss is selected. This way, a network’s performance can be degraded to chance level with as few as 17 flipped bits (on ImageNet, using AlexNet).

Also find this summary at [davidstutz.de](https://davidstutz.de/category/reading/).

Teo et al. propose a convex, robust learning framework allowing to integrate invariances into SVM training. In particular, they consider a set of valid transformations and define the cost of a training sample (i.e., pair of data and label) as the loss under the worst case transformation – this definition is very similar to robust optimization or adversarial training. Then, a convex upper bound on this cost is derived. Given, that the worst case transformation can be found efficiently, two different algorithms for minimizing this loss are proposed and discussed. The framework is evaluated on MNIST. However, considering error rate, the approach does not outperform data augmentation significantly (here, data augmentation means adding “virtual, i.e., transformed, samples to the training set). However, the proposed method seems to find sparser solutions, requiring less support vectors.

Also find this summary at [davidstutz.de](https://davidstutz.de/category/reading/).