Since all algorithms can be modeled as multiple conditional branch operations, this paper allows you to incorporate conventional algorithms into neural networks by dynamically building the neural computation graph based on outputs of these algorithms.

They obtain near SOTA on Quora Duplicate Questions and SQuAD without heavily fine tuning the architecture to each problem.

One limitation is that the algorithm itself is not affected by the learning process and so cannot be learned.

This method provides a nice way to incorporate non-differentiable code into differentiable computation graphs which can be learned via backprop like learning mechanisms.

Fernanda Viegas, Marin Wattenberg, and Kushal Dave describe a visualization system they have built called history flow that they use to visualize changes made to Wikipedia articles. The authors suggest that their papers makes three distinct contributions:

* History flow itself which is able to reveal editing patterns in Wikipedia and provide context for editors.
* Several examples of collaboration patterns that become visible using the visualization tool and contribute to the literature on Wikipedia.
* Implications of these patterns for design and governance of online social spaces.

The paper is largely an examination of Wikipedia and the early parts of the paper give background into the sites. It uses shortcomings in the design of the Wikipedia to motivate the history flow visualization which essentially depicts articles, over time, with colors representing authors who contributed text in question. Examples can be seen online at the IBM History Flow website. The interface is particularly good at representing major deletions and insertions.

The authors use a lightweight statistically analysis to reveal patterns of editing on Wikipedia (which at the time, were not widely studied). In particular, they show vandalism including mass-deletion, the creation of phony redirects, and addition of idiosyncratic copy and show that it rarely stays on the site for more than few minutes before being removed.

They also show a zig-zag patten that represents negotiation of content, often in the form of edit wars. They also attempt to provide some basic data on the stability of Wikipedia and the growth of articles on average. They suggest something that is now taken for granted by researchers of wikis: that studying Wikipedia may have important implications for other types of work.

#### Theoretical and practical relevance:

The paper is important more for its path-breaking work on Wikipedia -- now with its track at CHI, than for the history flow visualization which has not, for the most part, been widely deployed outside Wikipedia but which seems to hold promise in a variety of other contexts. The paper has been cited more than 400 times, mostly in the academic literature on Wikipedia.

This paper is a finalist for the Wikimedia France Research Award.

As in Q-learning, modern actor-critic methods suffer from value estimation errors due to high bias and variance. While there are many attempts to address this in Q-learning (such as Double DQN), not much was done in actor-critic methods.

Authors of the paper propose three modifications to DDPG and empirically show that they help address both bias and variance issues:

* 1.) Clipped Double Q-Learning:  
  Add a second pair of critics $Q_{\theta}$ and $Q_{\theta_\text{target}}$ (so four critics total) and use them to upper-bound the value estimate target update: $y = r + \gamma \min\limits_{i=1,2} Q_{\theta_{target,i}}(s', \pi_{\phi_1}(s'))$
* 2.) Reduce number of policy and target networks updates, and magnitude of target networks updates: $\theta_{target} \leftarrow \tau\theta + (1-\tau)\theta_{target}$
* 3.) Inject (clipped) random noise to the target policy: $\hat{a} \leftarrow \pi_{\phi_{target}}(s) + \text{clip}(N(0,\sigma), -c, c)$

Implementing these results, authors show significant improvements on seven continuous control tasks, beating not only reference DDPG algorithm, but also PPO, TRPO and ACKTR.

Full algorithm from the paper:

https://i.imgur.com/rRjwDyT.png

Source code: https://github.com/sfujim/TD3

#### Problem addressed: 
Estimation of the position of facial keypoints

#### Summary: 
The authors propose an effective convolutional network cascade for facial point detection. Deep convolutional networks at the first level provide highly robust initial estimations, while shallower convolutional networks at the following two levels finely tune the initial prediction to achieve high accuracy. The method therefore can avoid local minimum caused by ambiguity and data corruption. Networks at the first level are deep convolutional networks with four convolutional stages, absolute value rectification, and locally shared weights. Networks at the second and third levels share a common shallower structure. Since they are designed to extract local features, deep structures and locally sharing weights are unnecessary.

#### Novelty:
Three-level carefully designed convolution networks

#### Drawbacks:
High computational cost because of deep model

#### Datasets:
BioID, LFPW

#### Resources:
http://www.ee.cuhk.edu.hk/~xgwang/papers/sunWTcvpr13.pdf

#### Presenter:
Neeti Narayan

The paper extends the [WGAN](http://www.shortscience.org/paper?bibtexKey=journals/corr/1701.07875) paper by replacing the L2 norm in the transportation cost by some other metric $d(x, y)$. By following the same reasoning as in the WGAN paper one arrives at a dual optimization problem similar to the WGAN's one except that the critic $f$ has to be 1-Lipschitz w.r.t. a given norm (rather than L2). This, in turn, means that critic's gradient (w.r.t. input $x$) has to be bounded in the dual norm (only in Banach spaces, hence the name). Authors build upon the [WGAN-GP](http://www.shortscience.org/paper?bibtexKey=journals/corr/1704.00028) to incorporate similar gradient penalty term to force critic's constraint.

In particular authors choose [Sobolev norm](https://en.wikipedia.org/wiki/Sobolev_space#Multidimensional_case):
$$
||f||_{W^{s,p}} = \left( \int \sum_{k=0}^s ||\nabla^k f(x)||_{L_p}^p dx \right)^{1 / p}
$$

This norm is chosen because it not only forces pixel values to be close, but also the gradients to be close as well. The gradients are small when you have smooth texture, and big on the edges -- so this loss can regulate how much you care about the edges. Alternatively, you could express the same norm by first transforming the $f$ using the Fourier Transform, then multiplying the result by $1 + ||x||_{L_2}^2$ pointwise, and then transforming it back and integrating over the whole space:
$$
||f||_{W^{s,p}} = \left( \int \left( \mathcal{F}^{-1} \left[ (1 + ||x||_{L_2}^2)^{s/2} \mathcal{F}[f] (x) \right] (x) \right)^p dx \right)^{1 / p}
$$

Here $f(x)$ would be image pixels intensities, and $x$ would be image coordinates, so $\nabla^k f(x)$ would be spatial gradient -- the one you don't have access to, and it's a bit hard to estimate one with finite differences, so the authors go for the second -- fourier -- option. Luckily, a DFT transform is just a linear operator, and fast implementations exists, so you can backpropagate through it (TensorFlow already includes tf.spectal)

Authors perform experiments on CIFAR and report state-of-the-art non-progressive results in terms of Inception Score (though not beating SNGANs by a statistically significant margin). The samples they present, however, are too small to tell if the network really cared about the edges.