**Introduction :**

Corners, as feature cues in an image, is defined by two edge intersections. This definition has benefit in allowing precise location of the cue, although it is only valid when locality is maintained and the result is similar to the real corner location

**Related work:**
* Corner detector method present are SIFT global tracker that is using Difference of Gaussians and SURF that is using Haar wavelet to approximate Hessian determinat. These methods have drawback in high computation
* FAST method is a corner detector that performs better than conventional corner detectors such as Harris corner detection. Drawbacks of this method is that it depends on the environment, therefore decision tree needs to be always constructed from scratch (ID3 greedy algorithm)

**Approach:**
* In detecting corners, discretized circle pixels are used to be compared with the center area. A circle with 3.4 diameter is used as test mask
* Based on accelerated segment test, pixel is identified to be a corner when there are a number of pixel that has different value, either darker or lighter, than threshold of center pixel
* The number of pixel that is used in this paper is the same as FAST-9 method, which nine size segment as it has the best performance
* This number has the property to detect corners with some standard such that when different viewpoints are applied, it has the highest number of repeatability to detect corners correctly
* FAST algorithm is building ternary tree which has three possible states that are darker, lighter, or similar, added by unknown state that leads to $4^N$ configuration space, while FAST-9 has dissimilarity in the circle's thickness which is increased to 3 pixels
* Proposed corner detection describes the algorithm by testing one of the pixel with one question to pose. If a scenario of a pixel is given and question is evaluated, the next pixel and question in query is determined by the response
* This algorithm expands the configuration space by adding not lighter and not darker which produces a binary tree representation in which evaluation can be done in each of the nodes, therefore the configuration space has the size of $6^N$
* Memory is accessed by three types of cost which are second pixel comparison, same row pixel test, and other pixel test
* To make decision tree optimal, a method that has resemblance with backward induction is formulated. Configuration space is explored using Depth First Search algorithm where each leaf is described whether it has satisfy corner criteria by accelerated segment test
* Cost of each node is calculated by summation of minimum cost in each pair of child that has the positive or negative value with probability of pixel nodes both parent and child
* The algorithm calculates the probability of an image whether it has homogeneous and structured areas then proceed to make a decision tree according to it. The distribution of probability contains three probabilities that are mirror state probability and similar state probability

https://i.imgur.com/lIHh7gL.png

* Algorithm is improved to make it generic by jumping from one optimized tree to another based on the configuration of the respected leaf once it has termination condition based on the corner criteria
* This switching method has no cost as it happens in a leaf. However, it affects the time by one test delay. Thus, AGAST can only perform worse than FAST once it needs to jump between either homogeneous or heterogeneous pixels consecutively
* Corner detection is compared using three pixel mask sizes that are 4, 12 and 16. Comparison is done by applying Gaussian noise and blur to database of variety viewpoints of checkerboard database
* As limited computation using conventional computers are present, four state configuration space is used using three different arc lengths that are 9, 10 and 11
* As the mask and arc length enlarged, the more features found. Small arc defines the location of the real corner
* Large pattern leads to slower computation as it has more process to be done in order to detect the corner or evaluating features and it needs more advanced computing memory
* Smaller pattern can also lead to elimination of feature detection as the features located near to each other, therefore in smaller feature, post processing technique is removed
* When database is added by Gaussian blur and noise, the combination of pixel mask of 16 and arc length of 9 is more robust against the disturbance, therefore, repeatability is controlled by arc
* Decision tree is also evaluated by computing the response time of corner detection. This is done by calculating the tests number that has possible pixel arrangement in a mask 
* Pixels arrangements that have close similarities are grouped. By observing the standard deviation, the group with large number of pixels that are alike has unbalanced decision tree. This happens as possible pixel arrangements are limited. Observed as follows:

https://i.imgur.com/DLqCXhy.png

* However, when one tree is used, adaptive tree has better performance than the conventional method
* In comparison for trees that has different weight, the algorithm that jumps between trees or AGAST is optimized when the value of the weights are 0.1 and 1/3
* Performance of AGAST is also tested by comparison with FAST-9 where uniform probability distribution are used to make the trees
* Both algorithm are tested on five scenes, which are laboratory, outdoor, indoor, aerial, and medical. The optimized tree can speed the corner detection up for 13$\%$ while AGAST can speed up in the range of 23$\%$ to over 30$\%$ 

**Paper contributions**
* Paper provides an improved FAST corner detector that is able to dynamically adapt with its environment while also processing an image input
* AGAST method is improving its predecessors method in saving time spent to process the image and also memory that is being used
* It is also able to find more keypoints in corner detection

The paper provides derivations and intuitions about the learning dynamics for VAEs based on observations about [$\beta$-VAEs][beta]. Using this they derive an alternative way to constrain the training of VAEs that doesn't require typical heuristics, such as warmup or adding noise to the data.

How exactly would this change a typical implementation? Typically, SGD is used to [optimize the ELBO directly](https://github.com/pytorch/examples/blob/master/vae/main.py#L91-L95). Using GECO, I keep a moving average of my constraint $C$ (chosen based on what I want the VAE to do, but it can be just the likelihood plus a tolerance parameter) and use that to calculate Lagrange multipliers, which control the weighting of the constraint to the loss. [This implementation](https://github.com/denproc/Taming-VAEs/blob/master/train.py#L83-L97) from a class project appears to be correct.

With the stabilization of training, I can't help but think of this as batchnorm for VAEs.

[beta]: https://openreview.net/forum?id=Sy2fzU9gl

Karmon et al. propose a gradient-descent based method for obtaining adversarial patch like localized adversarial examples. In particular, after selecting a region of the image to be modified, several iterations of gradient descent are run in order to maximize the probability of the target class and simultaneously minimize the probability in the true class. After each iteration, the perturbation is masked to the patch and projected onto the valid range of [0,1] for images. On ImageNet, the authors show that these adversarial examples are effective against a normal, undefended network.

Also find this summary at [davidstutz.de](https://davidstutz.de/category/reading/).

Alaifari et al. propose an iterative attack to construct adversarial deformations of images. In particular, and in contrast to general adversarial perturbations, adversarial deformations are described through a deformation vector field – and the corresponding norm of this vector field may be bounded; an illustration can be found in Figure 1. The adversarial deformation is computed iteratively where the deformation itself is expressed in a differentiable manner. In contrast to very simple transformations such as rotations and translations, the computed adversarial deformations may contain significantly more subtle deformations as shown in Figure 2. The authors show that such deformations can successful attack MNIST and ImageNet models.

https://i.imgur.com/7N8rLaK.png
Figure 1: Illustration of the advantage of using general pixel-level deformations compared to simple transformations such as translations or rotations.

https://i.imgur.com/dCWBoI8.png
Figure 2: Illustration of untargeted (top) and targeted (bottom) attacks on ImageNet.

Also find this summary at [davidstutz.de](https://davidstutz.de/category/reading/).

  * They present a variation of Faster R-CNN, i.e. a model that predicts bounding boxes in images and classifies them.
  * In contrast to Faster R-CNN, their model is fully convolutional.
  * In contrast to Faster R-CNN, the computation per bounding box candidate (region proposal) is very low.

### How
  * The basic architecture is the same as in Faster R-CNN:
    * A base network transforms an image to a feature map. Here they use ResNet-101 to do that.
    * A region proposal network (RPN) uses the feature map to locate bounding box candidates ("region proposals") in the image.
    * A classifier uses the feature map and the bounding box candidates and classifies each one of them into `C+1` classes,
      where `C` is the number of object classes to spot (e.g. "person", "chair", "bottle", ...) and `1` is added for the background.
      * During that process, small subregions of the feature maps (those that match the bounding box candidates) must be extracted and converted to fixed-sizes matrices.
        The method to do that is called "Region of Interest Pooling" (RoI-Pooling) and is based on max pooling.
        It is mostly the same as in Faster R-CNN.
    * Visualization of the basic architecture:
      * ![Architecture](https://raw.githubusercontent.com/aleju/papers/master/neural-nets/images/R-FCN__architecture.jpg?raw=true "Architecture")
  * Position-sensitive classification
    * Fully convolutional bounding box detectors tend to not work well.
    * The authors argue, that the problems come from the translation-invariance of convolutions, which is a desirable property in the case of classification but not when precise localization of objects is required.
    * They tackle that problem by generating multiple heatmaps per object class, each one being slightly shifted ("position-sensitive score maps").
    * More precisely:
      * The classifier generates per object class `c` a total of `k*k` heatmaps.
      * In the simplest form `k` is equal to `1`. Then only one heatmap is generated, which signals whether a pixel is part of an object of class `c`.
      * They use `k=3*3`. The first of those heatmaps signals, whether a pixel is part of the *top left* corner of a bounding box of class `c`. The second heatmap signals, whether a pixel is part of the *top center* of a bounding box of class `c` (and so on).
      * The RoI-Pooling is applied to these heatmaps.
      * For `k=3*3`, each bounding box candidate is converted to `3*3` values. The first one resembles the top left corner of the bounding box candidate. Its value is generated by taking the average of the values in that area in the first heatmap.
      * Once the `3*3` values are generated, the final score of class `c` for that bounding box candidate is computed by averaging the values.
      * That process is repeated for all classes and a softmax is used to determine the final class.
      * The graphic below shows examples for that:
        * ![Architecture](https://raw.githubusercontent.com/aleju/papers/master/neural-nets/images/R-FCN__examples.jpg?raw=true "Examples")
    * The above described RoI-Pooling uses only averages and hence is almost (computationally) free.
      * They make use of that during the training by sampling many candidates and only backpropagating on those with high losses (online hard example mining, OHEM).
  * À trous trick
    * In order to increase accuracy for small bounding boxes they use the à trous trick.
    * That means that they use a pretrained base network (here ResNet-101), then remove a pooling layer and set the à trous rate (aka dilation) of all convolutions after the removed pooling layer to `2`.
      * The á trous rate describes the distance of sampling locations of a convolution. Usually that is `1` (sampled locations are right next to each other). If it is set to `2`, there is one value "skipped" between each pair of neighbouring sampling location.
    * By doing that, the convolutions still behave as if the pooling layer existed (and therefore their weights can be reused). At the same time, they work at an increased resolution, making them more capable of classifying small objects. (Runtime increases though.)
  * Training of R-FCN happens similarly to Faster R-CNN.

### Results
  * Similar accuracy as the most accurate Faster R-CNN configurations at a lower runtime of roughly 170ms per image.
  * Switching to ResNet-50 decreases accuracy by about 2 percentage points mAP (at faster runtime). Switching to ResNet-152 seems to provide no measureable benefit.
  * OHEM improves mAP by roughly 2 percentage points.
  * À trous trick improves mAP by roughly 2 percentage points.
  * Training on `k=1` (one heatmap per class) results in a failure, i.e. a model that fails to predict bounding boxes. `k=7` is slightly more accurate than `k=3`.