## IDS approaches for events
** Misuse-based: **
Detects events that violate system policy. *Snort* is a signature-based open-source IDS system used for misuse detection in this research. 

** Anomaly-based: ** 
Detects events that contain abnormal activity. Uses statistical, heuristic and data mining methods.  *Packet header anomaly detector (PHAD)* [1] and *Network traffic anomaly detector (NETAD)* [2] are used as *Snort* preprocessors for anomaly detection.


## Hybrid architecture
**Preprocessors** (for *Snort)*: Can filter and modify packets before it reaches main detection engine. They can also generate alerts individually.

In section 4, researchers explain *Snort* software details for preprocessor source-code integration and their implementation of the hybrid system.

## Evaluation of hybrid system
*MIT Lincoln Laboratory IDEVAL dataset* used for evaluation of IDS system. A more detailed explanation and simulation environment information about the dataset can be found in section 5, as well in reference articles [3-5].

https://i.imgur.com/SrAvySJ.png

Researchers evaluated *Snort* only, *Snort + PHAD* and *Snort + PHAD + NETAD * (hybrid IDS) variations and compared detection capabilities of each system based on detected intrusion counts. Results can be seen in the figure. Based on results hybrid IDS is the most successful on *IDEVAL dataset*. 

## References

[1] Mahoney MV, Chan PK. PHAD: packet header anomaly detection for identifying hostile network traffic. Florida Institute of Technology Technical Report, CS-2001-04.

[2] Mahoney MV. Network traffic anomaly detection based on packet bytes. In Proceedings of ACM-SAC; 2003.

[3] Lippman R, Haines JW, Fried DJ, Korba J, Das K. Analysis and results of the 1999 DARPA off-line intrusion detection evaluation. In Proceedings of the
third international workshop on recent advances in intrusion detection, Toulouse, France; 2–4 October 2000. p. 162–82.

[4] Haines JW, Lippman R, Fried DJ, Zissman MA, Tran E, Boswell SB. 1999 DARPA intrusion detection evaluation: design and procedures. MIT Lincoln Laboratory Technical Report, TR-1062, Massachusetts, USA; 2001.

[5] Data set, DARPA intrusion detection evaluation data set; 1999. <http://www.ll.mit.edu/IST/ideval/data/1999/1999_data_index.html>
.

  * They present a variation of Faster R-CNN.
    * Faster R-CNN is a model that detects bounding boxes in images.
  * Their variation is about as accurate as the best performing versions of Faster R-CNN.
  * Their variation is significantly faster than these variations (roughly 50ms per image).

### How
  * PVANET reuses the standard Faster R-CNN architecture:
    * A base network that transforms an image into a feature map.
    * A region proposal network (RPN) that uses the feature map to predict bounding box candidates.
    * A classifier that uses the feature map and the bounding box candidates to predict the final bounding boxes.
  * PVANET modifies the base network and keeps the RPN and classifier the same.
  * Inception
    * Their base network uses eight Inception modules.
    * They argue that these are good choices here, because they are able to represent an image at different scales (aka at different receptive field sizes)
      due to their mixture of 3x3 and 1x1 convolutions.
      * ![Receptive field sizes in inception modules](images/PVANET__inception_fieldsize.jpg?raw=true "Receptive field sizes in inception modules")
    * Representing an image at different scales is useful here in order to detect both large and small bounding boxes.
    * Inception modules are also reasonably fast.
    * Visualization of their Inception modules:
      * ![Inception modules architecture](images/PVANET__inception_modules.jpg?raw=true "Inception modules architecture")
  * Concatenated ReLUs
    * Before the eight Inception modules, they start the network with eight convolutions using concatenated ReLUs.
    * These CReLUs compute both the classic ReLU result (`max(0, x)`) and concatenate to that the negated result, i.e. something like `f(x) = max(0, x <concat> (-1)*x)`.
    * That is done, because among the early one can often find pairs of convolution filters that are the negated variations of each other.
      So by adding CReLUs, the network does not have to compute these any more, instead they are created (almost) for free, reducing the computation time by up to 50%.
    * Visualization of their final CReLU block:
      * TODO
      * ![CReLU modules](https://raw.githubusercontent.com/aleju/papers/master/neural-nets/images/PVANET__crelu.jpg?raw=true "CReLU modules")
  * Multi-Scale output
    * Usually one would generate the final feature map simply from the output of the last convolution.
    * They instead combine the outputs of three different convolutions, each resembling a different scale (or level of abstraction).
    * They take one from an early point of the network (downscaled), one from the middle part (kept the same) and one from the end (upscaled).
    * They concatenate these and apply a 1x1 convolution to generate the final output.
  * Other stuff
    * Most of their network uses residual connections (including the Inception modules) to facilitate learning.
    * They pretrain on ILSVRC2012 and then perform fine-tuning on MSCOCO, VOC 2007 and VOC 2012.
    * They use plateau detection for their learning rate, i.e. if a moving average of the loss does not improve any more, they decrease the learning rate. They say that this increases accuracy significantly.
    * The classifier in Faster R-CNN consists of fully connected layers. They compress these via Truncated SVD to speed things up. (That was already part of Fast R-CNN, I think.)

### Results
  * On Pascal VOC 2012 they achieve 82.5% mAP at 46ms/image (Titan X GPU).
    * Faster R-CNN + ResNet-101: 83.8% at 2.2s/image.
    * Faster R-CNN + VGG16: 75.9% at 110ms/image.
    * R-FCN + ResNet-101: 82.0% at 133ms/image.
  * Decreasing the number of region proposals from 300 per image to 50 almost doubles the speed (to 27ms/image) at a small loss of 1.5 percentage points mAP.
  * Using Truncated SVD for the classifier reduces the required timer per image by about 30% at roughly 1 percentage point of mAP loss.

Despite being an ill-posed problem, non-rigid image registration has been the subject of numerous works, which apply the framework on different applications where rigid and affine transformations cannot completely model the variations between image sets. One such application of non-rigid registration is to register pre- and post-contrast breast MR images for estimating contrast uptake, which in turn is an indicator of the tumor malignancy. Due to large variations between the pre- and post-contrast images in terms of patient movement, and breast movement which is both global and local, registration of these images become challenging. Classic methods cannot capture the exact semantics of movements that the images exhibit.

To this end, the authors propose a non-rigid registration method which combines advantages of voxel-based similarity measures like Mutual Information (MI) as well as non-rigid transformation models of the breast. The method is built using a hierarchical transformation model which capture both the global and local movement of the breast across pre- and post-contrast scans. 

The proposed method consists of two interesting contributions which model the motion of the breast across scans using a global and local model. The global motion model conists of a 3D affine transformation parameterized by 12 degrees of freedom. 

The local model is based upon FFD model, which is based upon B-splines which is a powerful tool for modeling 3D deformable objects. The main idea behind this approach is that FFDs can be used to deform an object by manipulating the underlying mesh of the control points, which is estimated in the form of a B-spline. The formulation exposes a trade-off between computational running time and accurate modelling of the object (breast). In order to achieve the best compromise, a hierarchical multi-resolution approach is implemented in which the resolution of the control mesh is increased along with the image resolution  in a coarse to fine fashion. 

In addition to modeling the movement of the breast, a regularization term is also added to the final optimization function which forces the B-spline based FFD transformation to be smooth. The term is zero in case of affine transformation, and only penalizes non-affine transformations. 

The function that the method optimizes is as follows:

$\mathcal{C}(\theta, \phi) = - \mathcal{C}_{similarity}\left( I(t_0), T(I(t))\right) + \lambda \mathcal{C}_{smooth}(T)$

The optimization of the above objective function is performed in multiple stages, and by using gradient descent algorithm which takes steps towards the gradient vector with a certain step size of $\mu$. Local optimum is assumed if $||\nabla \mathcal{C}|| <= \epsilon$.

In order to assess the quality of the proposed method, the method is tested on both clinical and volunteer 3D MR data. The results show that the rigid and affine only transformations based methods perform significantly worse than the proposed method. Moreover, it was shown that the results improve with better control point resolution. However the use of SSD as a quantitative metric is debatable since the contrast enhanced and pre-contrast images will have varying distributions of intensity values. 

Early study showing effect of an antibiotic on gut microbiome and the majority of community recovered within 4 wks

TLDR; The authors train a Hierarchical Recurrent Encoder-Decoder (HRED) network for dialog generation. The "lower" level encodes a sequence of words into a though vector, and the higher-level encoder uses these thought vectors to build a representation of the context. The authors evaluate their model on the *MoviesTriples* dataset using perplexity measures and achieve results better than plain RNNs and the DCGM model. Pre-training with a large Question-Answer corpus significantly reduces perplexity.

#### Key Points

- Three RNNs: Utterance encoder, context encoder, and decoder. GRU hidden units, ~300d hidden state spaces.
- 10k vocabulary. Preprocessing: Remove entities and numbers using NLTK
- The context in the experiments is only a single utterance
- MovieTriples is a small dataset, about 200k training triples. Pretraining corpus has 5M Q-A pairs, 90M tokens.
- Perplexity is used as an evaluation metric. Not perfect, but reasonable.
- Pre-training has a much more significant impact than the choice of the model architecture. It reduces perplexity ~10 points, while model architecture makes a tiny difference (~1 point).
- Authors suggest exploring architectures that separate semantic from syntactic structure
- Realization: Most good predictions are generic. Evaluation metrics like BLEU will favor pronouns and punctuation marks that dominate during training and are therefore bad metrics.


#### Notes/Questions

- Does using a larger dataset eliminate the need for pre-training?
- What about the more challenging task for longer contexts?