# Smart Reply: Automated Response Suggestion for Email

## Introduction

* Proposes a novel, end-to-end architecture for generating short email responses.
* Single most important benchmark of its success is that it is deployed in Inbox by Gmail and assists with around 10% of all mobile responses.
* [Link to the paper.](https://arxiv.org/abs/1606.04870)

## Challenges in deploying Smart Reply in a user-facing product

* Responses must always be of high quality. Ensured by constructing a target response set to select responses from.
* The likelihood of choosing the responses must be maximised. Ensured by normalising the responses and enforcing diversity.
* The system should not add latency to emails. Ensured by using a triggering model to decide if the email is suitable to undergo the response generation pipeline. Computation time is further reduced by finding approximate best result instead of the best result.
* Ensure privacy by encrypting all the data which adds challenge in verifying the model's quality and debugging the system.

## Architecture

## Preprocess Email

* Perform actions like language detection, tokenization, sentence segmentation etc on the input email.

## Triggering Model

* A feed-forward neural network (with embedding layer and 3 fully connected hidden layers) to decide if the input email is suitable for suggesting responses.

#### Data

* Training set of pairs *(o, y)* where *o* is the incoming message and *y* is a boolean variable to indicate if the message had a response.

#### Features

* Unigrams, bigrams from the messages.
* Signals like - is the recipient in the contact list of the sender.

## Response Selection

* LSTM network to predict the approximate best response for an incoming message *o*

#### Network

* Sequence to Sequence Learning.
* Reads the input message (token by token) and encode a vector representation.
* Compute softmax to get the probability of first output token given the input token sequence.
* Keep feeding in the previous response tokens and the input token sequence to compute the probability of next output token.
* During inference, approximate the most likely response greedily by taking the most likely response at each timestamp and feeding it back or by using the beam search approach.

## Response Set Generation

* Generate a set of high-quality responses that also capture the variability in the intent of the response.
* Canonicalize the email response by extracting the semantic structure using a dependency parser.
* Partition all response messages into "semantic" clusters.
* These semantic clusters define the response space for scoring and selecting possible responses and for promoting diversity among the responses.

## Semantic Intent Clustering

* Since a large, labelled dataset is not available, a graph based, semi-supervised approach is used.

#### Graph Construction

* Manually define a few clusters with a small number of example responses for each cluster.
* Construct a graph with frequent response messages (including the labelled nodes) as response nodes (V<sub>R</sub>).
* For each response node, extract a set of feature nodes (V<sub>F</sub>) corresponding to features like skip-gram and n-grams and add an edge between the response node and the feature node.
* Learn a semantic labelling for all response nodes by propagating semantic intent information (available because of labelled nodes) throughout the graph.
* After some iterations, sample some of the unlabeled nodes from the graph, manually label these sample nodes and repeat this algorithm until convergence.
* For validation, extract the top k members of each cluster and validate the quality with help of human evaluators.

## Suggestion Diversity

* Provide users with a varied set of response by omitting redundant response (by not selecting more than one response from any semantic cluster) and by enforcing negative (or positive) responses.
* If the top two responses contain at least one positive (negative) response and none of the top three responses is
negative (positive), the third response is replaced with a negative (positive) one.
* This is done by performing a second LSTM pass where the search is restricted to only positive (or negative) responses in the target set.

## Strengths

* The system is already in production and assists with around 10% of all mobile responses.

Low level tasks such as edge, contour and line detection are an essential precursor to any downstream image analysis processes. However, most of the approaches targeting these problems work as isolated and autonomous entities, without using any high-level image information such as context, global shapes, or user-level input. This leads to errors that can further propagate through the pipeline without providing an opportunity for future correction. In order to address this problem, Kass et al. investigate the application of an energy minimization based framework for edge, line and contour detection in 2D images. 

Although energy minimization had earlier been utilized for similar tasks, Kass et al’s framework exposes a novel external force factor, which allows external forces or stimuli to guide the ``snake" towards a “correct answer”. Moreover, the framework exhibits an active behaviour since it is designed to always minimize the energy functional. A “snake” is a controlled continuity spline which is under the influence of forces in the energy functional. The energy functional is made up of three terms, where $v(s)$ is a parametrized snake. 
$    E_{snake}^{*} = \int_{0}^{1}E_{internal}(v(s)) +  E_{image}(v(s)) + E_{constraint}(v(s)) $

The internal energy force term is entirely dependent on the shape of the curve, which constraints the snake to be “continuous” and well behaved. The force further encapsulates two terms which control the degree of stretchness of the curve (represented by the first derivative of the image), and ensure that the curve does not have too many bends (using the second derivative of the image). 

The image energy force term controls what kind of salient features does the snake track. The force encapsulates three terms, corresponding to the presence of lines, edges and a termination term. The line term uses raw image intensity as energy term, while the edge term uses a negative square of image gradient to make the snake attracted to contours with large image gradients. Further, a termination term allows the snake to find terminations of line segments and corners in the image. The combination of line and termination terms forces the snake to be attracted to edges and terminations. 

The constraint term is used to model external stimuli, which can come from high-level processes, or through user intervention. The framework allows the application of a spring to the curve to constraint the snake or move it in a desired direction. 

In order to test the framework, a user interface called ``Snake Pit" was created which allowed user control in the form of a spring attachment. The overall approach was tested on a number of different images, including the ones with contour illusion. The framework was also extended for application to stereo matching and motion tracking. For stereo, an extra energy term is added to constraint the snakes in two disparate images to stay close in the coordinate space, which models the fact that disparities in images do not change too rapidly. 
However the framework suffers when subjected to a high rate-of-change in both stereo matching and motion tracking problems.

The proposed framework performed acceptably in many challenging scenarios. However the framework's underlying assumption about following edges and contours using image gradients may fail in cases where there is not much gradient information present in images. 

TLDR; The authors train a multilingual Neural Machine Translation (NMT) system based on the Google NMT architecture by prepend a special `2[lang]` (e.g. `2fr`) token to the input sequence to specify the target language. They empirically evaluate model performance on many-to-one, one-to-many and many-to-many translation tasks and demonstrate evidence for shared representations (interlingua).

Cao and Gong introduce region-based classification as defense against adversarial examples. In particular, given an input (benign test input or adversarial example), the method samples random point in the neighborhood and classifies the test sample according to the majority vote of the obtained labels.

Also find this summary at [davidstutz.de](https://davidstutz.de/category/reading/).

Modeling and simulation of proton-exchange membrane fuel cells (PEMFC) may work as a powerful tool in the Research & development of renewable energy sources. The Open-Source PEMFC Simulation Tool (OPEM) is a modeling tool for evaluating the performance of proton exchange membrane fuel cells. This package is a combination of models (static/dynamic) that predict the optimum operating parameters of PEMFC. OPEM contained generic models that will accept as input, not only values of the operating variables such as anode and cathode feed gas, pressure and compositions, cell temperature and current density, but also cell parameters including the active area and membrane thickness. In addition, some of the different models of PEMFC that have been proposed in the OPEM, just focus on one particular FC stack, and some others take into account a part or all auxiliaries such as reformers. OPEM is a platform for collaborative development of PEMFC models.