This paper is a direct extension of the work of \cite{journals/corr/1305.2532}. Prior to this work, \cite{journals/corr/1305.2532} proposed an algorithm for learning to predict fixed size lists using either a single greedy policy (SCP) or a list of sequentially applied policies (CONSEQOPT). Given a sub-modular reward function, \cite{journals/corr/1305.2532} provided theoretical guarantees on the regret of the learned policies via a reduction to cost sensitive classification. In practice one can then approximate the cost sensitive classification loss with convex surrogate losses that can be efficiently learned.  In this work, the authors extend the framework of \cite{journals/corr/1305.2532} to knapsack problems, wherein each element in the list has an associated length and the sum total lengths may not exceed a fixed budget. (Note that I intentionally use the word "length" to differentiate from the "cost" of cost-sensitive classification in the reduction, which gets a bit confusing in the paper as the authors overload the word "cost" frequently.) They extend the algorithm of \cite{journals/corr/1305.2532} to be sensitive to the length of items added to the list, and extend the analysis of \cite{journals/corr/1305.2532} to provide regret guarantees in this setting. The authors then apply their algorithm to the problem of multi-document summarization, where items correspond to sentences and the budget constraints the total length of the summary. They show an improvement over prior work with their new approach.  

The paper is reasonably well written and straightforward to follow, with the exception of containing lots of notation and terminology that is sometimes difficult to keep straight (e.g. the definitions of the words "cost" and "weight", and the use of \ell as both cost sensitive losses and the length (cost? weight?) of each item for the budget.) The work itself is a useful extension to prior work that appears to be experimentally sound.  

Pros:  

+ Novel extension of prior work to incorporate budgetary constraints 
+ Works well experimentally  

Cons:  
- Overloading of symbols and words leads to confusion  

References:  
\cite{journals/corr/1305.2532} Ross, Stephane, Zhou, Jiaji, Yue, Yisong, Dey, Debadeepta, and Bagnell, .A. Learning policies for contextual submodular prediction. In ICML, 2013.

TLDR; The authors propose new ways to incorporate context (previous sentences) into a Recurrent Language Model (RLM). They propose 3 ways to model the context, and 2 ways to incorporate the context into the predictions for the current sentence. Context can be modeled with BoW, Sequence BoW (BoW for each sentence), and Sequence BoW with attention. Context can be incorporated using "early fusion", which gives the context as an input to the RNN, or "late fusion", which modifies the LSTM to directly incorporate the context. The authors evaluate their architecture on IMDB, BBC and Penn TreeBank corpora, and show that most approaches perform well (reducing perplexity), with Sequence BoW with attention + late fusion outperforming all others.

#### Key Points:

- Context as BoW: Compress N previous sentences into a single BoW vector
- Context as Sequential Bow: Compress each of the N previous sentences into a BoW vector and use an LSTM to "embed" them. Alternatively, use an attention mechanism.
- Early Fusion: Give the context vector as an input to the LSTM, together with the current word.
- Late Fusion: Add another gate to the LSTM that incorporates the context vector. Helps to combat vanishing gradients.
- Interestingly the Sequence BoW without attention performs very poorly. The reason here seems to be the same as for seq2seq, it's hard to compress the sentence vectors into a single fixed-length representation using an LSTM.
- LSTM models trained with 1000 units, Adadelta. Only sentences up to 50 words are considered.
- Noun phrases seem to benefit the most from the context, which makes intuitive sense.


#### Notes/Questions:

- A problem with current Language Models is that they are corpus-specific. A model trained on one corpus doesn't do well on another corpus because all sentences are treated as being independent. However, if we can correctly incorporate context we may be able to train a general-purpose LM that does well across various corpora. So I think this is important work.
- I am surprised that the authors did not try using a sentence embedding (skip-thought, paragraph-vector) to construct their context vectors. That seems like an obvious choice over using BoW.
- The argument for why the Sequence BoW without attention model performs poorly isn't convincing. In the seq2seq work the argument for attention was based on the length of the sequence. However, here the sequence is very short, so the LSTM should be able to capture all the dependencies. The performance may be poor due to the BoW representation, or due too little training data.
- Would've been nice to visualize what the attention mechanism is modeling.
- I'm not sure if I agree with the authors that relying explicit sentence boundaries is an advantage, I see it as a limiting factor.

  * They propose a two-stage GAN architecture that generates 256x256 images of (relatively) high quality.
  * The model gets text as an additional input and the images match the text.

### How
  * Most of the architecture is the same as in any GAN:
    * Generator G generates images.
    * Discriminator D discriminates betweens fake and real images.
    * G gets a noise variable `z`, so that it doesn't always do the same thing.
  * Two-staged image generation:
    * Instead of one step, as in most GANs, they use two steps, each consisting of a G and D.
    * The first generator creates 64x64 images via upsampling.
    * The first discriminator judges these images via downsampling convolutions.
    * The second generator takes the image from the first generator, downsamples it via convolutions, then applies some residual convolutions and then re-upsamples it to 256x256.
    * The second discriminator is comparable to the first one (downsampling convolutions).
    * Note that the second generator does not get an additional noise term `z`, only the first one gets it.
    * For upsampling, they use 3x3 convolutions with ReLUs, BN and nearest neighbour upsampling.
    * For downsampling, they use 4x4 convolutions with stride 2, Leaky ReLUs and BN (the first convolution doesn't seem to use BN).
  * Text embedding:
    * The generated images are supposed to match input texts.
    * These input texts are embedded to vectors.
    * These vectors are added as:
      1. An additional input to the first generator.
      2. An additional input to the second generator (concatenated after the downsampling and before the residual convolutions).
      3. An additional input to the first discriminator (concatenated after the downsampling).
      4. An additional input to the second discriminator (concatenated after the downsampling).
    * In case the text embeddings need to be matrices, the values are simply reshaped to `(N, 1, 1)` and then repeated to `(N, H, W)`.
    * The texts are converted to embeddings via a network at the start of the model.
      * Input to that vector: Unclear. (Concatenated word vectors? Seems to not be described in the text.)
      * The input is transformed to a vector via a fully connected layer (the text model is apparently not recurrent).
      * The vector is transformed via fully connected layers to a mean vector and a sigma vector.
      * These are then interpreted as normal distributions, from which the final output vector is sampled. This uses the reparameterization trick, similar to the method in VAEs.
      * Just like in VAEs, a KL-divergence term is added to the loss, which prevents each single normal distribution from deviating too far from the unit normal distribution `N(0,1)`.
      * The authors argue, that using the VAE-like formulation -- instead of directly predicting an output vector (via FC layers) -- compensated for the lack of labels (smoother manifold).
      * Note: This way of generating text embeddings seems very simple. (No recurrence, only about two layers.) It probably won't do much more than just roughly checking for the existence of specific words and word combinations (e.g. "red head").
  * Visualization of the architecture:
    * ![Architecture](https://raw.githubusercontent.com/aleju/papers/master/neural-nets/images/StackGAN__architecture.jpg?raw=true "Architecture")

### Results
  * Note: No example images of the two-stage architecture for LSUN bedrooms.
  * Using only the first stage of the architecture (first G and D) reduces the Inception score significantly.
  * Adding the text to both the first and second generator improves the Inception score slightly.
  * Adding the VAE-like text embedding generation (as opposed to only FC layers) improves the Inception score slightly.
  * Generating images at higher resolution (256x256 instead of 128x128) improves the Inception score significantly
    * Note: The 256x256 architecture has more residual convolutions than the 128x128 one.
    * Note: The 128x128 and the 256x256 are both upscaled to 299x299 images before computing the Inception score. That should make the 128x128 images quite blurry and hence of low quality.
  * Example images, with text and stage 1/2 results:
    * ![Examples](https://raw.githubusercontent.com/aleju/papers/master/neural-nets/images/StackGAN__examples.jpg?raw=true "Examples")
  * More examples of birds:
    * ![Examples birds](https://raw.githubusercontent.com/aleju/papers/master/neural-nets/images/StackGAN__examples_birds.jpg?raw=true "Examples birds")
  * Examples of failures:
    * ![Failure Cases](https://raw.githubusercontent.com/aleju/papers/master/neural-nets/images/StackGAN__failures.jpg?raw=true "Failure Cases")
    * The authors argue, that most failure cases happen when stage 1 messes up.

### **Keyword**: RNN, serialized model; non-differentiable backpropogarion; action detection in video 


**Abstract**: This paper uses an end-to-end model which is a recurrent neural network trained by REINFORCE to directly predict the temporal bounds of actions. The intuition is that people will observe moments in video and decide where to look to predict when an action is occurring. After training, Serena et al manage to achieve the state-of-art result by only observing 2% of the video frames.

**Model**: In order to take a long video and output all the instances of given action, they use two parts including an observation network and recurrent network. 
* observation network:  encode the visual representation of video frames.
     * input:  $ln$ -- the normalized location of the frame + frame $v_{ln}$
     * network: fc7 feature of finetuned VGG16 network
     * output: $on$ of 1024 dimension indicate time and frame feature 
     
* recurrent network: sequentially process the visual representations and decide where to watch next and whether to emit detection.
##### for each timestep:
     * input:  $on$ -- the representation of the frame + previous state $h_{n-1}$
     * network: $d_n = f_d(h_n; \theta_d)$. $pn = fp(h_n,\theta_p)$,$fd$ is fc. $fp$ is fc+sigmoid  
     * output: $d_n = (s_n,e_n,c_n )$as the candidate detection, where $s_n$,$e_n$ is the start and end of the detection, $c_n$ is confidence level;  $p_n$ whether $d_n$ is a valid detection. $l_{n+1}$ where to observe next. all the parameter falls in [0,1]
https://i.imgur.com/SeFianV.png


**Training**: in order to learn the supervision annotation in long videos and handle the non-differentiable components, authors use BP to train $d_n$ while use REINFORCE to train $p_n$ and $l_{n+1}$
* for $d_n$: $L(D) = \sum_n L_{cls}(d_n) + \sum_n \sum_m 1[y_{mn} = 1] L_{loc}(d_n,g_m)$
* for $p_n,l_{n+1}$: reward $J(\theta) = \sum_{a\in A} p(a) r(a)$ p(a) is the distribution of action and r(a) is the reward for the action. so training needs to maximize this.


**Summary**:
This paper uses a serialized model which first extract the feature from each frame, then use the frame feature and previous state info to generate the next observation time, detection and detection indicator. Specifically, in order to use previous information, they use RNN to store information and use REINFORCE to train $p_n$ and $l_n$, where the goal is to maximize reward for an action sequence and use Monte-Carlo sampling to numerically calculate the gradient for high dimension function.

**questions**:
1. why  $p_n$ and $l_n$ are non-differentiable components?
2. if $p_n$ and $l_n$ are non-differentiable components indeed, how do we come up with REINFORCE to compute the gradient?
3. why don't we get $p_n$ from $p_n = f_p(h_n, \theta_p)$ directly but rather use fp as the parameter in bernoulli distribution, similar question can be applied to calculation for $l_{n+1}$ in trainning time.

 
  

I enjoyed reading this paper. However I have a problem with the core statement of the problem which is described by the following sentence. 

"However, our visual experience is not at all like this. We experience the world as fully
detailed and there is currently no scientific explanation of this."

Who says we experience the world this way? I don't feel that I do at all. Let's say I'm looking at a large picture made of random pixels each of which is white or black at random. I first perceive the fact that I am looking at something with no real information. I know there are white and black pixels everywhere but I don't actually store which are white and which are black because I see no information in it and don't think it would be useful. I can focus on any small part of it and truly record what the arrangement is. If I close my eyes I can probably remember a 3x3 block accurately enough which is only 9 bits of information. If I practice for a long while, I could probably get a somewhat larger block. But I doubt I could get much higher than a 5x5 block of pixels. 

If I look at the scene around me right now, I see my familiar living room scene. It's not random like before. Again I can focus on any part of it, say a book on my bookshelf, and read the title. And there are lots of books. I can do the same for any of them but not all of them at once. I know they are have titles but I am under no illusion that I see them all at once and record the information. I know my eye and brain is not taking a 64 Megapixel picture every second like a video recorder. 

The other things in the room that show up in my peripheral vision can be summarized easily. I know there is a piano. It's large and brown and to the left side of the room. I know there are lamps and a beer glass. I can close my eyes and answer questions about the things in the room. I've compressed the useful information into a model and ignored the rest just as a JPEG or Autoencoder reduces the size of an image but keeps most of the relevant information. Perhaps my brain has approximately memorized the whole picture at low resolution. But I'm under no illusion that I'm really seeing it all at once at high resolution. 

Whether my eyes are open or closed I have this reduced model of what I'm looking at. I know from experience that there is detail everywhere without seeing it all at once. Wherever I focus my gaze I do see the detail and so I come to believe there is indeed detail everywhere even if I can't see and store all of it at once. 

The point is that there is no scientific problem here. Our brain very likely does something like what artificial neural networks do. They identify summarizing features which characterize most things at low resolution keeping at most only a small part of the field in high focus. Our brain being active can choose to query any other part of this to get higher resolution on that part. When it does it overwrites whatever it is that we were focusing on before. 

So in summary, I think the problem arises only from a confusion between "knowing that a picture has high-resolution information everywhere" and "actually recording that information". I certainly don't see any need to propose a supernatural explanation of vision. We're only beginning to understand how the brain and other artificial neural networks process information and in the latter we are often surprised how well they can do.