Data Noising as Smoothing in Neural Network Language Models on ShortScience.org

arxiv.org
scholar.google.com

Data Noising as Smoothing in Neural Network Language Models
Xie, Ziang and Wang, Sida I. and Li, Jiwei and Lévy, Daniel and Nie, Aiming and Jurafsky, Dan and Ng, Andrew Y.
arXiv e-Print archive - 2017 via Local Bibsonomy
Keywords: dblp

Summaries/Notes 1

[link] Summary by Marek Rei 6 years ago

The paper investigates better noising techniques for RNN language models.

https://i.imgur.com/cq5Kb0Y.png

A noising technique from previous work would be to randomly replace words in the context or replace them with a blank token. Here they investigate ways of choosing better which words to replace and choosing the replacements from a better distribution, inspired by methods in n-gram smoothing. They show improvement on language modeling (PTB and text8) and machine translation (English-German).

Your comment:

Write your summary here (You can use $\LaTeX$ and markdown syntax):

Anon Private