Neural Machine Translation with Recurrent Attention Modeling
Zichao Yang
and
Zhiting Hu
and
Yuntian Deng
and
Chris Dyer
and
Alex Smola
arXiv e-Print archive - 2016 via Local arXiv
Keywords:
cs.NE, cs.CL
First published: 2016/07/18 (8 years ago) Abstract: Knowing which words have been attended to in previous time steps while
generating a translation is a rich source of information for predicting what
words will be attended to in the future. We improve upon the attention model of
Bahdanau et al. (2014) by explicitly modeling the relationship between previous
and subsequent attention levels for each word using one recurrent network per
input word. This architecture easily captures informative features, such as
fertility and regularities in relative distortion. In experiments, we show our
parameterization of attention improves translation quality.
