Deep Reinforcement Learning with a Natural Language Action Space
Ji He
and
Jianshu Chen
and
Xiaodong He
and
Jianfeng Gao
and
Lihong Li
and
Li Deng
and
Mari Ostendorf
arXiv e-Print archive - 2015 via Local arXiv
Keywords:
cs.AI, cs.CL, cs.LG
First published: 2015/11/14 (8 years ago) Abstract: This paper introduces a novel architecture for reinforcement learning with
deep neural networks designed to handle state and action spaces characterized
by natural language, as found in text-based games. Termed a deep reinforcement
relevance network (DRRN), the architecture represents action and state spaces
with separate embedding vectors, which are combined with an interaction
function to approximate the Q-function in reinforcement learning. We evaluate
the DRRN on two popular text games, showing superior performance over other
deep Q-learning architectures. Experiments with paraphrased action descriptions
show that the model is extracting meaning rather than simply memorizing strings
of text.
TLDR; The authors train a DQN on text-based games. The main difference is that their Q-Value functions embeds the state (textual context) and action (text-based choice) separately and then takes the dot product between them. The authors call this a Deep Reinforcement Learning Relevance network. Basically, just a different Q function implementation. Empirically, the authors show that their network can learn to solve "Saving John" and "Machine of Death" text games.