Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
Ilya Kostrikov
and
Denis Yarats
and
Rob Fergus
arXiv e-Print archive - 2020 via Local arXiv
Keywords:
cs.LG, cs.CV, eess.IV, stat.ML
First published: 2020/04/28 (4 years ago) Abstract: We propose a simple data augmentation technique that can be applied to
standard model-free reinforcement learning algorithms, enabling robust learning
directly from pixels without the need for auxiliary losses or pre-training. The
approach leverages input perturbations commonly used in computer vision tasks
to regularize the value function. Existing model-free approaches, such as Soft
Actor-Critic (SAC), are not able to train deep networks effectively from image
pixels. However, the addition of our augmentation method dramatically improves
SAC's performance, enabling it to reach state-of-the-art performance on the
DeepMind control suite, surpassing model-based (Dreamer, PlaNet, and SLAC)
methods and recently proposed contrastive learning (CURL). Our approach can be
combined with any model-free reinforcement learning algorithm, requiring only
minor modifications. An implementation can be found at
https://sites.google.com/view/data-regularized-q.