Early Stopping without a Validation Set
Maren Mahsereci
and
Lukas Balles
and
Christoph Lassner
and
Philipp Hennig
arXiv e-Print archive - 2017 via Local arXiv
Keywords:
cs.LG, stat.ML
First published: 2017/03/28 (7 years ago) Abstract: Early stopping is a widely used technique to prevent poor generalization
performance when training an over-expressive model by means of gradient-based
optimization. To find a good point to halt the optimizer, a common practice is
to split the dataset into a training and a smaller validation set to obtain an
ongoing estimate of the generalization performance. In this paper we propose a
novel early stopping criterion which is based on fast-to-compute, local
statistics of the computed gradients and entirely removes the need for a
held-out validation set. Our experiments show that this is a viable approach in
the setting of least-squares and logistic regression as well as neural
networks.
