Xie et al. Propose to regularize deep neural networks by randomly disturbing (i.e., changing) training labels.  In particular, for each training batch, they randomly change the label of each sample with probability $\alpha$ - when changing a label, it’s sampled uniformly from the set of labels. In experiments, the authors show that this sort of loss regularization improves generalization. However, Dropout usually performs better; in their case, only the combination with leads to noticable improvements on MNIST and SVHN – and only compared to no regularization and data augmentation at all. In their discussion, they offer two interpretations of dropping labels. First, it canbe seen as learning an ensemble of models on different noisy label sets; second, it can be seen as implicitly performing data augmentation. Both interepretation area reasonable, but do not provide a definite answer to why disturbing training labels should work well.

https://i.imgur.com/KH36sAM.png
Figure 1: Comparison of training testing error rate during training for no regularization, dropout and DropLabel.

Also find this summary at [davidstutz.de](https://davidstutz.de/category/reading/).