ROBUST TRAINING METHODS FOR DEEP NEURAL NETWORKS WITH A VARIETY OF LABEL NOISE
Abstract
Requirement of significant amount of labeled training data is a major drawback in training deep neural networks (DNNs), due to the presence of mislabeled examples in these
datasets. This label noise is shown to have an adverse effect on the generalization performance of DNNs. Thus, reducing the consequences of label noise is of much research value.
In this dissertation, we focus on improving our understanding of label noise and, achieving
better generalization performance. Due to the lack of ground truth with real world noisy
datasets, most researchers create synthetic noisy datasets to develop robust training methods. Among these methods, stopping the training in the early stages is shown to achieve
better generalization performance with label noise. However, identifying such training stop
point without ground truth is a demanding problem. We propose novel training methods to
identify training stop point when noise rate is known and unknown. We further identify that
the significance of stopping the training in the early stages and the effectiveness of several
existing training methods are reduced with complex label noisy datasets. Thus, complex realistic noisy datasets that additionally provide ground truth are necessary to study and
develop robust training methods. Therefore, we propose novel Pseudo noisy datasets that
resemble complex noisy datasets.