Dimensionality-Driven Learning with Noisy Labels
X Ma, Yisen Wang, Michael E Houle, Shuo Zhou, Sarah M Erfani, Shu-tao Xia, Sudanthi Wijewickrema, James Bailey
35th International Conference on Machine Learning, ICML 2018 | JMLR | Published : 2018
Datasets with significant proportions of noisy (incorrect) class labels present challenges for training accurate Deep Neural Networks (DNNs). We propose a new perspective for understanding DNN generalization for such datasets, by investigating the dimensionality of the deep representation subspace of training samples. We show that from a dimensionality perspective, DNNs exhibit quite distinctive learning styles when trained with clean labels versus when trained with a proportion of noisy labels. Based on this finding, we develop a new dimensionality-driven learning strategy, which monitors the dimensionality of subspaces during training and adapts the loss function accordingly. We empirica..View full abstract
Awarded by Australian Research Council
James Bailey is in part supported by the Australian Research Council via grant number DP170102472. Michael E. Houle is partially supported by JSPS Kakenhi Kiban (B) Research Grants 15H02753 and 18H03296. Shu-Tao Xia is partially supported by the National Natural Science Foundation of China under grant No. 61771273.