Conference Proceedings

Information-theoretic analysis for transfer learning

X Wu, JH Manton, U Aickelin, J Zhu

IEEE International Symposium on Information Theory - Proceedings | IEEE | Published : 2020


Transfer learning, or domain adaptation, is concerned with machine learning problems in which training and testing data come from possibly different distributions (denoted as μ and μ', respectively). In this work, we give an informationtheoretic analysis on the generalization error and the excess risk of transfer learning algorithms, following a line of work initiated by Russo and Zhou. Our results suggest, perhaps as expected, that the Kullback-Leibler (KL) divergence D(μμ') plays an important role in characterizing the generalization error in the settings of domain adaptation. Specifically, we provide generalization error upper bounds for general transfer learning algorithms, and extend th..

View full abstract