Conference Proceedings

Forget Me Not: Reducing Catastrophic Forgetting for Domain Adaptation in Reading Comprehension

Y Xu, X Zhong, AJJ Yepes, JH Lau

2020 International Joint Conference on Neural Networks (IJCNN) | IEEE | Published : 2020


The creation of large-scale open domain reading comprehension data sets in recent years has enabled the development of end-to-end neural comprehension models with promising results. To use these models for domains with limited training data, one of the most effective approach is to first pre-train them on large out-of-domain source data and then fine-tune them with the limited target data. The caveat of this is that after fine-tuning the comprehension models tend to perform poorly in the source domain, a phenomenon known as catastrophic forgetting. In this paper, we explore methods that reduce catastrophic forgetting during fine-tuning without assuming access to data from the source domain. ..

View full abstract