Conference Proceedings
Cross-domain Feature Selection for Language Identification
M Lui, T Baldwin
Ijcnlp 2011 Proceedings of the 5th International Joint Conference on Natural Language Processing | IJCNLP2011 | Published : 2011
Abstract
We show that transductive (cross-domain) learning is an important consideration in building a general-purpose language identification system, and develop a feature selection method that generalizes across domains. Our results demonstrate that our method provides improvements in transductive transfer learning for language identification. We provide an implementation of the method and show that our system is faster than popular standalone language identification systems, while maintaining competitive accuracy.