Revisiting Probability Distribution Assumptions for Information Theoretic Feature Selection
Yuan Sun, Wei Wang, Michael Kirley, Xiaodong Li, Jeffrey Chan
THIRTY-FOURTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, THE THIRTY-SECOND INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE AND THE TENTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE | ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE | Published : 2020
Feature selection has been shown to be beneficial for many data mining and machine learning tasks, especially for big data analytics. Mutual Information (MI) is a well-known information-theoretic approach used to evaluate the relevance of feature subsets and class labels. However, estimating high-dimensional MI poses significant challenges. Consequently, a great deal of research has focused on using low-order MI approximations or computing a lower bound on MI called Variational Information (VI). These methods often require certain assumptions made on the probability distributions of features such that these distributions are realistic yet tractable to compute. In this paper, we reveal two se..View full abstract
Awarded by Australian Research Council
This work was partially supported by an ARC Discovery Grant (DP180101170) from Australian Research Council.