Stochastic complexity and model selection from incomplete data

Abstract

The principle of minimum description length (MDL) provides an approach for selecting the model class with the smallest stochastic complexity of the data among a set of model classes. However, when only incomplete data are available the stochastic complexity for the complete data cannot be numerically computed. In this paper, this problem is solved by introducing a notion of expected stochastic complexity for the complete data conditional on the observed data, which can be computed by the EM algorithm. Based on this notion, model selection from incomplete data can also be performed by the MDL principle. A simulation study is presented for illustration of the methodology.