DENSITY-ESTIMATION BY STOCHASTIC COMPLEXITY

Abstract

The results by Hall and Hannan on optimization of histogram density estimators with equal bin widths by minimization of the stochastic complexity, are extended and sharpened in two separate ways. As the first contribution, two generalized histogram estimators are constructed. The first has unequal bin widths which, together with the number of the bins, are determined by minimization of the stochastic complexity with help of dynamic programming. The other estimator consists of a mixture of equal bin width estimators, each of which is defined by the associated stochastic complexity. As the main contribution in this paper, two theorems are proved, which together extend the universal coding theo..