Parallel inference for big data with the group Bayesian method

Abstract

In recent years, big datasets are often split into several subsets due to the storage requirements. We propose a parallel group Bayesian method for statistical inference in sparse big data. This method improves the existing methods in two aspects: the total datasets are also split into a data subset sequence and the parameter vector is divided into several sub-vectors. Besides, we add a weight sequence to optimize the sub-estimators when each of them has a different covariance matrix. We obtain several theoretical properties of the estimator. The results of numerical simulations show that our method is consistent with the theoretical results and is more effective than classic Markov chain Mo..