Australian Journal of Intelligent Information Processing Systems, Vol 11, No 4 (2010): Adaptive Algorithms

Font Size:  Small  Medium  Large

Phase Transition of Variational Bayes Learning in Bernoulli Mixture

Daisuke Kaj, Kazuho Watanabe, Sumio Watanabe


The variational Bayes learning is widely used, because it provides both computational tractability and good generalization performance. However its theoretical properties, for example, the effect of hyperparameters to the learning result, has not yet been clarified. In this paper, we prove the asymptotic behavior of the variational free energy in a Bernoulli mixture model and show that the variational posterior distribution has phase transition according to the hyperparameters. We also discuss the design method of hyperparameters based on theoretical foundation.

Full Text: PDF