Abstract
Mean field methods provide computationally efficient approximations to posterior probability distributions for graphical models. Simple mean field methods make a completely factorized approximation to the posterior, which is unlikely to be accurate when the posterior is multimodal. Indeed, if the posterior is multi-modal, only one of the modes can be captured. To improve the mean field approximation in such cases, we employ mixture models as posterior approximations, where each mixture component is a factorized distribution. We describe efficient methods for optimizing the Parameters in these models.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bishop, C., Lawrence, N., Jaakkola, T. S., & Jordan, M. I. Approximating posterior distributions in belief networks using mixtures. In M. I. Jordan, M. J. Kearns, & S. A. Solla, Advances in Neural Information Processing Systems 10, MIT Press, Cambridge MA (1998).
Ghahramani, Z. & Jordan, M. I. Factorial Hidden Markov models. In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Advances in Neural Information Processing Systems 8, MIT Press, Cambridge MA (1996).
Neal, R. Connectionist learning of belief networks, Artificial Intelligence, 56: 71–113 (1992).
Parisi, G. Statistical Field Theory. Addison-Wesley: Redwood City (1988).
Peterson, C. & Anderson, J. R. A mean field theory learning algorithm for neural networks. Complex Systems1: 995–1019 (1987).
Saul, L. K., Jaakkola, T. S., & Jordan, M. I. Mean field theory for sigmoid belief networks. Journal of Artificial Intelligence Research, 4: 61–76 (1996).
Saul, L. K. & Jordan, M. I. Exploiting tractable substructures in intractable networks. In D. S. Touretzky, М. C. Mozer, & M. E. Hasselmo (Eds.), Advances in Neural Information Processing Systems 8, MIT Press, Cambridge MA (1996).
Shwe, М. A., Middleton, B., Beckerman, D. E., Henrion, M., Horvitz, E. J., Lehmann, H. P., & Cooper, G. F. Probabilistic diagnosis using a reformulation of the INTERNIST1/QMR knowledge base. Meth. Inform. Med. 30: 241–255 (1991).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Jaakkola, T.S., Jordan, M.I. (1998). Improving the Mean Field Approximation Via the Use of Mixture Distributions. In: Jordan, M.I. (eds) Learning in Graphical Models. NATO ASI Series, vol 89. Springer, Dordrecht. https://doi.org/10.1007/978-94-011-5014-9_6
Download citation
DOI: https://doi.org/10.1007/978-94-011-5014-9_6
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-6104-9
Online ISBN: 978-94-011-5014-9
eBook Packages: Springer Book Archive