MML clustering of multi-state, Poisson, vonMises circular and Gaussian distributions

  • Authors:
  • Chris S. Wallace;David L. Dowe

  • Affiliations:
  • Computer Science and Software Engineering, Monash University, Clayton, Vic. 3168, Australia. csw@cs.monash.edu.au;Computer Science and Software Engineering, Monash University, Clayton, Vic. 3168, Australia. dld@cs.monash.edu.au

  • Venue:
  • Statistics and Computing
  • Year:
  • 2000

Quantified Score

Hi-index 0.00

Visualization

Abstract

Minimum Message Length (MML) is an invariantBayesian point estimation technique which is also statisticallyconsistent and efficient. We provide a brief overview of MMLinductive inference (Wallace C.S. and Boulton D.M. 1968. ComputerJournal, 11: 185–194; Wallace C.S. and FreemanP.R. 1987. J. RoyalStatistical Society (Series B), 49: 240–252; WallaceC.S. and DoweD.L. (1999). Computer Journal), and how it has both aninformation-theoretic and a Bayesian interpretation. We then outlinehow MML is used for statistical parameter estimation, and how the MMLmixture modelling program, Snob (Wallace C.S. and Boulton D.M. 1968.Computer Journal, 11: 185–194; Wallace C.S. 1986.In: Proceedings ofthe Nineteenth Australian Computer Science Conference (ACSC-9), Vol.8, Monash University, Australia, pp. 357–366; Wallace C.S.and Dowe D.L. 1994b.In: Zhang C. et al. (Eds.), Proc. 7th AustralianJoint Conf. on Artif.Intelligence. World Scientific, Singapore, pp. 37–44. Seehttp://www.csse.monash.edu.au/-dld/Snob.html) uses the messagelengths from various parameter estimates to enable it to combineparameter estimation with selection of the number of components andestimation of the relative abundances of the components. The messagelength is (to within a constant) the logarithm of the posteriorprobability (not a posterior density) of the theory. So, theMML theory can also be regarded as the theory with the highestposterior probability. Snob currently assumes that variables areuncorrelated within each component, and permits multi-variate datafrom Gaussian, discrete multi-category (or multi-state ormultinomial), Poisson and von Mises circular distributions, as wellas missing data. Additionally, Snob can do fully-parameterisedmixture modelling, estimating the latent class assignments inaddition to estimating the number of components, the relativeabundances of the parameters and the component parameters. We alsoreport on extensions of Snob for data which has sequential or spatialcorrelations between observations, or correlations betweenattributes.