Not so naive Bayesian classification
Geoff Webb (Monash University)
NICTA SML SEMINARDATE: 2011-11-11
TIME: 11:00:00 - 12:00:00
LOCATION: NICTA - 7 London Circuit
CONTACT: JavaScript must be enabled to display this email address.
ABSTRACT:
Machine learning is classically conceived as search through a hypothesis space for a hypothesis that best fits the training data. In contrast, naive Bayes performs no search, extrapolating an estimate of a high-order conditional probability by composition from lower-order conditional probabilities. In this talk I show how this searchless approach can be generalised, by systematically relaxing naive Bayes' conditional independence assumption. The resulting family of learners provide a principled method for controlling the bias/variance trade-off. At one extreme very low variance can be achieved as appropriate for small data. Bias can be decreased for larger data in a manner that ensures Bayes optimal asymptotic error. These algorithms have the desirable properties of - training time that is linear with respect to training set size, - learning from a single pass through the data, - allowing incremental learning, - supporting parallel and anytime classification, - providing direct prediction of class probabilities, - supporting direct handling of missing values, and - robust handling of noise. Despite being generative, they deliver classification accuracy competitive with state-of-the-art discriminative techniques.
BIO:
Geoff Webb is Professor of Information Technology Research in the Faculty of Information Technology at Monash University, where he heads the Centre for Research in Intelligent Systems. Prior to Monash he held appointments at Griffith University and then Deakin University, where he received a personal chair. His primary research areas are machine learning, data mining, and user modelling. He has published 12 papers in the prestigious Machine Learning journal, including the most highly cited paper published in the journal in 2005. He is known for his contribution to the debate about the application of Occam's razor in machine learning and for the development of numerous methods, algorithms and techniques for machine learning, data mining and user modelling. His commercial data mining software, Magnum Opus, incorporates many techniques from his association discovery research. Many of his learning algorithms are included in the widely-used Weka machine learning workbench. He is editor-in-chief of Data Mining and Knowledge Discovery, co-editor of the Springer Encyclopedia of Machine Learning, a member of the advisory board of Statistical Analysis and Data Mining and a member of the editorial boards of Machine Learning and ACM Transactions on Knowledge Discovery from Data.
