Bayesian Nonparametric Structured Prediction

Trung Nguyen

CS HDR MONITORING

DATE: 2012-11-02
TIME: 15:10:00 - 15:40:00
LOCATION: NICTA - 7 London Circuit
CONTACT: JavaScript must be enabled to display this email address.

ABSTRACT:
Structured prediction is a framework for solving problems and learning tasks whose correlations make them more complicated but also present the opportunity for doing better than considering each of them independently. It has been applied in a wide variety of applications including but not limited to geostatistics, computational biology, dynamical systems, and consumer preferences. My work focuses on one instance of structured prediction namely multi-task outputs, which is also known as multivariate regression in the statistics literature. In particular, I study variational inference for the Gaussian Process Regression Networks (GPRN) which is an adaptive network that models output correlations with latent functions generated from Gaussian Processes. I proposed two family of distributions to approximate the intractable posteriors with attractive learning properties. In the first family, mean-field, the posterior is approximated by a factorised distribution where each factor is a Gaussian distribution. In the second family, nonparametric variational inference, the posterior is approximated by a mixture of Gaussians. I showed that for both families, with a particular factorisation form of the approximation, the evidence lower bound is analytically tractable which is not achievable in general. Furthermore the number of variational parameters in mean-field can be reduce to O(N) instead of O(N^2) as believed hence making computation more feasible. The empirical results demonstrate that NPV, although more complex, generalises better in terms of both prediction and variability.
BIO: