Applying the mixture density recurrent neural network

People

Supervisor

Research areas

A simple MDN RNN architecture diagram

Description

The MDRNN is an exciting sequence model that can generate multiple continuous valued signals from a Gaussian mixture model at each step in time. This project will involve applying and extending my Keras MDRNN models into new applications in the creative arts and beyond. We’ve tried using the MDRNN for voice synthesis, motion capture data synthesis, and musical control data synthesis, but there are lots of other potential applications waiting for you to discover, e.g.: predicting future sensor values, generating robot movements, generating world models for video games or real life situations etc.

Goals

  • gain an understanding of mixture density networks and artificial neural network designs
  • apply the MDN to a new application area, this could be a creative arts application, sequence learning from real data, or some other application that you dream up!
  • develop your system and train with real-world data

Requirements

  • Python programming
  • completed coursework in machine learning or artificial intelligence
  • motivation to work on a new problem and obtain a dataset for training and testing

Background Literature

  1. Christopher M. Bishop. 1994. Mixture Density Networks. Technical Report NCRG/94/004. Neural Computing Research Group, Aston University. http://publications.aston.ac.uk/373/
  2. Axel Brando. 2017. Mixture Density Networks (MDN) for distribution and uncertainty estimation. Master’s thesis. Universitat Politècnica de Catalunya.
  3. A. Graves. 2013. Generating Sequences With Recurrent Neural Networks. ArXiv e-prints (Aug. 2013). https://arxiv.org/abs/1308.0850
  4. David Ha and Douglas Eck. 2017. A Neural Representation of Sketch Drawings. ArXiv e-prints (April 2017). https://arxiv.org/abs/1704.03477
  5. Charles P. Martin and Jim Torresen. 2018. RoboJam: A Musical Mixture Density Network for Collaborative Touchscreen Interaction. In Evolutionary and Biologically Inspired Music, Sound, Art and Design: EvoMUSART ’18, A. Liapis et al. (Ed.). Lecture Notes in Computer Science, Vol. 10783. Springer International Publishing. DOI:10.1007/9778-3-319-77583-8_11

Keywords

mixture density network, machine learning, applications

Updated:  1 June 2019/Responsible Officer:  Head of School/Page Contact:  CECS Marketing