Role: Research and Development: Deep learning and Music
Work at Monash

I am currently working on making tools for improvising musicians that utilise deep learning. These tools are music generating systems that listen to, and react to improvisers in real-time. They are trained on performance data including audio recordings and symbolic representations of how and when physical instruments are used.

I will be at the NeurIPS workshop on Machine Learning for Creativity and Design, Dec 11 to introduce MANDI the Musically Attentive Neural Drum Improviser. MANDI uses at Temporal Convolutional Network to learn performance patterns observed in recorded improvisations between drummers and melodic instrumentalists. These patterns are then used to generate drum improvisations to play along with musicians in real-time. While not intended to perform like a human drummer would, the system produces musically appropriate virtual drum performances that reacts to improvisers and can compliment or surprise.

For the many hours a day that improvising musicians spend practising alone, these tools provide a means of challenging and supporting musicians with responsive behaviour.

Other research in this area is currently under review - but more details will appear here when it is possible.