Seminar: Probabilistic Programming and the Challenge of Gradient Variance - Department of Mechanical & Industrial Engineering

Monday, November 19, 2018
12:00pm-1:00pm

MC 331
5 King's College Road

Abstract: Probabilistic models are used from psychology to biomedicine and ecology. The dream is that a user can simply describe a model, and get analysis without worrying about algorithms. Recently, a turn towards stochasticity has allowed variational inference (VI) to address larger-scale data. The fundamental idea is to reduce inference to the problem of repeatedly estimating stochastic gradients. However, there are many alternative methods for stochastic VI and significant algorithmic engineering is often needed in practice.

This largely tutorial talk will discuss why you might care about probabilistic inference, when VI might be a good approach, and the critical role of gradient variance in recent methods. We will see that there is a huge design space for gradient estimators, with no single “best” one. Towards the end of the talk I will briefly describe some recent research attempting to automatically find the best combination of a set of candidate estimators.

Bio: Justin Domke received a Ph.D. from the University of Maryland, College Park, and worked at RIT (In Rochester, New York) and National ICT Australia (in Canberra and Sydney) before coming to Massachusetts. Since Fall 2016 he is an assistant professor in Computer Science at the University of Massachusetts, Amherst.

Traditional Land Acknowledgement

We wish to acknowledge this land on which the University of Toronto operates. For thousands of years it has been the traditional land of the Huron-Wendat, the Seneca, and the Mississaugas of the Credit. Today, this meeting place is still the home to many Indigenous people from across Turtle Island and we are grateful to have the opportunity to work on this land.