Seminar: Probabilistic Programming and the Challenge of Gradient Variance

Monday, November 19, 2018

MC 331
5 King's College Road

Abstract: Probabilistic models are used from psychology to biomedicine and ecology. The dream is that a user can simply describe a model, and get analysis without worrying about algorithms. Recently, a turn towards stochasticity has allowed variational inference (VI) to address larger-scale data. The fundamental idea is to reduce inference to the problem of repeatedly estimating stochastic gradients.  However, there are many alternative methods for stochastic VI and significant algorithmic engineering is often needed in practice.


This largely tutorial talk will discuss why you might care about probabilistic inference, when VI might be a good approach, and the critical role of gradient variance in recent methods. We will see that there is a huge design space for gradient estimators, with no single “best” one. Towards the end of the talk I will briefly describe some recent research attempting to automatically find the best combination of a set of candidate estimators.


Bio: Justin Domke received a Ph.D. from the University of Maryland, College Park, and worked at RIT (In Rochester, New York) and National ICT Australia (in Canberra and Sydney) before coming to Massachusetts. Since Fall 2016 he is an assistant professor in Computer Science at the University of Massachusetts, Amherst.


© 2021 Faculty of Applied Science & Engineering