MIE Distinguished Seminar Series: Explaining Markov Decision Processes

Friday, January 12, 2018

MC 102
5 King's College Road


Speaker: Pascal Poupart
Affiliation: University of Waterloo


In many situations, a sequence of decisions must be taken by an individual or system (e.g., course selection by students, inspection of parts for testing in a factory, etc.). However, deciding on a course of action is notoriously difficult when there is uncertainty in the effects of the actions and the objectives are complex. Markov decision processes (MDPs) provide a principled approach for automated planning under uncertainty. While the beauty of an automated approach is that the computational power of machines can be harnessed to optimize difficult sequential decision making tasks, the drawback is that users may not understand why certain actions are recommended. This lack of understanding is a serious bottleneck that is currently holding back the widespread use of automated tools such as MDPs in recommender systems. Hence, there is a need for explanations that enhance the user’s understanding and trust of these recommendations.

In this talk, I will present a generic technique to explain policies in arbitrary domains where the sequential decision making problem is formulated as a factored Markov decision process. The explanations consist of template sentences that are filled with relevant information to justify why some action was recommended in a given state. I will describe a mechanism to determine a minimal set of templates that are sufficient to completely justify the action choice. The approach will be demonstrated and evaluated with a user study in the context of advising undergraduate students in their course selection.


Pascal Poupart is a Full Professor in the David R. Cheriton School of Computer Science at the University of Waterloo, Waterloo (Canada).  He received the B.Sc. in Mathematics and Computer Science at McGill University, Montreal (Canada) in 1998, the M.Sc. in Computer Science at the University of British Columbia, Vancouver (Canada) in 2000 and the Ph.D. in Computer Science at the University of Toronto, Toronto (Canada) in 2005.  His research focuses on the development of algorithms for reasoning under uncertainty and machine learning with application to Assistive Technologies, Natural Language Processing ad Telecommunication Networks.  He is most well known for his contributions to the development of approximate scalable algorithms for partially observable Markov decision processes (POMDPs) and their applications in real-world problems, including automated prompting for people with dementia for the task of handwashing and spoken dialog management. Other notable projects that his research team are currently working on include deep learning with clear semantics, structure learning, personalized transfer learning, conversational agents, adaptive satisfiability, sports analytics and stress detection based on wearable devices.

Pascal Poupart received a Cheriton Faculty Fellowship (2015-2018), a best student paper honourable mention (SAT-2017), an outstanding collaborator award from Huawei Noah’s Ark (2016), a top reviewer award (ICML-2016), the best main track solver and best application solver (SAT-2016 competition), a best reviewer award (NIPS-2015), an Early Researcher Award from the Ontario Ministry of Research and Innovation (2008), two Google reserch awards (2007-2008), a best paper award runner up (UAI-2008) and the IAPR best paper award (ICVS-2007).  He serves as associate editor of the Journal of Artificial Intelligence Research (JAIR) (2017 – present), member of the editorial board of the Journal of Machine Learning Research (JMLR) (2009 – present) and guest editor for the Machine Learning Journal (MLJ) (2012 – present).  He routinely serves as area chair or senior program committee member for NIPS, ICML, AISTATS, IJCAI, AAAI and UAI.  He serves as scientific advisor for Huawei Technologies, ElementAI, TalkIQ and ProNavigator. His research collaborators include Huawei Technologies, Google, Intel, Kik Interactive, In the Chat, Slyce, HockeyTech, ProNavigator, the Alzheimer Association, the UW-Schlegel Research Institute for Aging, Sunnybrook Health Science Centre and the Toronto Rehabilitation Institute.

© 2024 Faculty of Applied Science & Engineering