Unlocking the Potential of Sequential Decision-Making: From Theory to Applications

Speaker: Johannes Kirschner - Swiss Data Science Center (SDSC)

Abstract:
Sequential decision-making is a machine learning framework that captures a variety of settings of practical interest, from gameplay, molecular design, algorithm discovery, and control of complex physical systems. Unlike in classical supervised and unsupervised learning, the algorithm's decisions impact the future training data. This provides a substantial opportunity for accelerated learning, targeted exploration and experimentation, and going beyond what is possible using only static data sets. One major challenge is the exploration-exploitation trade-off: How can we balance the cost of acquiring data and the value it provides for achieving a target objective? In this talk, I will briefly review recent advances in the mathematical foundations of sequential decision-making. I will introduce information-directed sampling (IDS), a principled and versatile algorithm for sequential decision-making that directly optimizes the underlying statistical information trade-off. In particular, IDS is the first algorithm to achieve near-optimal instance and worst-case performance guarantees in a wide range of settings - all within a single and practical framework. I will then demonstrate the potential of sequential decision-making in two scientific applications: Tuning particle accelerators using safe Bayesian optimization, and data-driven experimental design for computed X-ray tomography via Diffusion Active Learning. I will conclude the talk by outlining how to combine sequential decision-making with large-scale learning as an exciting avenue for future research.

Biography:
Johannes Kirschner is a Senior Data Scientist in the Academic Team of the Swiss Data Science Center (SDSC), where he develops state-of-the-art machine learning algorithms for challenging scientific applications. Before joining the SDSC, he completed a postdoc with Csaba Szepesvári at the University of Alberta (supported by an Early Postdoc Mobility fellowship of the Swiss National Foundation). He earned his PhD in 2016 at ETH Zurich with Prof. Andreas Krause. Dr. Kirschner's research focuses on developing practical and principled sequential decision-making algorithms, including reinforcement learning, experimental design, and Bayesian optimization. His work is widely recognized at top venues (NeurIPS, ICML, COLT ...) and spans mathematical foundations to impactful real-world applications. He actively serves as a reviewer at major machine learning conferences and journals, co-organizes the popular reinforcement learning theory online seminars and an ICML workshop on RL and Control Theory, and acted as associate chair for ICML 2022.

Host: Prof. Laura Pozzi