Seminar:
Offline Learning With Function Approximation: Theory and Algorithms

ABSTRACT

Reinforcement learning has achieved remarkable empirical success in a wide range of challenging tasks, yet it often requires extensive online interaction, such as game-play with other expert players or some form of self-play. Such online interaction may not be feasible in many real-world scenarios due to concerns about cost, safety, and ethics (e.g., healthcare and autonomous driving). Offline learning is an alternative learning paradigm that allows learning from pre-collected datasets, with little to no online interaction. In this talk, we will focus on the theoretical and algorithmic foundations of offline learning, with an emphasis on problems involving large state spaces that require function approximation (e.g., deep neural networks, transformers) for generalization. We will discuss the interplay between distribution shift—induced by the constraint of learning from offline data—and function approximation, in characterizing the statistical feasibility and efficiency of offline learning. We will explore a generic algorithmic principle that unifies existing algorithms, facilitates the design of novel ones, and achieves state-of-the-art learning guarantees under a wide and novel regime of distribution shift. We will also discuss connections and extensions to other modern learning paradigms, such as hybrid learning and transfer learning.

Thanh Nguyen

Johns Hopkins University

Thanh Nguyen-Tang is a postdoctoral researcher in the Department of Computer Science at Johns Hopkins University, where he works with Raman Arora. He is broadly interested in the theoretical and algorithmic foundations of machine learning for modern artificial intelligence (AI), with current research spanning reinforcement learning, transfer learning, multi-agent learning, trustworthy AI, and large language models. He is an area chair for AISTATS 2025 and a senior program committee member for AAAI 2025, 2024, 2023. He completed his PhD at the Applied Artificial Intelligence Institute, Deakin University, Australia, where he was awarded the Alfred Deakin Medal for the most outstanding thesis.