Lunjun Zhang

I am a CS PhD student in the Machine Learning Group at University of Toronto, and a Student Researcher at Google DeepMind working on large language models.

Previously, I spent two and a half years (2021-2024) working on autonomous driving at Waabi.

Before that, I was an undergraduate student in Engineering Science at University of Toronto (2017-2021), and interned at Vector Institute, Mila, and Uber Advanced Technologies Group.

Contact: Email / Google Scholar / Github / Twitter

Research

I work on unsupervised learning and reinforcement learning.

I am fascinated by the following questions:

How can AI agents autonomously learn from experience in open-ended environments without external supervision?
What mechanisms of recursive self improvement can lead to unbounded growth in model capabilities?
Can reinforcement learning shift the scaling laws of generative models to a hard takeoff?

Selected Publications

Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion

Lunjun Zhang, Yuwen Xiong, Ze Yang, Sergio Casas, Rui Hu, Raquel Urtasun

International Conference on Learning Representations (ICLR), 2024

[Paper] [Proceedings] [Poster] [Website]

A foundation model for self-driving that explicitly reasons in both 3D space and time.

Towards Unsupervised Object Detection from LiDAR Point Clouds

Lunjun Zhang, Anqi Joyce Yang, Yuwen Xiong, Sergio Casas, Bin Yang, Mengye Ren, Raquel Urtasun

Conference on Computer Vision and Pattern Recognition (CVPR), 2023

[Paper] [Proceedings] [Poster] [Website]

Self-supervised, scalable object discovery in the wild.

World Model as a Graph: Learning Latent Landmarks for Planning

Lunjun Zhang, Ge Yang, Bradly Stadie

International Conference on Machine Learning (ICML), 2021 (Long Talk)

[Paper] [Proceedings] [Poster] [Website] [Code]

Unsupervised long-horizon planning via graph-structured world models.