AI, ML and Friends is a weekly seminar series within the School of Computing on Artificial Intelligence, Machine Learning, and related topics. We are open to attendees and presenters external to the school. Please sign up to the mailing list to receive weekly announcements including zoom details, and email the seminar organiser to schedule a talk.

Upcoming Seminars #

11 December 2023, 11:00 #

Codec Avatars for Social Telepresence #

Speaker: Dr. Jason Saragih

Abstract: Social Telepresence bears promise for overcoming “the tyranny of distance” by facilitating interactions that are indistinguishable from those in-person. Recent advances in virtual- and augmented-reality technologies have unlocked a key component of face-to-face interaction, the perception of a shared 3D space. Additionally, internet infrastructure now supports the high-bandwidth payloads required for immersive content. However, existing VR systems have so far failed to provide a compelling alternative to in-person meetings, or even replace video conferencing as a preferred medium for long-distance communication. A key challenge has been the accurate reconstruction of human motion and appearance. In particular, difficulties in overcoming the “uncanny valley” have pushed existing systems towards stylized representations, limiting their application to gaming and entertainment. In this talk, we will present Codec Avatars; a neural representation of humans that can accurately capture an individual’s appearance and faithfully reproduce social signals from limited sensors mounted on VR headsets. It marries advances in deep learning, neural rendering, and the development of specialized multi-view capture systems to bridge the data-gap required for enabling high-fidelity modelling. Their development at Meta Reality Labs over the past five years will be elaborated, referencing publications in the computer vision and graphics communities. In particular, the system that enabled the first interview in the Metaverse with photorealistic avatars will be described.

Bio: Dr. Jason Saragih is a Senior Director of Research Science with Meta Reality Labs (Facebook) in Pittsburgh, USA. His research focuses on neural modelling of human appearance and motion for enabling social interactions in AR/VR. Jason has made major contributions to computer vision in AR/VR applications. He is one of the senior contributors to the work of “neural volumes” which inspired the later development of NerF (Neural Radiance Fields) technique. He is also a co-author on the Coded-Avatar work—i.e., the technique used in Mark Zuckerberg’s first interview in the Metaverse with Lex Fridman. He obtained his PhD degree in 2008, from the RSISE of Australian National University, where he worked with Roland Goecke, Richard Hartley, and Hongdong Li etc. He completed a postdoctoral fellowship in 2010 at Carnegie Mellon University, Pittsburgh. Before joining Meta, Jason founded a startup which developed custom computer vision solutions specializing in cellphone-based AR for industry clients, including Luxottica and Adobe.

Where: Building 145, Room 3.41