A Practical Guide to Radiance Fields for XR Research and Applications

IEEE VR 2025 Tutorial on March 8, 2025, Vauban 1

shohei portrait
University of Stuttgart
ke portrait
University of Hamburg
mana.jpg
Keio University

The advent of photorealistic scene representations such as Neural Radiance Fields (NeRF) and 3D Gaussian Splattings (3DGS) enable us to replicate complex real-world elements. This paradigm shift in graphics lets us revisit the way of capturing, storing, and replicating the real world and rethink what to bring in XR (X for any type).

This tutorial provides an opportunity for learning the basics to "start with" future XR research around the topic. Namely, the attendees will learn the principles of view synthesis using machine learning technology and practical aspects of implementations using open-source projects. It also provides discussions and networking with VR attendees who share the same interests.

The tutorial will begin with a general overview, summarizing the latest developments and current trends in radiance field research for XR applications. This will be followed by an in-depth, live demonstration and tutorial on creating basic NeRF and 3DGS scenes for VR using readily available open-source software such as immersive-ngp and UnityGaussianSplatting toolkit. To conclude, we will summarize the key takeaways and facilitate an interactive discussion with the audience, exploring future trends and developments in radiance fields for XR.

PROGRAM
08:30 - 08:40

Opening and Welcoming

08:40 - 09:00

Introduction to Radiance Fields for XR (PDF)

Shohei Mori presents the basics of NeRF and 3DGS, and how they have evolved view synthesis compared to conventional image-based rendering approaches.

09:00 - 09:25

Live Tutorial on NeRF for VR (PDF)

Ke Li presents details of NeRF and immersive-ngp , demonstrating how to get a NeRF model in Unity and VR.

09:25 - 09:50

Live Tutorial on 3DGS for VR (PDF)

Mana Masuda presents details of 3DGS and UnityGaussianSplatting , demonstrating how to get 3DGS in Unity and VR.

09:50 - 10:15

Interactive Discussion - Future Challenges in XR

Shohei Mori gives a summary and shares challenges for future XR research using NeRF and 3DGS.

We also welcome the following three short presentations from the main conference:

  • "Semantics-Controlled Gaussian Splatting for Outdoor Scene Reconstruction and Rendering in Virtual Reality", Hannah Schieber, TUM (PDF)
  • "Gaussian Splatting for Immersive Anatomy Visualization", Constantin Kleinbeck, TUM, (Project Page, PDF)
  • "DimSplat: A Real-Time Diminished Reality System for Revisiting Environments Using Gaussian Splats in Mobile WebXR", Kristoffer Waldow, TH Köln (PDF)

We welcome all attendees to an interactive discussion and exchange.

MEMORIAL PHOTOS

Acknowledgement This work was supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy – EXC 2120/1 – 390831618 and European Union’s Horizon Europe research and innovation program under grant agreement No 101135025, PRESENCE project.