A Practical Guide to Radiance Fields for XR Research and Applications

IEEE VR 2025 Tutorial on March 8, 2025, Vauban 1

shohei portrait
University of Stuttgart
ke portrait
University of Hamburg
mana.jpg
Keio University

The advent of photorealistic scene representations such as Neural Radiance Fields (NeRF) and 3D Gaussian Splattings (3DGS) enable us to replicate complex real-world elements. This paradigm shift in graphics lets us revisit the way of capturing, storing, and replicating the real world and rethink what to bring in XR (X for any type).

This tutorial provides an opportunity for learning the basics to "start with" future XR research around the topic. Namely, the attendees will learn the principles of view synthesis using machine learning technology and practical aspects of implementations using open-source projects. It also provides discussions and networking with VR attendees who share the same interests.

The tutorial will begin with a general overview, summarizing the latest developments and current trends in radiance field research for XR applications. This will be followed by an in-depth, live demonstration and tutorial on creating basic NeRF and 3DGS scenes for VR using readily available open-source software such as immersive-ngp and UnityGaussianSplatting toolkit. To conclude, we will summarize the key takeaways and facilitate an interactive discussion with the audience, exploring future trends and developments in radiance fields for XR.

PROGRAM (Tentative)
08:30 - 08:40

Opening and Welcoming

08:40 - 09:00

Introduction to Radiance Fields for XR

Shohei Mori presents NeRF and 3DGS and their surrounding fields, focusing at the current challenges of using these technologies in XR research.

09:00 - 09:25

Live Tutorial on NeRF for VR

Ke Li presents immersive-ngp , showing how to get a NeRF model in Unity and VR.

09:25 - 09:50

Live Tutorial on 3DGS for VR

Mana Masuda presents UnityGaussianSplatting , showing how to get 3DGS in Unity and VR.

09:50 - 10:00

Future Challenges in XR

Shohei Mori gives a summary and shares challenges for future XR research using NeRF and 3DGS.

10:00 - 10:15

Interactive Discussion

We welcome all attendees to an interactive discussion and exchange.

Acknowledgement This work was supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy – EXC 2120/1 – 390831618 and European Union’s Horizon Europe research and innovation program under grant agreement No 101135025, PRESENCE project.