This repository is closely associated with our survey paper: Radiance Fields in XR: A Survey on How Radiance Fields are Envisioned and Addressed for XR Research, with the preprint avaliable: here: https://www.arxiv.org/abs/2508.04326.
If you find this repository valuable, please consider citing it in your work and giving it a star ⭐.
- 🔬: XR-study - Papers that include an in-depth evaluation with users via XR setups and scenarios (e.g., performance evaluation using an XR headset) or contributions that include technical benchmark using XR-specific metrics whose results are directly transferrable to XR applications (e.g., performance tests of a foveated rendering algorithm using pre-collected user eye-gaze data).
- ⭐: XR-showcase - Papers that mostly showcase XR applications in the form of preliminary prototypes or demonstrations. Some of these do not have specific user-involved benchmarking or evaluations in the firm context of XR, but we value these researches, whose findings could easily transfer to actual immersive XR settings.
- 🔓: Open Source - The source code is publicly available.
- ➕: New Papers - New relevant papers that were not in the original survey but are awesome RF4XR papers :)
We categorize the papers into nine main themes that are crucial for advancing RF in actual XR applications.
- 🎮 Interactive Experiences
- 👀 User-centric Rendering
- 📈 Benchmarking
- 🎤 Multi-modal
- 🎨 Content Editing
- ✍️ Content Generation
- 🌍 Spatial Registration
- 🏃 Dynamic Capture
- ⚡ Optimization Techniques
🎮 1. Interactive Experiences
- 🔬 Multi-Layer Gaussian Splatting for Immersive Anatomy Visualization (TVCG 2025) Paper
- 🔬 VirtualNexus: Enhancing 360-Degree Video AR/VR Collaboration with Environment Cutouts and Virtual Replicas (UIST 2024) Paper / Video
- 🔬 SharedNeRF: Leveraging Photorealistic and View-dependent Rendering for Real-time and Remote Collaboration (CHI 2024) Paper
- 🔬 Radiance Fields for Robotic Teleoperation (IROS 2024) 🔓 Paper
- 🔬 Reality Fusion: Robust Real-time Immersive Mobile Robot Teleoperation with Volumetric Visual Data Fusion (IROS 2024) 🔓 Paper
- ⭐ Real-Time, Free-Viewpoint Holographic Patient Rendering for Telerehabilitation via a Single Camera (TVCG 2025) 🔓 Paper
- ⭐ 3DCrewCap: Applying 3D Volumetric video capture for XR helicopter rescue crew training and simulation (SIGGRAPH 2024) Paper
- ⭐ Tele-Aloha: A Telepresence System with Low-budget and High-authenticity Using Sparse RGB Cameras (SIGGRAPH 2024) Paper
- ⭐ VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Reality (SIGGRAPH 2024) 🔓 Paper
- ⭐ The Pop-Up Metaverse: A Multi-User, Multi-Tasking Spatial Computing Environment for Collaborative Spatial Problem-Solving (CHI EA 2024) Paper
- ⭐ Experiencing Thing2Reality: Transforming 2D Content into Conditioned Multiviews and 3D Gaussian Objects for XR Communication (UIST-Adj 2024) Paper
- ⭐ ARthroNeRF: Field of View Enhancement of Arthroscopic Surgeries (ISMAR 2024) Paper
- ⭐ Interactive Data Fusion of Neural Radiance Fields for Industrial Facility Inspection in Virtual Reality (VRW 2024) 🔓 Paper / Code
- ⭐ VOODOO XP: Expressive One-Shot Head Reenactment for VR Telepresence (TOG 2024) Paper / Project
- ⭐ RealityGit: Cross Reality Version Control of R&D Optical Workbench (ISMAR-Adj 2023) Paper
- ⭐ Interacting with Neural Radiance Fields in Immersive Virtual Reality (CHI EA 2023) 🔓 Paper
- ⭐ LiveHand: Real-time and Photorealistic Neural Hand Rendering (ICCV 2023) 🔓 Paper
- ⭐ Real-Time Radiance Fields for Single-Image Portrait View Synthesis (TOG 2023) Paper / Project
- ⭐ Drivable Volumetric Avatars using Texel-Aligned Features (SIGGRAPH 2022) 🔓 Paper
- ⭐ Artemis: articulated neural pets with appearance and motion synthesis (TOG 2022) 🔓 Paper / Project
👀 2. User-centric Rendering
- 🔬 Fov-GS: Foveated 3D Gaussian Splatting for Dynamic Scenes (TVCG 2025) Paper
- 🔬 Scene-aware Foveated Neural Radiance Fields (TVCG 2024) Paper
- 🔬 VPRF: Visual Perceptual Radiance Fields for Foveated Image Synthesis (TVCG 2024) Paper
- 🔬 VRS-NeRF: Accelerating Neural Radiance Field Rendering with Variable Rate Shading (ISMAR 2023) Paper
- 🔬 FoV-NeRF: Foveated Neural Radiance Fields for Virtual Reality (TVCG 2022) 🔓 Paper / Code
- ⭐ ImmersiveNeRF: Hybrid Radiance Fields for Unbounded Immersive Light Field Reconstruction (TVCG 2024) Paper
- ⭐ Omnidirectional Neural Radiance Field for Immersive Experience (VRW 2022) Paper
- ⭐ Real-Time Omnidirectional Roaming in Large Scale Indoor Scenes (SIGGRAPH A. 2022) Paper
📈 3. Benchmarking
- 🔬 Creating Virtual Environments with 3D Gaussian Splatting: A Comparative Study (VRW 2025) Paper
- 🔬 Fast and Robust 3D Gaussian Splatting for Virtual Reality (SIGGRAPH A. 2024) Paper
- 🔬 Is 3DGS Useful?: Comparing the Effectiveness of Recent Reconstruction Methods in VR (ISMAR 2024) Paper / Video
- 🔬 NeARportation: A Remote Real-time Neural Rendering Framework (VRST 2022) Paper
- ⭐ Artistic Exploration of a Pixilated Choreography in Virtual Reality (VRW 2025) Paper
- ⭐ Exploring Radiance Field Content Generation for Virtual Reality (VRW 2024)
- ⭐ VR-NeRF: High-Fidelity Virtualized Walkable Spaces (SIGGRAPH A. 2023) 🔓 Paper
- ⭐ Bringing Instant Neural Graphics Primitives to Immersive Virtual Reality (VRW 2023) 🔓 Paper / Code
🎤 4. Multi-modal
🎨 5. Content Editing
- 🔬 OmniPlane: A Recolorable Representation for Dynamic Scenes in Omnidirectional Videos (TVCG 2025) 🔓 Paper
- 🔬 Semantics-Controlled Gaussian Splatting for Outdoor Scene Reconstruction and Rendering (VR 2025) Paper
- 🔬 PaletteGaussian: 3D Photorealistic Color Editing with Gaussian Splatting (ISMAR 2024) Paper
- ⭐ DimSplat: A Real-Time Diminished Reality System (VRW 2025) Paper
- ⭐ GaussianShopVR: Facilitating Immersive 3D Authoring (UIST 2025) Paper / Code / Project
- ⭐ LIVE-GS: LLM Powers Interactive VR by Enhancing Gaussian Splatting (VRW 2025) Paper
- ⭐ A Real-time Method for Inserting Virtual Objects into Neural Radiance Fields (TVCG 2024) Paper
- ⭐ VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactment (CVPR 2024) 🔓 Paper / Project
- ⭐ Dynamic Mesh-Aware Radiance Fields (ICCV 2023) 🔓 Paper
- ⭐ Neural Fields Meet Explicit Geometric Representations for Inverse Rendering of Urban Scenes (CVPR 2023) Paper
✍️ 6. Content Generation
🌍 7. Spatial Registration
🏃 8. Dynamic Capture
- ⭐ Realtime-Rendering of Dynamic Scenes with Neural Radiance Fields (VRW 2025) Paper
- ⭐ Robust Dual Gaussian Splatting for Immersive Human-centric Volumetric Videos (TOG 2024) Paper / Project
- ⭐ SAILOR: Synergizing Radiance and Occupancy Fields for Live Human Performance Capture (TOG 2023) Paper
- ⭐ Human Performance Modeling and Rendering via Neural Animated Mesh (TOG 2022) 🔓 Paper / Project
- ⭐ Mixture of volumetric primitives for efficient neural rendering (TOG 2021) 🔓 Paper / Project
⚡ 9. Optimization Techniques
- 🔬 SRBF-Gaussian: Streaming-Optimized 3D Gaussian Splatting (VR 2025) Paper
- 🔬 FSVFG: Towards Immersive Full-Scene Volumetric Video Streaming (MM 2024) Paper
- 🔬 HybridNeRF: Efficient Neural Rendering via Adaptive Volumetric Surfaces (CVPR 2024) Paper / Project
- 🔬 Superpixel-guided Sampling for Compact 3D Gaussian Splatting (VRST 2024) 🔓 Paper / Code
- ⭐ Human-in-the-Loop Gaussian Model Enhancement with Mobile Robotic Re-Capture (VRW 2025) Paper
- ⭐ HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian Splatting (CVPR 2024) Paper / Project
- ⭐ VideoRF: Rendering Dynamic Radiance Fields as 2D Feature Video Streams (CVPR 2024) Paper / Project
- ⭐ Re-ReND: Real-time Rendering of NeRFs across Devices (ICCV 2023) 🔓 Paper
Please consider citing this list if you find this repository useful:
@article{li2025radiance,
title={Radiance Fields in XR: A Survey on How Radiance Fields are Envisioned and Addressed for XR Research},
author={Li, Ke and Masuda, Mana and Schmidt, Susanne and Mori, Shohei},
journal={arXiv preprint arXiv:2508.04326},
year={2025},
url={https://arxiv.org/abs/2508.04326},
doi={10.48550/arXiv.2508.04326}
}