8 December 2023
DESY
Europe/Berlin timezone

Immersive Neural Graphics for Robot Teleoperation and Remote Inspection at Complex Physics Facilities

8 Dec 2023, 14:12
12m
Flash Seminar Room (DESY)

Flash Seminar Room

DESY

Flash Talk Session III

Speaker

Ke Li (MCS (Control System))

Description

Large industrial facilities are complex systems that not only require regular maintenance and upgrades but are often inaccessible to humans due to various safety hazards. Therefore, a virtual reality (VR) system that can quickly replicate real-world remote environments to provide users with a high level of spatial and situational awareness is crucial for facility maintenance planning and robot teleoperation. However, the exact 3D shapes of these facilities are often too complex to be accurately modeled with geometric meshes through the traditional rasterization pipeline.
In this work, we present how neural graphics primitives such as neural radiance fields (NeRF) and 3D Gaussian radiance fields (3DGS) can be used to rapidly create one-to-one replications of complex physics facilities at advanced optics laboratories and particle accelerators. We demonstrate how users can interact with such photorealistic virtual environments in immersive mixed and virtual reality with sample applications in robot teleoperation and facility inspection and planning.

Primary author

Ke Li (MCS (Control System))

Co-authors

Prof. Frank Steinicke (UHH) Dr Reinhard Bacher (DESY-MCS) Prof. Wim Leemans (DESY)

Presentation materials