Indexed in

License and use

Citations

Altmetrics

Grant support

This work has been supported in part by the European Commission under the SNS project IMAGINE-B5G (Advanced 5G Open Platform for Large Scale Trials and Pilots across Europe), grant number 101096452. This project has received funding from the CHIPS Joint Undertaken as part of the European Union's Horizon Europe research and innovation programme, SMARTY Project, grant agreement No. 101140087. B. Sonkoly was supported by the Janos Bolyai Research Scholarship of the Hungarian Academy of Sciences.

Analysis of institutional authors

Share

Enablers of low-latency immersive interaction in future remote-rendered Mixed Reality applications

Publicated to: 170-180 - 2025-01-01 (), DOI: 10.1145/3712676.3714448

Authors: Doka, Janos; Nagy, Balint Gyorgy; Jocha, David; Formanek, Bence; Viciedo, Ivan; Rodrigo, Adrian; Gomez-Barquero, David; Sonkoly, Balazs

Affiliations

Abstract

Mixed Reality (MR) has launched from science fiction but its entrance in reality can reshape our society. By combining the physical and virtual worlds, it provides novel ways of immersive interactions and experiences and by these means enables a new generation of applications. The most exciting and challenging ones support collaborative, multi-user operation in large geographical scale, require real-time environment comprehension and high visual fidelity. The success or failure is definitely impacted by the capabilities and performance limits of edge cloud platforms and 5G/6G networks providing the offloading features for CPU/GPU intensive MR functions. In addition, the desired quality of user experience calls for further mechanisms at the application level hiding the consequences of varying network characteristics. In this paper, we propose a novel edge cloud based architecture for future remote-rendered MR applications supporting low-latency immersive interactions. Our contribution is threefold. First, the system architecture is presented focusing on the remote rendering, 3D simulation and environment detection control loops. Second, we highlight the main features of our proof-of-concept prototype and our dedicated application, namely the Mixed Reality version of a Rocket League inspired game. Third, the concepts are validated via experiments in a Beyond 5G infrastructure where we analyze the operation and latency characteristics of the overall system. In addition, the quality of the user experience is also evaluated via real-life experiments conducted as part of a student competition. The results show that the latency and jitter characteristics of the most sensitive render loop can be managed efficiently together by a network-level control (slice priorities) and an application-level (dynamic jitter buffer) mechanism.

Keywords