Device Inter-Communication for Smartphone-Based XR Systems

Sammanfattning

Recent developments in hardware manufacturing have allowed mobile devices,such as smartphones, to achieve the computational power necessary to run ExtendedReality (XR) applications. However, smartphones generally lack the appropriate dis-play hardware and specialized sensor units required to make a smartphone poweredVR or XR experience both comfortable and truly immersive. To mitigate this lim-itation, external hardware, such as a dedicated display, specialized sensors, and asingle-board computer (SBC) acting as a sensor aggregator, can be used to supple-ment the smartphone’s capabilities. This distributed architecture requires real-time,bidirectional communication between the smartphone and the SBC.This thesis lays the foundation for a communication framework designed to fulfillthis role. To evaluate the feasibility and performance of this approach, a motion-to-photon latency of 20 ms or lower was identified as the primary evaluation metricand industry standard for comfortable XR. An artifact based on an optimized, open-source communication pipeline was instantiated and subjected to a controlled ex-periment across both restricted (Raspberry Pi 5) and unrestricted (laptop) hardwareenvironments.The results demonstrate that the artifact is functionally correct, consistently achiev-ing latency below the 20 ms threshold in the unrestricted baseline environment. How-ever, in the restricted target environment, the hardware presented a performance bot-tleneck. The primary limitation was not the USB transport channel or the smart-phone’s encoding capacity, but the decoding capabilities of the SBC. While the useof H.265/HEVC with hardware-accelerated decoding yielded the most stable resultson the Raspberry Pi 5, the latency remained above the 20 ms threshold. Ultimately,this thesis proves the viability of a smartphone-to-SBC communication frameworkfor XR, while highlighting that future iterations must utilize SBCs with stronger,codec-specific hardware acceleration to achieve the required real-time performance.