Demo: Attention and Mobility-Aware On-Device Perception for Mobile Augmented Reality

With the ability to seamlessly integrate physical and virtual worlds to provide immersive experiences, Mobile Augmented Reality, is a promising technology for several futuristic applications such as on the-job training and guidance in smart factories, human-autonomy teaming, etc. However, the hardware limitations of most modern AR/VR headsets are prohibitive in that the processing latencies for common scene understanding tasks, a foundational primitive for such applications, can be in the order of 1 leading to poor quality of experience (QoE). The problem is further exacerbated by contention for resources between processing and rendering. State of-the-art in perception for these headsets rely on assistance from nearby edge servers [1, 2] which may not always be available in practical settings. In this demonstration, we show how user atten tion and mobility-aware scheduling can improve the processing latency without significant loss in perception accuracy and QoE for on-device perception.