The 2026 NSF Workshop on Networking and Systems Challenges in Immersive Computing

Abstract: Extended Reality (XR) devices distinguish themselves from other mobile devices by providing an immersive and interactive experience. The ability of these devices to collect information presents challenges and opportunities to improve existing security and privacy techniques in this domain. In this talk, I will discuss how readily available eye-tracking sensor data can be used to improve existing methods for assuring security and protecting the privacy of those near the device. Our research has presented three new systems: BystandAR (MobiSys'23), ShouldAR (IMWUT'24), and GazePair (TMC'23), which leverage the user's eye gaze to improve security and privacy expectations in or with XR. Finally, we present a practical attack, EyeSpy (S&P’26), which exploits a novel GPU side channel in foveated rendering to reconstruct users’ gaze data, along with lightweight defense mechanisms to mitigate the threat. As these XR devices grow in power and number, such solutions are necessary to prevent perception and privacy failures that hindered earlier devices. This work is presented in the hope that these solutions can improve and expedite the adoption of these powerful and useful XR devices.

Bio: Bo Ji is an Associate Professor of Computer Science and Electrical and Computer Engineering (by courtesy), and a College of Engineering Faculty Fellow at Virginia Tech. His research interests include interdisciplinary intersections of computing and networking systems, artificial intelligence and machine learning, security and privacy, and extended reality. More info: https://people.cs.vt.edu/boji.

Abstract: The concurrent revolutions in generative AI and spatial computing are fundamentally transforming how we work, communicate, and live. While AI research is accelerated by mature frameworks and benchmarks, inventing AI-driven XR interactions remains a high-friction process. This raises a critical question: how can we make AI + XR technology more accessible and useful to creators?

We introduce XR Blocks (https://xrblocks.github.io), an open-source and cross-platform framework to accelerate human-centered AI + XR innovation. Driven by the mission of “minimizing code from idea to reality”, XR Blocks provides core abstractions and samples that empower creators to move from concept to interactive AI + XR applications. We present live demonstrations of Vibe Coding XR (https://xrblocks.github.io/gem), an end-to-end rapid prototyping workflow that leverages Gemini to translate natural language intent directly into functional XR software. We provide a preliminary technical evaluation on a pilot dataset (VCXR60) alongside diverse application scenarios highlighting mixed-reality realism, multi-modal interaction, and generative AI integrations. By democratizing spatial system creation on the open web, we offer a visionary glimpse into the future and the automatic evolution of the AI + XR flywheel.

Bio: Ruofei Du is the Interactive Perception & Graphics Lead at Google XR, where he drives AI + XR innovations. An established expert in his field, he serves as an Associate Chair on the program committees for both ACM CHI (2021-2026) and UIST (2022-2026); an Associate Editor for IEEE Transactions on Circuits and Systems for Video Technology. Dr. Du's extensive contributions includes 7 US patents and over 45 peer-reviewed publications in top venues across HCI, Computer Graphics, and Computer Vision (including CHI, SIGGRAPH, UIST, TVCG, CVPR, and ICCV). His research has garnered significant recognition, including a Distinguished Paper Award in ACM IMWUT, Best Paper Awards at SIGGRAPH Web3D 2016 and SIGGRAPH I3D 2024, and multiple Honorable Mentions Awards at ACM CHI and IEEE TVCG. He holds a Ph.D. in Computer Science from the University of Maryland, College Park. Website: https://duruofei.com.

Abstract: Autonomous drones are gaining momentum. In this talk, I would like to introduce our ongoing efforts towards autonomous drones for our daily uses. To bridge the gaps from ideal lab settings to real-world environments, we face many practical challenges in understanding and reasoning a 3D physical world while tackling the tension between precision and responsiveness to guarantee almost 100% safety. We will use several showcase applications to unveil challenges and opportunities in one of the most challenging scenarios - below 100ft above the ground level.

Bio: Chunyi Peng is an Associate Professor in the Department of Computer Science at Purdue University. Her research interests are in the broad areas of mobile networking, system and security, with a recent focus on renovating 5G/xG radio access technologies, AI for networks, 5G/IoT security, mobile edge computing (mainly for autonomous drones, vehicles and robots). More information can be found at her homepage: https://www.cs.purdue.edu/homes/chunyi/.

Abstract: Today's wireless access networks operate on an application-agnostic "one-size-fits-all" principle, which falls short of addressing the specific performance and user experience requirements of immersive applications such as Extended Reality (XR). In this circumstance, while application-agnostic network Quality of Service (QoS) remains vital, it alone cannot guarantee optimal user experience, as even identical QoS can result in significantly varying application performance. To bring service awareness directly to underlying wireless access, This talk explores cutting-edge trends and technologies focused on integrating service awareness into wireless access, paving the way for network-application co-optimization.

Bio: Nakjung Choi obtained his B.S. and Ph.D. degrees from the School of Computer Science and Engineering at Seoul National University in 2002 and 2009, respectively. He currently leads the Mobile Network Systems department within the Network Systems and Security Research Lab at Nokia Bell Labs. He has been honored with several accolades, including Best Paper Awards, Awards of Excellence, and the Edison Patent Award. His research focuses on network orchestration, automation, and control, immersive networking, and cellular technologies including autonomous AI RAN.

Abstract: Immersive systems such as XR, teleoperation, and human-robot interaction require real-time AI to support continuous, responsive experiences. However, minimizing latency at every inference step is not always necessary or sufficient. In streaming and interactive settings, what matters is maintaining a stable response rate under dynamic network and compute conditions. This talk rethinks real-time AI as a rate- and consistency-aware control problem. Using device-server collaborative DNN/LLM streaming as a case study, we show how adaptive inference decisions can improve responsiveness and stability, and discuss implications for next-generation immersive computing systems.

Bio: Xueyu Hou is an Assistant Professor in Electrical and Computer Engineering at the University of Maine. Her research focuses on edge computing, real-time AI systems, and immersive computing, with an emphasis on enabling low-latency and adaptive intelligence for XR, robotics, and interactive applications. She develops cross-layer systems that jointly optimize communication, computation, and machine learning under dynamic network and resource constraints. Her work has been published in venues such as ACM MobiCom, ACM MobiSys, IEEE IPDPS, IEEE Transactions on Mobile Computing, and IEEE Transactions on Cloud Computing. Her current research explores real-time DNN systems, collaborative edge intelligence, and AI-driven immersive experiences.

Abstract: Spatial computing is driving new demands on digital infrastructure, requiring datacenters to support low-latency, high-bandwidth, and AI-intensive workloads at scale. This talk is an overview of several Bell Labs research efforts into how datacenter architectures are evolving to enable immersive applications such as augmented reality, digital twins, robotics, and real-time simulation. It highlights the role of accelerated computing, edge-cloud integration, advanced networking, and energy-efficient design in meeting performance and sustainability goals. It also outlines how next-generation datacenters can become foundational platforms for the emerging spatial computing ecosystem worldwide.

Bio: Eugene Chai is a Principal Researcher in the IP Networks department at Nokia Bell Labs. His broad research interests include datacenter architectures for AI workflows and AR/XR experiences, edge computing, mobile networking, 5G infrastructure and its use in autonomous platforms, immersive experiences and security in XR systems.

Abstract: Data analysis has become a critical component of many modern professional workflows. Data-driven workflows are inherently complex, characterized by their iterative, interactive, and often prolonged nature. Analysts rely heavily on digital input and output systems to externalize information and augment human cognitive capacity. These systems serve as crucial intermediaries between raw data and meaningful insights, facilitating the exploration, manipulation, and interpretation of large datasets. The evolution of digital tools for data analysis has seen significant advancements in user interface design, from early command-line interfaces to sophisticated graphical environments, continuously adapting to better align with users’ mental models and improve overall efficiency.

Immersive technologies are rapidly evolving toward becoming a mainstream computing environment, presenting significant potential for revolutionizing data analysis practices. Virtual/augmented reality (VR/AR) systems, capable of projecting interactive screens onto any surface or immersive graphics into any space around the user, offer unprecedented possibilities for data analysis. With VR/AR, users can spatially arrange numerous data-related artifacts and intuitively interact with data through body movements. In this talk, Yalong will introduce some of his work in exploring how to take advantage of those features to build novel and effective interactive systems in VR/AR.

Bio: Yalong Yang is an Assistant Professor in the School of Interactive Computing at Georgia Tech. Before joining Georgia Tech, he spent two wonderful years at Virginia Tech as a faculty member. Prior to this, he was a Postdoctoral Fellow in the Visual Computing Group at Harvard University and received his Ph.D. from Human-Centred Computing Department, Monash University, Australia. His research focuses on VR/AR/MR/XR (spatial computing), Visualization (VIS), and Human-Computer Interaction (HCI). He regularly publishes in premier venues such as ACM CHI, ACM UIST, IEEE TVCG, IEEE VR, IEEE VIS, ISMAR, and EuroVis, with his work earning three Best Paper Honorable Mention Awards (IEEE VIS 2016 & 2022; ACM CHI 2021). He is an NSF CAREER Award recipient and was selected as a Google Research Scholar in 2025. He also actively serves the research communities by serving on organization and program committees for ACM CHI, IEEE VR, IEEE VIS, and ISMAR. More information is available at https://ivi.cc.gatech.edu/pi.html.

Abstract: In this talk we propose an enhanced cellular network control plane (agent plane) which decouples QoS management from traditional network signaling plane. The talk outlines the design of the proposed agent plane and illustrates how it can be used to support on‑demand allocation of dedicated network resources on a per‑call basis.

Bio: Amit Sheoran is a researcher at AT&T Labs - Research. His research focus on mobile networks, cloud computing, and applied machine learning.

Abstract: We are witnessing a historic inflection point in visual computing. For decades, progress has come from separately advancing optics, graphics, and algorithms. Cameras captured, computers processed, and displays rendered. But in the age of AI, these silos are dissolving. Learning-based models now blur the line between sensing and inference, between display and perception, and between physical optics and digital computation. As AI becomes an inseparable part of how we capture, compute, and experience visual information, we must reimagine visual computing not as a stack of components, but as a unified system that is adaptive, perception-aware, and ultimately, human-centered.

In this talk, I will share a vision for rethinking visual computing systems when AI is not just an add-on, but a first-class design principle. I will highlight examples from our research where this shift is already transforming the landscape: holographic displays that adapt to human perception, metasurface optics designed in tandem with neural solvers, and multimodal sensor fusion that captures reality at unprecedented speed and fidelity. These case studies point toward a future where the entire pipeline—from photons to pixels to perception—is co-designed with intelligence at its core.

Bio: Praneeth Chakravarthula is an Assistant Professor of Computer Science at the University of North Carolina at Chapel Hill. His research lies at the intersection of graphics, vision, optics, and AI, and has been recognized with multiple best paper awards at SIGGRAPH, IEEE VR, and IEEE ISMAR, as well as the IEEE VR Best Dissertation Award and DARPA Disruptive Idea Award. Before joining UNC, Praneeth completed postdoctoral research at Princeton University, earned his PhD and MS at UNC, and B.Tech and M.Tech degrees in Electrical Engineering from IIT Madras.

Abstract: Immersive AR/VR experiences depend on knowing precisely where a headset or device is in the world — but most localization methods rely on cameras that fail in the dark, in smoke, or in visually featureless spaces. Our recent work introduces mmNeRF, a system that uses millimeter-wave (mmWave) radar — the same technology found in modern automotive sensors — to build a rich 3D map of an environment and continuously track a device's exact position and orientation in full 6 degrees of freedom. The key insight is to apply Neural Radiance Fields (NeRF), a technique originally developed for photorealistic image synthesis, to radar signals: mmNeRF learns a compact neural representation of a space purely from radar scans, and can then synthesize new views and self-localize. Our research demonstrates strong and consistent localization performance across a range of challenging indoor environments. This opens the door to robust, vision-free tracking for AR/VR headsets and mixed-reality devices, even in conditions where cameras fall short.

Bio: Parth Pathak is an associate professor in the Computer Science Department in the School of Computing at George Mason University. His research interests are broadly in the areas of wireless networking and mobile computing, including high-speed millimeter-wave wireless networks for supporting applications such as AR/VR/MR, application of machine learning in wireless networking and sensing, low-power, high-speed Internet-of-Things like backscatter systems, and wireless sensing and imaging for autonomous robotic systems, including UGVs and drones.

Abstract: A browser fetches individual objects—images, scripts, styles—and composites them locally into the final display. Video delivery works nothing like this: every pixel is encoded, transmitted, and decoded as a monolithic stream. What if recurring visual objects could be downloaded once, cached on the client, and composited into each frame with lightweight local processing? We explore this question in the context of cloud gaming, where the server has full scene knowledge. We present early results from ongoing work on a system that segments recurring objects from game frames—such as weapons or dashboards—before encoding, and transmits compact metadata enabling the client to reconstruct frames using locally cached templates. A conservative gating mechanism ensures graceful fallback to standard streaming when reconstruction cannot be guaranteed. Early experiments across multiple game genres show promising bandwidth savings while preserving visual quality.

Bio: Zahaib Akhtar is a Senior Applied Scientist at Amazon Prime Video and serves as an Adjunct Assistant Professor in the Computer Science Department at North Carolina State University (NCSU). At Amazon Prime Video he architects and deploys systems that enhance streaming experience for millions of users. His research investigates a range of areas such as streaming quality optimizations, cloud gaming delivery, performance observability and large-scale operations. His work regularly bridges the gap between academic research and industry applications by bringing state-of-the-art techniques to production-scale video streaming challenges.