Eduardo Cuervo

MAUI: making smartphones last longer with code offload

This paper presents MAUI, a system that enables fine-grained energy-aware offload of mobile code to the infrastructure. Previous approaches to these problems either relied heavily on programmer support to partition an application, or they were coarse-grained requiring full process (or full VM) migration. MAUI uses the benefits of a managed code environment to offer the best of both worlds: it supports fine-grained code offload to maximize energy savings with minimal burden on the programmer. MAUI decides at run-time which methods should be remotely executed, driven by an optimization engine that achieves the best energy savings possible under the mobile devices current connectivity constrains. In our evaluation, we show that MAUI enables: 1) a resource-intensive face recognition application that consumes an order of magnitude less energy, 2) a latency-sensitive arcade game application that doubles its refresh rate, and 3) a voice-based language translation application that bypasses the limitations of the smartphone environment by executing unsupported components remotely.

Outatime: Using Speculation to Enable Low-Latency Continuous Interaction for Mobile Cloud Gaming

Gaming on phones, tablets and laptops is very popular. Cloud gaming - where remote servers perform game execution and rendering on behalf of thin clients that simply send input and display output frames - promises any device the ability to play any game any time. Unfortunately, the reality is that wide-area network latencies are often prohibitive; cellular, Wi-Fi and even wired residential end host round trip times (RTTs) can exceed 100ms, a threshold above which many gamers tend to deem responsiveness unacceptable. In this paper, we present Outatime, a speculative execution system for mobile cloud gaming that is able to mask up to 120ms of network latency. Outatime renders speculative frames of future possible outcomes, delivering them to the client one entire RTT ahead of time, and recovers quickly from mis-speculations when they occur. Clients perceive little latency. To achieve this, Outatime combines: 1) future state prediction; 2) state approximation with image-based rendering and event time-shifting; 3) fast state checkpoint and rollback; and 4) state compression for bandwidth savings. To evaluate the Outatime speculation system, we use two high quality, commercially-released games: a twitch-based first person shooter, Doom 3, and an action role playing game, Fable 3. Through user studies and performance bench- marks, we find that players strongly prefer Outatime to traditional thin-client gaming where the network RTT is fully visible, and that Outatime successfully mimics playing across a low-latency network.

Demo: FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization

Virtual reality head-mounted displays (VR HMDs) are attracting users with the promise of full sensory immersion in virtual environments. Creating the illusion of immersion for a near-eye display results in very heavy rendering workloads: low latency, high framerate, and high visual quality are all needed. Tethered VR setups in which the HMD is bound to a powerful gaming desktop limit mobility and exploration, and are difficult to deploy widely. Products such as Google Cardboard and Samsung Gear VR purport to offer any user a mobile VR experience, but their GPUs are too power-constrained to produce an acceptable framerate and latency, even for scenes of modest visual quality. We present FLASHBACK, an unorthodox design point for HMD VR that eschews all real-time scene rendering. Instead, FLASHBACK aggressively precomputes and caches all possible images that a VR user might encounter. FLASHBACK memoizes costly rendering effort in an offline step to build a cache full of panoramic images. During runtime, FLASHBACK constructs and maintains a hierarchical storage cache index to quickly lookup images that the user should be seeing. On a cache miss, FLASHBACK uses fast approximations of the correct image while concurrently fetching more closely-matching entries from its cache for future requests. Moreover, FLASHBACK not only works for static scenes, but also for dynamic scenes with moving and animated objects. We evaluate a prototype implementation of FLASHBACK and report up to a 8x improvement in framerate, 97x reduction in energy consumption per frame, and 15x latency reduction compared to a locally-rendered mobile VR setup. In some cases, FLASHBACK even delivers better framerates and responsiveness than a tethered HMD configuration on graphically complex scenes.

CrowdLab: An architecture for volunteer mobile testbeds

Researchers investigating mobile and wireless systems can run experiments on many testbeds, but no existing option supports experimentation “in the wild“ without sacrificing features such as access to low-level wireless state and efficient scheduling of co-local guests. To fill this void, we present a new architecture for mobile testbeds called CrowdLab. CrowdLab allows researchers to run guest virtual machines on volunteer mobile nodes and ensures efficient use of testbed resources through a new dual-mode networking abstraction and a weakly-consistent, replicated state store called a site directory. We have implemented two CrowdLab prototypes, one for x86 laptops and one for ARM-based Nokia N810 Internet Tablets, and evaluated them using power measurements, micro-benchmarks, and trace-driven emulation. Our evaluation demonstrates that handheld users can contribute 2.5 hours per day to CrowdLab and still have over 12.5 hours of idle time remaining. In addition, emulated mobility-trace replays show that CrowdLabs fault-tolerance mechanisms allow experiments to run uninterrupted, even in the face of high churn rates.

Demo: Kahawai: high-quality mobile gaming using GPU offload

This paper presents Kahawai1, a system that provides high-quality gaming on mobile devices, such as tablets and smartphones, by offloading a portion of the GPU computation to server-side infrastructure. In contrast with previous thin-client approaches that require a server-side GPU to render the entire content, Kahawai uses collaborative rendering to combine the output of a mobile GPU and a server-side GPU into the displayed output. Compared to a thin client, collaborative rendering requires significantly less network bandwidth between the mobile device and the server to achieve the same visual quality and, unlike a thin client, collaborative rendering supports disconnected operation, allowing a user to play offline - albeit with reduced visual quality. Kahawai implements two separate techniques for collaborative rendering: (1) a mobile device can render each frame with reduced detail while a server sends a stream of per-frame differences to transform each frame into a high detail version, or (2) a mobile device can render a subset of the frames while a server provides the missing frames. Both techniques are compatible with the hardware-accelerated H.264 video decoders found on most modern mobile devices. We implemented a Kahawai prototype and integrated it with the idTech 4 open-source game engine, an advanced engine used by many commercial games. In our evaluation, we show that Kahawai can deliver gameplay at an acceptable frame rate, and achieve high visual quality using as little as one-sixth of the bandwidth of the conventional thin-client approach. Furthermore, a 50-person user study with our prototype shows that Kahawai can deliver the same gaming experience as a thin client under excellent network conditions.

Experimenting in mobile social contexts using JellyNets

Programmable consumer devices have placed computation within arms reach at all times and in all places. Unfortunately, researchers interested in investigating this phenomenon often struggle with the expense, inconvenience, and limited scale of existing experimental platforms. In this paper, we introduce a new experimental platform for mobile and pervasive computing based on JellyNets, an abstraction for exposing experiments to arbitrary mobile social contexts.

Proxy-guided image-based rendering for mobile devices

VR headsets and hand‐held devices are not powerful enough to render complex scenes in real‐time. A server can take on the rendering task, but network latency prohibits a good user experience. We present a new image‐based rendering (IBR) architecture for masking the latency. It runs in real‐time even on very weak mobile devices, supports modern game engine graphics, and maintains high visual quality even for large view displacements. We propose a novel server‐side dual‐view representation that leverages an optimally‐placed extra view and depth peeling to provide the client with coverage for filling disocclusion holes. This representation is directly rendered in a novel wide‐angle projection with favorable directional parameterization. A new client‐side IBR algorithm uses a pre‐transmitted level‐of‐detail proxy with an encaging simplification and depth‐carving to maintain highly complex geometric detail. We demonstrate our approach with typical VR / mobile gaming applications running on mobile hardware. Our technique compares favorably to competing approaches according to perceptual and numerical comparisons.

Demo: DeLorean: using speculation to enable low-latency continuous interaction for mobile cloud gaming

Playing games on mobile devices is very popular. Recently, cloud gaming – where datacenter servers execute the games on behalf of thin clients that merely transmit UI input events and display output rendered by the servers – has emerged as an interesting alternative to traditional clientside game execution. Cloud Gaming-as-a-Service (GaaS) offers several advantages salient to mobile clients. First, users with low end devices can get the same high quality experience as users with high end devices. Second, mobile game developers avoid two challenges that arise with the huge diversity of mobile devices: platform compatibility headaches and per-platform performance tuning. Third, upgrading servers (e.g., for bug fixes, game updates, etc.) becomes far easier than redeploying new software to clients. Finally, players can select from a vast library of games and instantly play any of them. However, GaaS on mobile devices faces a key technical dilemma: how can players attain real-time interactivity in the face of wide-area latency? Real-time interactivity means client input events should be quickly reflected on the client display. User studies have shown that players are sensitive to as little as 60 ms latency, and are aggravated at latencies in excess of 100 ms [1]. A further delay degradation from 150 ms to 250 ms lowers user engagement by 75% [2]. Instead, we propose to mitigate wide-area latency via speculative execution. We present DeLorean a system that delivers real-time gaming interactivity as fast as traditional local client-side execution, despite with network latencies. DeLorean’s basic approach combines input prediction with speculative execution to render mulitple possible frame outputs which could occur RTT milliseconds in the future. DeLorean employs the following techniques to accomplish this. Future Input Prediction: Given the user’s historical tendencies and recent behavior, we show that some categories of user actions are highly predictable. We develop a Markovbased prediction model that examines recent user input to forecast expected future input. We use two techniques to improve prediction quality: supersampling of input events,

Poster: Mobile Virtual Reality for Head-mounted Displays With Interactive Streaming Video and Likelihood-based Foveation

Immersive virtual reality has long been an aspiration of many. For a truly immersive VR experience, three properties are essential: quality, responsiveness and mobility. By quality, we mean that realistic and life-like visual portrayals in a virtual environment heighten our sense of immersion. By responsiveness, we mean that any motion, especially of the head, must be reflected as quickly as possible in visual feedback because ocular proprioception sensitivity is very high. By mobility, we mean that we ought to be able to move untethered in physical space, free to explore our virtual world. Unfortunately mobile devices like phones are tablets are power-constrained and cannot produce acceptable framerate. While cloud offloading allows them to deliver high-quality and framerate it places high bandwidth requirements and introduces an unacceptable amount of latency. This demo introduces Matia, a stereo HMD system that simultaneously attains quality, responsiveness and mobility. Matia achieves this by offloading rendering work to a highend GPU across the WAN. The HMD client is mobile and only requires a low-end GPU, yet receives high quality imagery. To overcome WAN latencies, Matia borrows speculative execution techniques inspired by previous work Outatime [1]. However, speculative execution alone is insufficient. This is because HMD sensitivities are substantially more stringent along several key dimensions: responsiveness and quality.These factors suggest that we ought to greatly benefit from maximally utilizing all available bandwidth between client and server to deliver the highest resolution imagery possible. At the same time, any latency or bandwidth changes must be handled very responsively, lest they deteriorate the user experience and induce simulator sickness. To get the highest possible resolution while remaining responsive to network changes, Matia employs panoramic stereo video and likelihood-based foveation. Specifically, Matia renders a wide field-of-view (FOV) panoramic stereo video. In turn, any possible stereo view (e.g., due to unexpected head movement or network fluctuations) can be generated. Matia then foveates this panorama by reallocating pixels to areas where the user is most likely to look (Maximum Likelyhood View). We divide our panoramic video into three regions described by their image quality: High, Medium and Low. Embedded within is a convex-like optimizer that constantly adapts to real-time data analysis of user head movement and network conditions to figure out the most fruitful foveation configuration, on expectation of these regions. As a result, we expect to achieve high quality immersiveness in the common case, yet still deliver gracefully degraded experiences in case of network degradation. At the same time, Matia greatly expands the scope of viable HMD content while requiring only modest fixed function computation on the HMD device. In this demo we will show Matia running on an Intel Compute Stick driving an Oculus DK2. Our demo shows a scene so graphically detailed that the compute stick is unable to handle it fast enough without offloading it to a server. During the demo, users will be able to observe the high responsiveness and quality of the image even after introducing variable amounts of bandwidth and latency.

FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization

Driven by recent advances in the mobile computing hardware ecosystem, wearable Virtual Reality (VR) is experiencing a boom in popularity, with many offerings becoming available. Modern VR head-mounted displays (HMDs) fall into two device classes: (i) Tethered HMDs: HMDs tethered to powerful, expensive gaming desktops, such as the Oculus Rift, HTC Vive, and Sony PlayStation VR; (ii) Mobile-rendered HMDs: self-contained, untethered HMDs that run on mobile phones slotted into head mounts, e.g., Google Cardboard and Samsung Gear VR.