Ravi Netravali

Neural Adaptive Video Streaming with Pensieve

Client-side video players employ adaptive bitrate (ABR) algorithms to optimize user quality of experience (QoE). Despite the abundance of recently proposed schemes, state-of-the-art ABR algorithms suffer from a key limitation: they use fixed control rules based on simplified or inaccurate models of the deployment environment. As a result, existing schemes inevitably fail to achieve optimal performance across a broad set of network conditions and QoE objectives. We propose Pensieve, a system that generates ABR algorithms using reinforcement learning (RL). Pensieve trains a neural network model that selects bitrates for future video chunks based on observations collected by client video players. Pensieve does not rely on pre-programmed models or assumptions about the environment. Instead, it learns to make ABR decisions solely through observations of the resulting performance of past decisions. As a result, Pensieve automatically learns ABR algorithms that adapt to a wide range of environments and QoE metrics. We compare Pensieve to state-of-the-art ABR algorithms using trace-driven and real world experiments spanning a wide variety of network conditions, QoE metrics, and video properties. In all considered scenarios, Pensieve outperforms the best state-of-the-art scheme, with improvements in average QoE of 12%--25%. Pensieve also generalizes well, outperforming existing schemes even on networks for which it was not explicitly trained.

Mahimahi: a lightweight toolkit for reproducible web measurement

This demo presents a measurement toolkit, Mahimahi, that records websites and replays them under emulated network conditions. Mahimahi is structured as a set of arbitrarily composable UNIX shells. It includes two shells to record and replay Web pages, RecordShell and ReplayShell, as well as two shells for network emulation, DelayShell and LinkShell. In addition, Mahimahi includes a corpus of recorded websites along with benchmark results and link traces (https://github.com/ravinet/sites). Mahimahi improves on prior record-and-replay frameworks in three ways. First, it preserves the multi-origin nature of Web pages, present in approximately 98% of the Alexa U.S. Top 500, when replaying. Second, Mahimahi isolates its own network traffic, allowing multiple instances to run concurrently with no impact on the host machine and collected measurements. Finally, Mahimahi is not inherently tied to browsers and can be used to evaluate many different applications. A demo of Mahimahi recording and replaying a Web page over an emulated link can be found at http://youtu.be/vytwDKBA-8s. The source code and instructions to use Mahimahi are available at http://mahimahi.mit.edu/.

Authenticating a mobile device's location using voice signatures

Providers of location-based services seek new methods to authenticate the location of their clients. We propose a novel infrastructure-based solution that provides spontaneous and transaction-oriented mobile device location authentication via an integrated 802.11× wireless access point and 3G femtocell access system. By simply making a voice call while remotely monitoring femtocell activity, a calling party can verify a (co-operating) called partys location even when the participants have no pre-existing relationship. We show how such a traffic signature can be reliably detected even in the presence of heavy cross-traffic introduced by other femtocell users. We describe how the verification proceeds without revealing details of the authentication - or even the parties involved - to the location provider.

Multi-Sensor Fusion of Infrared and Electro-Optic Signals for High Resolution Night Images

Electro-optic (EO) image sensors exhibit the properties of high resolution and low noise level at daytime, but they do not work in dark environments. Infrared (IR) image sensors exhibit poor resolution and cannot separate objects with similar temperature. Therefore, we propose a novel framework of IR image enhancement based on the information (e.g., edge) from EO images, which improves the resolution of IR images and helps us distinguish objects at night. Our framework superimposing/blending the edges of the EO image onto the corresponding transformed IR image improves their resolution. In this framework, we adopt the theoretical point spread function (PSF) proposed by Hardie et al. for the IR image, which has the modulation transfer function (MTF) of a uniform detector array and the incoherent optical transfer function (OTF) of diffraction-limited optics. In addition, we design an inverse filter for the proposed PSF and use it for the IR image transformation. The framework requires four main steps: (1) inverse filter-based IR image transformation; (2) EO image edge detection; (3) registration; and (4) blending/superimposing of the obtained image pair. Simulation results show both blended and superimposed IR images, and demonstrate that blended IR images have better quality over the superimposed images. Additionally, based on the same steps, simulation result shows a blended IR image of better quality when only the original IR image is available.

Traffic Signature-based Mobile Device Location Authentication

Spontaneous and robust mobile device location authentication can be realized by supplementing existing 802.11x access points (AP) with small cells. We show that by transferring network traffic to a mobile computing device associated with a femtocell while remotely monitoring its ingress traffic activity, any internet-connected sender can verify the cooperating receivers location. We describe a prototype non-cryptographic location authentication system we constructed, and explain how to design both voice and data transmissions with distinct, discernible traffic signatures. Using both analytical modeling and empirical results from our implementation, we demonstrate that these signatures can be reliably detected even in the presence of heavy cross-traffic introduced by other femtocell users.

Vroom: Accelerating the Mobile Web with Server-Aided Dependency Resolution

The existing slowness of the web on mobile devices frustrates users and hurts the revenue of website providers. Prior studies have attributed high page load times to dependencies within the page load process: network latency in fetching a resource delays its processing, which in turn delays when dependent resources can be discovered and fetched. To securely address the impact that these dependencies have on page load times, we present Vroom, a rethink of how clients and servers interact to facilitate web page loads. Unlike existing solutions, which require clients to either trust proxy servers or discover all the resources on any page themselves, Vrooms key characteristics are that clients fetch every resource directly from the domain that hosts it but web servers aid clients in discovering resources. Input from web servers decouples a clients processing of resources from its fetching of resources, thereby enabling independent use of both the CPU and the network. As a result, Vroom reduces the median page load time by more than 5 seconds across popular News and Sports sites. To enable these benefits, our contributions lie in making web servers capable of accurately aiding clients in resource discovery and judiciously scheduling a clients receipt of resources.

Improved fusing infrared and electro-optic signals for high-resolution night images

Electro-optic (EO) images exhibit the properties of high resolution and low noise level, while it is a challenge to distinguish objects with infrared (IR), especially for objects with similar temperatures. In earlier work, we proposed a novel framework for IR image enhancement based on the information (e.g., edge) from EO images. Our framework superimposed the detected edges of the EO image with the corresponding transformed IR image. Obviously, this framework resulted in better resolution IR images that help distinguish objects at night. For our IR image system, we used the theoretical point spread function (PSF) proposed by Russell C. Hardie et al., which is composed of the modulation transfer function (MTF) of a uniform detector array and the incoherent optical transfer function (OTF) of diffraction-limited optics. In addition, we designed an inverse filter based on the proposed PSF to transform the IR image. In this paper, blending the detected edge of the EO image with the corresponding transformed IR image and the original IR image is the principal idea for improving the previous framework. This improved framework requires four main steps: (1) inverse filter-based IR image transformation, (2) image edge detection, (3) images registration, and (4) blending of the corresponding images. Simulation results show that blended IR images have better quality over the superimposed images that were generated under the previous framework. Based on the same steps, the simulation result shows a blended IR image of better quality when only the original IR image is available.

A new optical flow estimation method in joint EO/IR video surveillance

Electro-Optical (EO) and Infra-Red (IR) sensors have been jointly deployed in many surveillance systems. In this work we study the special characteristics of optical flow in IR imagery, and introduce an optical flow estimation method using co-registered EO and IR image frames. The basic optical flow calculation is based on the combined local and global (CLG) method (Bruhn, Weickert and Schnorr, 2002), which seeks solutions that simultaneously satisfy a local averaged brightness constancy constraint and a global flow smoothness constraint. While CLG method can be directly applied to IR image frames, the estimated optical flow fields usually manifest high level of random motions caused by thermal noise. Furthermore, IR sensors operating at different wavelengths, e.g. meddle-wave infrared (MWIR) and long-wave infrared (LWIR), may yield inconsistent motions in optical flow estimation. Because of the availability of both EO and IR sensors in many practical scenarios, we propose to estimate optical flow jointly using both EO and IR image frames. This method is able to take advantage of the complementary information offered by these two imaging modalities. The joint optical flow calculation fuses the motion fields from EO and IR images using a cross-regularization mechanism and a non-linear flow fusion model which aligns the estimated motions based on neighbor activities. Experiments performed on the OTCBVS dataset demonstrated that the proposed approach can effectively eliminate many unimportant motions, and significantly reduce erroneous motions, such as sensor noise.

Femtocell-assisted location authentication

Location-based applications (e.g., foursquare, Groupon) rely on each clients assertion of his or her location (e.g., uploaded GPS coordinates). Yet these service providers seek methods to authenticate the location of clients to enhance targeted service delivery and support their advertisers. We propose an intelligent infrastructure-based solution that provides spontaneous, transaction-oriented, collusion-resistant mobile device location authentication to a remote party via an integrated 802.11× wireless access point and 3G femtocell access system.

Multi-sensor fusion of electro-optic and infrared signals for high resolution visible images: part I

Electro-Optic (EO) image sensors exhibit the properties of high resolution and low noise level, but they cannot reflect information about the temperature of objects and do not work in dark environments. On the other hand, infrared (IR) image sensors exhibit the properties of low resolution and high noise level, but IR images can reflect information about the temperature of objects all the time. Therefore, in this paper, we propose a novel framework to enhance the resolution of EO images using the information (e.g., temperature) from IR images, which helps distinguish temperature variation of objects in the daytime via high-resolution EO images. The proposed novel framework involves four main steps: (1) select target objects with temperature variation in original IR images; (2) fuse original RGB color (EO) images and IR images based on image fusion algorithms; (3) blend the fused images of target objects in proportion with original gray-scale EO images; (4) superimpose the target objects’ temperature information, onto original EO images via the modified NTSC color space transformation. Therein, the image fusion step will be conducted by qualitative (frame pipeline) approach. Revealing temperature information in EO images for the first time is the most significant contribution of this paper. Simulation results will show the transformed EO images with the targets’ temperature information.