Siddhartha Chattopadhyay

Evaluating Quality of Experience for Streaming Video in Real Time

We present a scalable, lightweight, no-reference framework to infer video QoE. Our framework revolves around a one time offline construction of a k-dimensional space, which we call the QoE space. The k-dimensions accommodate k parameters (network-dependent/independent) that potentially affect video quality. The k-dimensional space is partitioned to N representative zones, each with a QoE index. Instantaneous parameter values are matched with the indices to infer QoE. To validate our framework, we construct a 3-dimensional QoE space with bit-rate, loss, and delay as the principal components. We create 18 video samples with unique combinations of the 3 parameters. 77 human subjects rated these video samples on a scale of 1 to 5 to create the QoE space. In a second set of survey, our predicted MOS was compared to 49 human responses. Results show that our MOS predictions are in close agreement with subjective perceptions. An implementation of our framework on standard Linux PC shows we can compute 20 MOS calculations per second with 3 parameters and 18 partitions of the QoE space.

Human Motion Capture Data Compression by Model-Based Indexing: A Power Aware Approach

Human motion capture (MoCap) data can be used for animation of virtual human-like characters in distributed virtual reality applications and networked games. MoCap data compressed using the standard MPEG-4 encoding pipeline comprising of predictive encoding (and/or DCT decorrelation), quantization, and arithmetic/Huffman encoding, entails significant power consumption for the purpose of decompression. In this paper, we propose a novel algorithm for compression of MoCap data, which is based on smart indexing of the MoCap data by exploiting structural information derived from the skeletal virtual human model. The indexing algorithm can be fine-controlled using three predefined quality control parameters (QCPs). We demonstrate how an efficient combination of the three QCPs results in a lower network bandwidth requirement and reduced power consumption for data decompression at the client end when compared to standard MPEG-4 compression. Since the proposed algorithm exploits structural information derived from the skeletal virtual human model, it is observed to result in virtual human animation of visually acceptable quality upon decompression

A framework for encoding and caching of video for quality adaptive progressive download

Progressive download of multimedia objects over the Internet (e.g. www.youtube.com), where the video is downloaded and viewed during the download process, has become an increasingly popular alternative to multimedia streaming. Due to the fluctuating bandwidth and latency of the Internet, progressive download is often not fast enough, often resulting in intermittent stalling of the video. In this paper, we first propose a variation of the existing MPEG Fine Grained Scalability (FGS) profile to create a layered video representation that is suitable for progressive download in an environment characterized by varying bitrate. We also propose an efficient caching scheme that is specifically tailored for the proposed layered video representation. The proposed layered version of the Greedy-Dual-Size cache replacement policy is shown to reduce the latency observed by the client during progressive download of video in a varying bitrate environment. Experimental results demonstrate that the proposed caching scheme improves the latency of progressive video downloads as well as the server efficiency.

Model-Based Power Aware Compression Algorithms for MPEG-4 Virtual Human Animation in Mobile Environments

MPEG-4 body animation parameters (BAP) are used for animation of MPEG-4 compliant virtual human-like characters. Distributed virtual reality applications and networked games on mobile computers require access to locally stored or streamed compressed BAP data. Existing MPEG-4 BAP compression techniques are inefficient for streaming, or storing, BAP data on mobile computers, because: 1) MPEG-4 compressed BAP data entails a significant number of CPU cycles, hence significant, unacceptable power consumption, for the purpose of decompression, 2) the lossy MPEG-4 technique of frame dropping to reduce network throughput during streaming leads to unacceptable animation degradation, and 3) lossy MPEG-4 compression does not exploit structural information in the virtual human model. In this article, we propose two novel algorithms for lossy compression of BAP data, termed as BAP-Indexing and BAP-Sparsing. We demonstrate how an efficient combination of the two algorithms results in a lower network bandwidth requirement and reduced power for data decompression at the client end when compared to MPEG-4 compression. The algorithm exploits the structural information in the virtual human model, thus maintaining visually acceptable quality of the resulting animation upon decompression. Consequently, the hybrid algorithm for BAP data compression is ideal for streaming of motion animation data to power- and network-constrained mobile computers

Ligne-claire video encoding for power constrained mobile environments

Digital video playback on mobile devices is fast becoming widespread and popular. Since mobile devices are typically resource constrained in terms of network bandwidth, battery power and available screen resolution, it is often necessary to formulate special encoding techniques in order to optimize power consumption during video streaming and playback. The existing H.264 standard is popular for video encoding on mobile devices, since it results in a low-bitrate video with visual clarity that is adequate for video playback on mobile devices. However, due to the complexity of the H.264 representation, the video decoding procedure is typically computationally intensive. In this paper, we propose a novel lossy video representation termed as Ligne-Claire (LC) video. LC videos are obtained via graphics overlay of outlines or silhouettes of objects in the video over an approximated texture video. Since the playback of LC video is typically meant for mobile devices, the visual quality of video is adequate for most mobile applications wherein the semantic content of the video can be characterized by object shapes and approximate texture information. Experimental results presented in the paper demonstrate that the proposed lossy LC video encoding scheme results in power savings of 50% or more during video playback compared to standard H.264-encoded videos, of similar video file size. In order to evaluate the visual quality of the LC video, we compare the performance of LC videos with H.264-encoded videos in the context of some typical computer vision tasks. Our results indicate that the performance of the computer vision algorithms on these videos is similar. This fact, coupled with subjective evaluation, and the resulting significant power savings, indicates that the proposed LC representation can be used effectively to encode video for power-constrained mobile devices.

Efficient compression and delivery of stored motion data for avatar animation in resource constrained devices

Animation of Virtual Humans (avatars) is done typically using motion data files that are stored on a client or streaming motion data from a server. Several modern applications require avatar animation in mobile networked virtual environments comprising of power constrained clients such as PDAs, Pocket-PCs and notebook PCs operating in battery mode. These applications call for efficient compression of the motion animation data in order to conserve network bandwidth, and save power at the client side during data reception and motion data reconstruction from the compressed file. In this paper, we have proposed and implemented a novel file format, termed the Quantized Motion Data (QMD) format, which enables significant, though lossy, compression of the motion data. The motion distortion resulting from the reconstructed motion from the QMD file is minimized by intelligent use of the hierarchical structure of the skeletal avatar model. The compression gained by using the QMD files for the motion data is more than twice achieved via standard MPEG-4 compression using a pipeline comprising of quantization, predictive encoding and arithmetic coding. In addition, considerably fewer CPU cycles are needed to reconstruct the motion data from the QMD files compared to motion data compressed using the MPEG-4 standard.

Efficient dissemination of personalized video content in resource-constrained environments

Video streaming on mobile devices such as PDAs, laptop PCs, pocket PCs and cell phones is becoming increasingly popular. These mobile devices are typically constrained by their battery capacity, bandwidth, screen resolution and video decoding and rendering capabilities. Consequently, video personalization strategies are used to provide these resource-constrained mobile devices with personalized video content that is most relevant to the clients request while simultaneously satisfying the clients resource constraints. Proxy-based caching of video content is a proven strategy to reduce both client latencies and server loads. In this paper, we propose novel video personalization server and caching mechanisms, the combination of which can efficiently disseminate personalized videos to multiple resource-constrained clients. The video personalization servers use an automatic video segmentation and video indexing scheme based on semantic video content. The caching proxies implement a novel cache replacement and multi-stage client request aggregation algorithm, specifically suited for caching personalized video files generated by the personalization servers. The cache design also implements a personalized video segment calculation algorithm based on clients content preferences and resource constraints. The paper reports series of experiments that demonstrate the efficacy of the proposed techniques in scalably disseminating personalized video content to resource constrained client-devices.

Hybrid layered video encoding and caching for resource constrained environments

Video playback on a mobile device is a resource-intensive task. Since the battery life of a mobile device decreases with time, it is desirable to have a video representation which adapts dynamically to the available battery life during the playback process. A novel Hybrid Layered Video (HLV) encoding scheme is proposed, which comprises of content-aware, multi-layer encoding of texture and a generative sketch-based representation of the object outlines. Different combinations of the texture- and sketch-based representations are shown to result in distinct video states, each with a characteristic power consumption profile. Further, a smart content-aware caching scheme is proposed which is suitable for low-latency dissemination of HLV over the Internet. The proposed HLV representation, combined with the proposed caching scheme, is shown to be effective for video playback and dissemination on power-constrained mobile devices.

BAP sparsing: a novel approach to MPEG-4 body animation parameter compression

The MPEG-4 standard includes support not only for natural video and audio, but also for synthetic graphics and sounds. In the MPEG-4 specifications, body animation parameters (BAPs) and body definition parameters (BDPs) allow virtual bodies and their animation to be compressed using a standard compression pipeline comprising of quantization, predictive encoding and arithmetic coding of these parameters. In this paper, we propose and implement a new stage within the standard prediction-based compression pipeline for the BAPs, termed as BAP sparsing. BAP sparsing compresses a complete block of motion data, consisting of an initial 1-frame followed by subsequent P-frames, required for creating the animation. It exploits the inherent hierarchical structure of the human skeletal model to intelligently drop and modify the P-frames, while preserving animation quality. BAP sparsing is shown to result in superior compression of the BAP data, with negligible loss in the motion animation quality. It is also shown to result in a lower network throughput requirement and fewer CPU cycles on the client end to create the animation. The proposed method is particularly well suited for animation using BAP data.

Hybrid layered video encoding for mobile internet-based computer vision and multimedia applications

Mobile networked environments are typically resource constrained in terms of the available bandwidth and battery capacity on mobile devices. Real-time video applications entail the analysis, storage, transmission, and rendering of video data, and are hence resource-intensive. Since the available bandwidth in the mobile Internet is constantly changing, and the battery life of a mobile video application decreases with time, it is desirable to have a video representation scheme that adapts dynamically to the available resources. A Hybrid Layered Video (HLV) encoding scheme is proposed, which comprises of content-aware, multi-layer encoding of texture and a generative sketch-based representation of the object outlines. Different combinations of the texture- and sketch-based representations result in distinct video states, each with a characteristic bandwidth and power consumption profile. The proposed HLV encoding scheme is shown to be effective for mobile Internet-based multimedia applications such as background subtraction, face detection, face tracking and face recognition on resource-constrained mobile devices.