Bart Masschelein

Delivering scalable video with QoS to the home

User satisfaction is a key factor in the success of novel multimedia services. Yet, to enable service providers and network operators to control and maximize the quality (QoS, QoE) of delivered video streams, quite some challenges remain. In this paper, we particularly focus on three of them. First of all, objectively measuring video quality requires appropriate quality metrics and methods of assessing them in a real-time fashion. Secondly, the recent Scalable Video Coding (SVC) format opens opportunities for adapting video to the available (network) resources, yet the appropriate configuration of video encoding as well as real-time streaming adaptation are largely unaddressed research areas. Thirdly, while bandwidth reservation mechanisms in access/core networks do exist, service providers lack a means for guaranteeing QoS in the increasingly complex home networks (which they are not in full control of). In this paper we offer a broad view on these interrelated issues, by presenting the developments originating in a Flemish research project (including proof-of-concept demonstrations). From a developmental perspective, we propose an architecture combining a real-time video quality monitoring platform, on-the-fly adaptation (optimizing the video quality) and QoS reservation in a heterogeneous home network based on UPnP QoSxa0v3. From a research perspective, we propose a new subjective test procedure that revealed user preference for temporal scalability over quality scalability. In addition, an extensive study on optimizing HD SVC encoding in IPTV scenarios with fluctuating bandwidth showed that under certain bandwidth constraints (prohibiting sufficient fidelity) spatial scalability is a better option than quality scalability.

A scalable MPEG-4 wavelet-based visual texture compression system with optimized memory organization

The realization of new MPEG-4 functionality, applicable to three-dimensional graphics texture compression and image database access over the Internet, is demonstrated on a heterogeneous platform with several unique features. First, applying our system-level design methodologies effectively removes all data transfer and storage overhead that comprises the main bottleneck in the original system description. Second, a first-of-a-kind application specific solution, called Ozone, accelerates the embedded-zero-tree based encoding and is capable of compressing 30 color CIF images per second. The entire application is running on the Ozone coupled to a PC.

Local wavelet transform: a cost-efficient custom processor for space image compression

Thanks to its intrinsic scalability features, the wavelet transform has become increasingly popular as decorrelator in image compression applications. Throuhgput, memory requirements and complexity are important parameters when developing hardware image compression modules. An implementation of the classical, global wavelet transform requires large memory sizes and implies a large latency between the availability of the input image and the production of minimal data entities for entropy coding. Image tiling methods, as proposed by JPEG2000, reduce the memory sizes and the latency, but inevitably introduce image artefacts. The Local Wavelet Transform (LWT), presented in this paper, is a low-complexity wavelet transform architecture using a block-based processing that results in the same transformed images as those obtained by the global wavelet transform. The architecture minimizes the processing latency with a limited amount of memory. Moreover, as the LWT is an instruction-based custom processor, it can be programmed for specific tasks, such as push-broom processing of infinite-length satelite images. The features of the LWT makes it appropriate for use in space image compression, where high throughput, low memory sizes, low complexity, low power and push-broom processing are important requirements.

A novel CMOS-compatible, monolithically integrated line-scan hyperspectral imager covering the VIS-NIR range

Imec has developed a process for the monolithic integration of optical filters on top of CMOS image sensors, leading to compact, cost-efficient and faster hyperspectral cameras. Different prototype sensors are available, most notably a 600- 1000 nm line-scan imager, and two mosaic sensors: a 4x4 VIS (470-620 nm range) and a 5x5 VNIR (600-1000 nm). In response to the users’ demand for a single sensor able to cover both the VIS and NIR ranges, further developments have been made to enable more demanding applications. As a result, this paper presents the latest addition to imec’s family of monolithically-integrated hyperspectral sensors: a line scan sensor covering the range 470-900 nm. This new prototype sensor can acquire hyperspectral image cubes of 2048 pixels over 192 bands (128 bands for the 600- 900 nm range, and 64 bands for the 470-620 nm range) at 340 cubes per second for normal machine vision illumination levels.

An extremely compact and high-speed line-scan hyperspectral imager covering the SWIR range

SWIR (Short Wave Infrared) imaging can be of great use in precision agriculture, food processing and recycling industry, among other fields. However, hyperspectral SWIR cameras are costly and bulky, preventing their widespread deployment on the field. To answer the market need for compact and cost-efficient hyperspectral cameras covering the SWIR range, imec and SCD have joined efforts to develop a novel integration approach combining imec know-how in pixel level patterned thin film spectral filter technology, with SCD’s InGaAs technology. The here presented line-scan SWIR hyperspectral camera covers the 1.1-1.65 μm range with 100+ bands and a spectral resolution better than 10 nm. This imager uses a set of patterned Fabry-Pérot interferometers processed using semiconductor grade thin-film technology. The optical filters are then integrated directly on top of the sensing side of the InGaAs detector with high accuracy and with a minimum gap between filters and Focal Plane Array to limit cross-talk. The resulting line-scan camera, measuring only 70x62x60 mm and with a weight below 0.5kg, is the lightest and most compact SWIR hyperspectral camera on the market. Full sensor readout can be performed at up to 350 fps. An imecpatented SnapScan system with internal scanning was also developed, capable of acquiring data cubes of 640x512x128 pixels in a second. Maximum cube size is 1200x640x128. By selecting a subset of contiguous spectral bands and a reduced spatial resolution the sensor could be operated @ +1000 fps, for example enabling cube acquisitions of 320x512x64 in less than 300 ms.

Image registration software data correction algorithm for hyperspectral imager

In this paper we present a technique to accurately build a 3D hyperspectral image cube from a 2D imager overlaid with a wedge filter with up to hundreds of spectral bands, providing time-multiplexed data through scanning. The correctness of the spectral curve of each pixel in the physical scene, being the combination of its spectral information captured over different time stamps, is directly related to the alignment accuracy and scanning sensitivity. To overcome the accumulated alignment errors from scanning inaccuracies, frequency- dependent scaling from lens, spectral band separations and the imager’s spectral filter technology limitations, we have designed a new image alignment algorithm based on Random Sample Consensus (RANSAC) model fitting. It estimates many mechanical and optical system model parameters with image feature matching over the spectral bands, ensuring high immunity against the spectral reflectance variations, noise, motion-blur, blur etc. The estimated system model parameters are used to align the images captured over different bands in the 3D hypercube, reducing the average alignment error to 0.5 pixels, much below the alignment error obtained with state-of-the-art techniques. The image feature correspondences between the images in different bands of the same object are consistently produced, resulting in a hardware-software co-designed hyperspectral imager system, conciliating high quality and correct spectral curve responses with low-cost.