Michael Anthony Isnardi

Bilateral filtering-based optical flow estimation with occlusion detection

Using the variational approaches to estimate optical flow between two frames, the flow discontinuities between different motion fields are usually not distinguished even when an anisotropic diffusion operator is applied. In this paper, we propose a multi-cue driven adaptive bilateral filter to regularize the flow computation, which is able to achieve the smoothly varied optical flow field with highly desirable motion discontinuities. First, we separate the traditional one-step variational updating model into a two-step filtering-based updating model. Then, employing our occlusion detector, we reformulate the energy functional of optical flow estimation by explicitly introducing an occlusion term to balance the energy loss due to the occlusion or mismatches. Furthermore, based on the two-step updating framework, a novel multi-cue driven bilateral filter is proposed to substitute the original anisotropic diffusion process, and it is able to adaptively control the diffusion process according to the occlusion detection, image intensity dissimilarity, and motion dissimilarity. After applying our approach on various video sources (movie and TV) in the presence of occlusion, motion blurring, non-rigid deformation, and weak textureness, we generate a spatial-coherent flow field between each pair of input frames and detect more accurate flow discontinuities along the motion boundaries.

Encoding for Compatibility and Recoverability in the ACTV System

This paper describes an advanced compatible television (ACTV) system intended for the single-channel transmission of widescreen EDTV images. Existing NTSC receivers display a selected 4: 3 portion of the widescreen image with standard NTSC resolution. However, a new widescreen receiver, tuned to the same 6-MHz RF channel, displays a widescreen image with a resolution in excess of 400 lines/picture height in both spatial dimensions. To produce the NTSC-compatible signal, the original widescreen high-definition signal is separated into four components: (1) the main NTSC signal with the side panel low frequencies time compressed into the horizontal overscan regions, (2) the time-expanded side panel high frequencies, (3) the extended horizontal-luminance detail, and (4) the extended vertical-temporal luminance detail. Components 2 and 3 are quadrature modulated at low amplitude by a new phase-controlled subcarrier at 3.1 MHz. Component 4 is inserted in quadrature with the RF picture carrier. This paper discusses how the four components are processed to insure full compatibility at the NTSC receiver and complete recoverability at the ACTV receiver.

Decoding issues in the ACTV system

The advanced compatible television (ACTV) system is a proposal for the single-channel transmission of widescreen enhanced-definition television (EDTV) images. A widescreen high-definition source is encoded into a signal that is NTSC-compatible. Existing NTSC receivers display a selected 4:3 portion of the widescreen image with standard NTSC resolution. A new widescreen receiver is proposed, tuned to the same 6 MHz RF channel, that displays a widescreen image with a resolution in excess of 400 lines/picture height in both spatial dimensions. The encoding process is reviewed and the recovery of various signal components to produce the widescreen image in the ACTV receiver is discussed. >

Reduced resolution residual coding for H.264-based compression system

The current state-of-the-art video coding standard, H.264, offers substantially better compression performance than earlier video coding standards. However, even with these gains, better compression is still desirable. This paper describes a macroblock-based mixed resolution video encoding system. By reducing the spatial resolution of some macroblock residuals intelligently, our preliminary experiments show up to 20% improvement in video compression efficiency compared to H.264 while maintaining good subjective video quality

Multidimensional interpretation of NTSC encoding and decoding

NTSC color encoding and decoding are analyzed in spatiotemporal frequency space. The mapping of important 1-D electrical frequencies, such as the color subcarrier, into 3-D space is discussed. Spatiotemporal frequencies that contribute to cross-color and cross-luma artifacts become readily apparent in the 3-D spectral domain. The use of matched pre- and postfilters to decrease and theoretically eliminate luma/chroma crosstalk is discussed. Lowpass/bandpass, line-comb, and field-comb implementations are examined. >

A half D1 MPEG-4 encoder on the BSP-15 DSP

In this paper, we present the work on implementation of a half-D1interlaced MPEG-4 encoder with Equator Technology DSP chip, BSP-15. The BSP-15 DSP consists mainly of a VLIW core, Co-processors, and media I/O interfaces. The encoder utilizes several BSP-15 functional blocks in parallel. In general, the VLIW performs pixel procesing that is computationally intensive. The VLx coprocessor completes variable length coding. Further parallelism is obtained by pre-loading data cache and doubling data buffers. Given the DSP processing power and real time requirements, a complexity control scheme is implemented. A frame-level quantization scheme with quality and rate control is employed. The current implementation for video at 30 fps consumes about 90% of the chip performance at a bit rate ~2Mbps.