Petri Haavisto

Video coding using motion compensation with polynomial motion vector fields

Abstract This paper describes a method for motion-compensated compression of video sequences. The proposed method enables very efficient encoding of video frames partitioned into arbitrary-shaped regions. The implementation described in this paper employs simple quadtree-based segmentation of video frames and a polynomial model of the motion vector field. Polynomial motion vector fields are estimated using an iterative minimization method. Application of complex polynomial motion vector fields to coding of video requires their compact encoding, especially for low bit-rate applications. The paper proposes novel algorithms enabling low-complexity encoding of polynomial motion vector fields which are optimal in least mean square error sense. Specifically, an efficient method for merging of regions resulting from quadtree segmentation is presented. The advantage of this method is its ability to achieve large reduction in the number of regions in typical coded sequences with only a minor increase of prediction error. An algorithm for reduction of the number of polynomial motion vector field coefficients is also presented. This algorithm enables adaptation of the motion vector field model to the complexity of motion in the coded scene. The problem of quantization of polynomial motion coefficients is also addressed. Performance of the proposed algorithms has been evaluated in a simple motion-compensated DCT codec and compared to an ITU-T H.263 codec. Simulations show that a consistent reduction of bit-rate in excess of 25% can be achieved over a wide range of sequences. Depending on the envisaged application the described algorithms can be used together with segmentation of desired accuracy and therefore provide simultaneously high coding efficiency as well as ability of content-based encoding of video.

GSM enhanced full rate speech codec

This paper describes the GSM enhanced full rate (EFR) speech codec that has been standardised for the GSM mobile communication system. The GSM EFR codec has been jointly developed by Nokia and University of Sherbrooke. It provides speech quality at least equivalent to that of a wireline telephony reference (32 kbit/s ADPCM). The EFR codec uses 12.2 kbit/s for speech coding and 10.6 kbit/s for error protection. Speech coding is based on the ACELP algorithm (algebraic code excited linear prediction). The codec provides substantial quality improvement compared to the existing GSM full rate and half rate codecs. The old GSM codecs lack wireline quality even in error-free channel conditions, while the EFR codec provides wireline quality not only for error-free conditions but also for the most typical error conditions. With the EFR codec, wireline quality is also sustained in the presence of background noise and in tandem connections (mobile to mobile calls).

Motion compensated video sequence interpolation using digital image warping

A new method for temporal interpolation of video sequences is proposed. For each pair of original frames a new frame is created by averaging frames obtained by motion compensated warping of original frames. The parameters of the geometrical transformation applied to original frames are determined by the motion in the sequence. The advantage of the proposed method is that it can compensate for various motion types such as translations, rotations, and scaling. The primary application of the presented algorithm is the frame interpolation of video sequences coded at very low bitrates where the warping scheme ran also he used for motion compensated prediction. In that case the decoder does not have to perform motion estimation but can use motion vectors from encoder for frame interpolation.<<ETX>>

Noise compensation for speech recognition in car noise environments

A noise compensation algorithm for HMM based speech recognition systems, which utilizes the parallel model combination concept, is presented. The algorithm was tested using the TIDIGITS database with artificially added car noise. Very promising results were obtained. The results show that at -10 dB SNR the recognition accuracy could be improved from 34% to 89%. The noise compensation algorithm was also tested using a database which was recorded in a car. Improved performance was obtained, but the improvement; was clearly smaller than with the artificially added noise.

GSM EFR based multi-rate codec family

This paper describes a multi-rate codec family developed as a potential candidate for the GSM adaptive multi-rate (AMR) codec standard. The codec family consists of the GSM enhanced full rate (EFR) codec and lower bit-rate extensions thereof. The codec family consists of several codecs, i.e., modes that have different bit-rate partitionings between source coding and error protection. All the source codecs use the same ACELP-method (algebraic code excited linear predictive coding) used also in the GSM EFR codec. The codec operates at gross bit-rates of 22.8 kbit/s in the GSM full rate (FR) channel and 11.4 kbit/s in the GSM half rate (HR) channel. In the full rate channel, the codec provides improved error robustness over the GSM enhanced full rate (EFR) codec. It extends wireline quality (equal to or better than G.726-32 ADPCM) to poor channel error conditions with low C/I-ratios of 7 dB or even below. When operated in the half rate channel, the codec provides improved channel capacity while still providing wireline quality at high C/I-ratios above 16-19 dB.

An improved noise compensation algorithm for speech recognition in noise

When moving a speech recognition system whose models were trained in a clean laboratory condition to real environments, one of most important issues is how to modify the models according to the changing environments. Using an HMM composition technique we present an algorithm to compensate the dynamic cepstral coefficients for HMM based speech recognition systems in noise environments. Noise compensation for acceleration parameters and for dynamic parameters which are calculated using longer linear regression are discussed. The experimental results show a clear improvement when the algorithm was applied to a speech database recorded in a car. A noise compensation system based realtime speech recognizer using the TMS320C40 was implemented and achieves a good performance in noisy environments.

Temporal image sequence prediction using motion field interpolation

Abstract A new method for motion-compensated temporal prediction of image sequences is proposed. Motion vector fields in natural scenes should possess two basic properties. First, the field should be smoothly varying within moving objects to compensate for nonrigid or rotational motion, and scaling of objects. Second, the field should be discontinuous along the boundaries of the objects. In the proposed method the motion vector field is modelled using finite element methods and interpolated using adaptive interpolators to satisfy the above-stated requirements. This is particularly important when only very sparse estimates of motion vector fields are available in the decoder due to bit-rate constraints limiting the amount of overhead information that can be transmitted. The proposed prediction method can be applied for low-bit-rate video coding in conventional codecs based on motion-compensated prediction and transform coding, as well as in model-based codecs. The performance of the proposed method is compared with standard motion-compensated prediction based on block matching. It is shown that for simple video telephony scenes a reduction of more than 30% in the energy of the prediction error can be achieved with an unchanged number of transmitted motion vectors and with only a modest increase in computational complexity. When implemented in an H.261 codec the new prediction method can improve the peak SNR 1–2 dB producing a significant visual improvement.

Name dialing-how useful is it?

Progress in automatic speech recognition technology has resulted in an increasing amount of deployed applications. Typically, the measure of success has been the amount of deployed or sold units and it has been much more difficult to evaluate the real user benefit from the technology. Actually, the usability, or usefulness, has largely remained an open issue. In this paper we focus on name dialing and discuss its usefulness from different angles, with a strong emphasis on mass market use and inexperienced users. As a concept, name dialing brings us back from where telephony started: an operator assisted way of making calls without a need to remember numbers. In essence, name dialing offers a solution to a minor inconvenience-using the directory. Even the arguably biggest advantage of name dialing, simplified car usage, is still less significant than the various concerns of users. Until time and further technical improvements alleviate the main concerns, usage of name dialing will remain as an occasional, rather than a primary, way of making calls.

Motion estimation and representation for arbitrarily shaped image regions

This paper discusses the problem of motion compensated prediction in a segmentation based video coding scheme. The problem is considered in the framework of a generic video coder utilizing spatial image segmentation and a polynomial model of the motion vector field for each image region. It is shown that very impressive reductions in prediction error can be achieved by this approach compared to traditional block matching. The cost of encoding the motion information is also addressed. We derive an analytical method for the reduction of the number of motion coefficients, which is optimal is the sense of the least increase in the prediction error. Finally, we derive a low complexity method for motion assisted merging of regions resulting from spatial segmentation which leads to a dramatic reduction in the number of regions.

A method for motion compensated video prediction

A new method for temporal prediction of image sequences is proposed. In the proposed method the motion of a selected set of individual pixels, instead of blocks, is estimated. The resulting low resolution estimate of the motion vector field is interpolated using an adaptive scheme. This yields piecewise smooth motion vector field. The proposed prediction scheme can compensate various types of motion and deformations. It can also be incorporated in current hybrid video compression systems with little additional complexity and no change in the bit stream syntax. It is shown that the proposed scheme improves the peak signal-to-noise ratio of the prediction by up to 1.9 dB compared to conventional methods.