Stefano Belfiore

Concealment of whole-frame losses for wireless low bit-rate video based on multiframe optical flow estimation

In low bit-rate packet-based video communications, video frames may have very small size, so that each frame fills the payload of a single network packet; thus, packet losses correspond to whole-frame losses, to which the existing error concealment algorithms are badly suited and generally not applicable. In this paper, we deal with the problem of concealment of whole frame-losses, and propose a novel technique which is capable of handling this very critical case. The proposed technique presents other two major innovations with respect to the state-of-the-art: i) it is based on optical flow estimation applied to error concealment and ii) it performs multiframe estimation, thus optimally exploiting the multiple reference frame buffer featured by the most modern video coders such as H.263+ and H.264. If data partitioning is employed, by e.g., sending headers, motion vectors, and coding modes in prioritized packets as can be done in the DiffServ network model, the algorithm is capable of exploiting the motion vectors to improve the error concealment results. The algorithm has been embedded in the H.264 test model software, and tested under both independent and correlated packet loss models with parameters typical of the wireless environment. Results show that the proposed algorithm significantly outperforms other techniques by several dBs in peak signal-to-noise ratio (PSNR), provides good visual quality, and has a rather low complexity, which makes it possible to perform real-time operation with reasonable computational resources.

An error concealment algorithm for streaming video

A known problem in video streaming is that loss of a packet usually results into loss of a whole video frame. In this paper we propose an error concealment algorithm specifically designed to handle this sort of losses. The technique exploits information in a few past frames (namely the motion vectors) in order to estimate the forward motion vectors of the last received frame. This information is used to project the last frame onto an estimate of the missing frame. The algorithm has been tested on MPEG-2 video, providing very satisfactory results, and outperforming by several dBs in PSNR the concealment technique based on repetition of the last received frame.

Spatio-temporal video error concealment with perceptually optimized mode selection

We proposed a spatio-temporal error concealment algorithm for video transmission in an error-prone environment. The proposed technique employs motion vector estimation, edge-preserving interpolation, and texture analysis/synthesis. It has two main advantages with respect to existing methods, namely: i) it aims at optimizing the visual quality of the restored video, and not only PSNR; and ii) it employs an automatic mode selection algorithm in order to decide, on a macroblock basis, whether to use the spatial restoration, the temporal one, or a combination thereof. The algorithm has been applied to H26L video, providing satisfactory performance over a large set of operating conditions.

Robust and edge-preserving video error concealment by coarse-to-fine block replenishment

In this paper we propose a novel error concealment algorithm for video transmission over wireless networks potentially subject to packet erasures. In particular, we develop a technique for the replenishment of missing macroblocks, which aims at minimizing the impact of the lost data on the resulting video with respect to the human visual system. The proposed algorithm operates three reconstruction stages at different scales, by first recovering smooth large-scale patterns, then large-scale structures, and finally local edges in the lost macroblock. Experimental results show that the proposed algorithm achieves improved visual quality of the reconstructed frames with respect to other state-of-the-art techniques, as well as better PSNR results.

Spatiotemporal error concealment with optimized mode selection and application to H.264

In this paper we propose an error concealment algorithm for video transmission over lossy packet networks. The proposed technique is based on temporal and spatial interpolation. A sophisticated mode selection algorithm decides whether to employ the temporal or the spatial part, or a combination thereof, to estimate a missing macroblock; the selection does not rely on knowledge of the original coding modes. The resulting error concealment algorithm is designed so as to optimize both PSNR and visual quality of the restored video sequence, and employs directional interpolation and texture analysis/synthesis. The technique has been applied to H.264 coded video, providing satisfactory results on a number of test sequences.

An edge and texture preserving algorithm for video error concealment

We present a novel error concealment algorithm for block-based video transmission over error-prone networks. We develop a spatial error concealment technique, which combines edge-preserving interpolation and texture analysis and synthesis, providing a reconstruction of lost macroblocks optimized for visual perception. In particular, the algorithm recovers image edges by coarse-to-fine MAP estimation with a Markov random field prior, and replenishes lost textured areas with a texture synthesized from neighboring macroblocks. Experimental results show that texture synthesis allows achieving improved visual quality of the reconstructed area with respect to other state-of-the-art spatial concealment techniques.

Evaluation of Resiliency Solutions for Real-Time Multimedia Streams in a UTRAN Scenario

UMTS, as a third generation mobile communications system, has been developed to offer, in addition to traditional basic services - such as voice telephony, broader data connections and multimedia services. Channel reliability, the most common problem affecting mobile communications, strongly influences such new services and, in particular, the quality of multimedia streams. Nevertheless, current solutions do not foresee resiliency mechanisms suitable for real-time applications. In this paper we evaluate the benefits and effects of some resiliency tools onto the final quality of real-time multimedia streams in a UTRAN scenario: the proposed solutions span from FEC and ARQ to redundant video-information and enhanced packet-queue management. The analysis is performed by means of simulations modeling UTRAN FDD mode, with error events at the PDU level (simulated by means of a hidden Markov model)

Joint despeckling and edge detection of SAR images based on the Mumford-Shah functional

In this paper, we propose a joint despeckling and edge detection algorithm based on the Mumford-Shah functional, which accomplishes the image filtering and segmentation as a result of an analytical variational problem. This approach turns out to be well suited to jointly despeckle and segment SAR image data; the experimental results demonstrate that the proposed technique yields high quality despeckling without impairing critical image features and with the additional advantage to provide a detailed edge map