Marie-Line Alberi Morel

Single-Image Super-Resolution via Linear Mapping of Interpolated Self-Examples

This paper presents a novel example-based single-image superresolution procedure that upscales to high-resolution (HR) a given low-resolution (LR) input image without relying on an external dictionary of image examples. The dictionary instead is built from the LR input image itself, by generating a double pyramid of recursively scaled, and subsequently interpolated, images, from which self-examples are extracted. The upscaling procedure is multipass, i.e., the output image is constructed by means of gradual increases, and consists in learning special linear mapping functions on this double pyramid, as many as the number of patches in the current image to upscale. More precisely, for each LR patch, similar self-examples are found, and, because of them, a linear function is learned to directly map it into its HR version. Iterative back projection is also employed to ensure consistency at each pass of the procedure. Extensive experiments and comparisons with other state-of-the-art methods, based both on external and internal dictionaries, show that our algorithm can produce visually pleasant upscalings, with sharp edges and well reconstructed details. Moreover, when considering objective metrics, such as Peak signal-to-noise ratio and Structural similarity, our method turns out to give the best performance.

Neighbor embedding based single-image super-resolution using Semi-Nonnegative Matrix Factorization

This paper describes a novel method for single-image super-resolution (SR) based on a neighbor embedding technique which uses Semi-Nonnegative Matrix Factorization (SNMF). Each low-resolution (LR) input patch is approximated by a linear combination of nearest neighbors taken from a dictionary. This dictionary stores low-resolution and corresponding high-resolution (HR) patches taken from natural images and is thus used to infer the HR details of the super-resolved image. The entire neighbor embedding procedure is carried out in a feature space. Features which are either the gradient values of the pixels or the mean-subtracted luminance values are extracted from the LR input patches, and from the LR and HR patches stored in the dictionary. The algorithm thus searches for the K nearest neighbors of the feature vector of the LR input patch and then computes the weights for approximating the input feature vector. The use of SNMF for computing the weights of the linear approximation is shown to have a more stable behavior than the use of LLE and lead to significantly higher PSNR values for the super-resolved images.

Super-resolution using neighbor embedding of back-projection residuals

In this paper we present a novel algorithm for neighbor embedding based super-resolution (SR), using an external dictionary. In neighbor embedding based SR, the dictionary is trained from couples of high-resolution and low-resolution (LR) training images, and consists of pairs of patches: matching patches (m-patches), which are used to match the input image patches and contain only low-frequency content, and reconstruction patches (r-patches), which are used to generate the output image patches and actually bring the high-frequency details. We propose a novel training scheme, where the m-patches are extracted from enhanced back-projected interpolations of the LR images and the r-patches are extracted from the back-projection residuals. A procedure to further optimize the dictionary is followed, and finally nonnegative neighbor embedding is considered at the SR algorithm stage. We consider singularly the various elements of the algorithm, and prove that each of them brings a gain on the final result. The complete algorithm is then compared to other state-of-the-art methods, and its competitiveness is shown.

Compact and coherent dictionary construction for example-based super-resolution

This paper presents a new method to construct a dictionary for example-based super-resolution (SR) algorithms. Example-based SR relies on a dictionary of correspondences of low-resolution (LR) and high-resolution (HR) patches. Having a fixed, prebuilt, dictionary, allows to speed up the SR process; however, in order to perform well in most cases, we need to have big dictionaries with a large variety of patches. Moreover, LR and HR patches often are not coherent, i.e. local LR neighborhoods are not preserved in the HR space. Our designed dictionary learning method takes as input a large dictionary and gives as an output a dictionary with a “sustainable” size, yet presenting comparable or even better performance. It firstly consists of a partitioning process, done according to a joint k-means procedure, which enforces the coherence between LR and HR patches by discarding those pairs for which we do not find a common cluster. Secondly, the clustered dictionary is used to extract some salient patches that will form the output set.

Optimization of unicast services transmission for broadcast channels in practical situations

The problem of constellation shaping for broadcast transmission in degraded channels remains a challenge. This is especially so when a single source communicates simultaneously with two receivers using a finite dimension constellation. This paper focuses on a practical situation where unicast service to each user is transmitted over broadcast channels. We investigate the optimization of an achievable rate closure region by using non-uniform constellations issued from superimposition of high-rate information on low-rate information and by using a nonequiprobable distribution of the transmitted symbols. The achievable rate region is derived for a two-user additive white Gaussian noise (AWGN) broadcast channel and for finite input pulse amplitude modulation (PAM) constellations. A noticeable shaping gain up to 3.5 dB maximum was shown on signal-to-noise ratio (SNR), compared with the equiprobable distribution of transmitted symbols obtained for a 4-PAM constellation when achievable rates are maximized over the probability distribution of channel input signals and the constellation shape.

Video super-resolution via sparse combinations of key-frame patches in a compression context

In this paper we present a super-resolution (SR) method for upscaling low-resolution (LR) video sequences, that relies on the presence of periodic high-resolution (HR) key frames, and validate it in the context of video compression. For a given LR intermediate frame, the HR details are retrieved patch-by-patch by taking sparse linear combinations of patches found in the neighbor key frames. The performance of the video SR algorithm is assessed in a scheme where only some key frames from an original HR sequence are directly encoded; the remaining intermediate frames are down-sampled to LR and encoded as well, with a possibly different quantization parameter. SR is then finally employed to upscale these frames. For comparison, we consider the best case where the whole original HR sequence is encoded. With respect to this case, our SR-based approach is shown to bring a certain gain for low bit-rates (consistent when all frames are encoded independently), i.e. when a poor encoding can actually benefit of the special processing of the intermediate frames, so proving that video SR can be an useful tool in realistic scenarios.

K-WEB: Nonnegative dictionary learning for sparse image representations

This paper presents a new nonnegative dictionary learning method, to decompose an input data matrix into a dictionary of nonnegative atoms, and a representation matrix with a strict ℓ0-sparsity constraint. This constraint makes each input vector representable by a limited combination of atoms. The proposed method consists of two steps which are alternatively iterated: a sparse coding and a dictionary update stage. As for the dictionary update, an original method is proposed, which we call K-WEB, as it involves the computation of k WEighted Barycenters. The so designed algorithm is shown to outperform other methods in the literature that address the same learning problem, in different applications, and both with synthetic and “real” data, i.e. coming from natural images.

Variable FEC decoding delay and playout slowdown method for low start delay and fast channel change for video streaming in DVB-SH mobile broadcast systems

In the digital video broadcasting satellite services to handhelds (DVB-SH) system for broadcasting multimedia services to mobile receivers, a multiprotocol-encapsulation inter-burst forward error correction (MPE-IFEC) technique has been introduced to mitigate the deep signal fading events which primarily occur in the land-mobile satellite channel. It is based on a clever organization of data and parities with long interleaving. The media data is thus buffered at the client side until all interleaved data or redundancies are received before decoding begins. The more the data is protected against loss, the more video impairment decreases and latency increases. Latency negatively impacts the playout start time and switching time. A tradeoff is usually necessary between the video quality and the forward error correction (FEC) decoding latency. Such a tradeoff is realized in a static mode. Alternatively, in this paper we investigate how an adaptive media playout and variable FEC decoding delay scheme can provide low latency while providing good video quality. We then evaluate user quality of experience when the method is used in DVB-SH broadcast networks.