Karl S. Ni

Image Superresolution Using Support Vector Regression

A thorough investigation of the application of support vector regression (SVR) to the superresolution problem is conducted through various frameworks. Prior to the study, the SVR problem is enhanced by finding the optimal kernel. This is done by formulating the kernel learning problem in SVR form as a convex optimization problem, specifically a semi-definite programming (SDP) problem. An additional constraint is added to reduce the SDP to a quadratically constrained quadratic programming (QCQP) problem. After this optimization, investigation of the relevancy of SVR to superresolution proceeds with the possibility of using a single and general support vector regression for all image content, and the results are impressive for small training sets. This idea is improved upon by observing structural properties in the discrete cosine transform (DCT) domain to aid in learning the regression. Further improvement involves a combination of classification and SVR-based techniques, extending works in resolution synthesis. This method, termed kernel resolution synthesis, uses specific regressors for isolated image content to describe the domain through a partitioned look of the vector space, thereby yielding good results

An Adaptable

We propose an image interpolation algorithm that is nonparametric and learning-based, primarily using an adaptive k-nearest neighbor algorithm with global considerations through Markov random fields. The empirical nature of the proposed algorithm ensures image results that are data-driven and, hence, reflect ldquoreal-worldrdquo images well, given enough training data. The proposed algorithm operates on a local window using a dynamic k -nearest neighbor algorithm, where k differs from pixel to pixel: small for test points with highly relevant neighbors and large otherwise. Based on the neighbors that the adaptable k provides and their corresponding relevance measures, a weighted minimum mean squared error solution determines implicitly defined filters specific to low-resolution image content without yielding to the limitations of insufficient training. Additionally, global optimization via single pass Markov approximations, similar to cited nearest neighbor algorithms, provides additional weighting for filter generation. The approach is justified in using a sufficient quantity of training per test point and takes advantage of image properties. For in-depth analysis, we compare to existing methods and draw parallels between intuitive concepts including classification and ideas introduced by other nearest neighbor algorithms by explaining manifolds in low and high dimensions.

k

Support vector machine (SVM) regression is considered for a statistical method of single frame superresolution in both the spatial and discrete cosine transform (DCT) domains. As opposed to current classification techniques, regression allows considerably more freedom in the determination of missing high-resolution information. In addition, since SVM regression approaches the superresolution problem as an estimation problem with a criterion of image correctness rather than visual acceptableness, its optimization results have better mean-squared error. With the addition of structure in the DCT coefficients, DCT domain image superresolution is further improved

-Nearest Neighbors Algorithm for MMSE Image Interpolation

This paper proposes the application of learned kernels in support vector regression to superresolution in the discrete cosine transform (DCT) domain. Though previous works involve kernel learning, their problem formulation is examined to reformulate the semi-definite programming problem of finding the optimal kernel matrix. For the particular application to superresolution, downsampling properties derived in the DCT domain are exploited to add structure to the learning algorithm. The advantage of the proposed method over other learning-based superresolution algorithms include specificity with regard to image content, structured consideration of energy compaction, and the added degrees of freedom that regression has over classification-based algorithms

Single Image Superresolution Based on Support Vector Regression

A novel technique for exploring the use of indexing metadata in improving coding efficiency is proposed in this paper. The technique uses an MPEG-7 descriptor as the basis for a fast mode decision algorithm for H.264/AVC encoders. The descriptor is used to form homogenous clusters for each frame, within which limited available coding modes for each macroblock are defined. The coding mode of an already coded macroblock that belongs to the same cluster in the same frame as well as the statistics of the coding modes of similar clusters in previous frames, are used for limiting the range of available coding modes within each cluster. The results show that the proposed algorithm is able to achieve an average of 47% time-saving when compared to the full search method and 21% when compared to the fast mode decision algorithm employed in the JM12.2 reference H.264 software encoder. In both cases, results yield only a small degradation in rate-distortion performance and a negligible loss in subjective quality.General rights This document is made available in accordance with publisher policies. Please cite only the published version using the reference above. Full terms of use are available: Explore Bristol Research is a digital archive and the intention is that deposited content should not be removed. However, if you believe that this version of the work breaches copyright law please contact open-access@bristol.ac.uk and include the following information in your message: • Your contact details • Bibliographic details for the item, including a URL • An outline of the nature of the complaint On receipt of your message the Open Access Team will immediately investigate your claim, make an initial judgement of the validity of the claim and, where appropriate, withdraw the item in question from public view.

Learning the Kernel Matrix for Superresolution

The proposed algorithm in this work provides superresolution for color images by using a learning based technique that utilizes both generative and discriminant approaches. The combination of the two approaches is designed with a stochastic classification-regression framework where a color image patch is first classified by its content, and then, based on the class of the patch, a learned regression provides the optimal solution. For good generalization, the classification portion of the algorithm determines the probability that the image patch is in a given class by modeling all possible image content (learned through a training set) as a Gaussian mixture, with each Gaussian of the mixture portraying a single class. The regression portion of the algorithm has been chosen to be a modified Support Vector Regression, where the kernel has been learned by solving a semi definite programming (SDP) and quadratically constrained quadratic programming (QCQP) problem. The SVR is further modified by scaling the training points in the SDP and QCQP problems by their relevance and importance to the examined regression. The result is a weighted average of different regressions depending on how much a single regression is likely to contribute, where advantages include reduced problem complexity, specificity with regard to image content, added degrees of freedom from a nonlinear approaches, and excellent generalization that a combined methodology has over its individual counterparts.

Exploiting MPEG-7 texture descriptors for fast H.264 mode decision

This work considers a combination classification-regression based framework with the proposal of using learned kernels in modified support vector regression to provide superresolution. The usage of both generative and discriminative learning techniques is examined first by assuming a distribution for image content for classification and then providing regression via semi-definite programming (SDP) and quadratically constrained quadratic programming (QCQP) problems. The advantage of the proposed method over other learning-based superresolution algorithms include reduced problem complexity, specificity with regard to image content, added degrees of freedom from the nonlinear approach, and excellent generalization that a combined methodology has over its individual counterparts.

Color Image Superresolution Based on a Stochastic Combinational Classification-Regression Algorithm

An empirical study of the domain of patch-based learning algorithms for image and video processing is conducted. As patch-based algorithms are commonly used, knowledge of the properties of fixed size image patches would prove particularly useful and interesting. We are concerned with investigating the overall distribution of vectorized patches of general images. A multivariate distribution model is proposed and analyzed using various techniques, which include univariate histograms and modified k-nearest neighbors. The model is verified and an application using the distribution model is introduced and compared.

Kernel Resolution Synthesis for Superresolution

Empirical filter designs generalize relationships inferred from training data to effect realistic solutions that conform well to the human visual system. Complex algorithms involving multiple linear regressions produce optimal results, but a single zero-phase filter yields comparable image quality at a fraction of the computational load. We propose an algorithm that builds a single symmetrical linear filter based purely on collected training data. Such a filtering technique balances the tradeoff between performance and complexity. Previous implementations of zero-phase interpolation filters as well as other learning-based interpolating algorithms are analyzed and examined. The proposed algorithm utilizes a Type-I symmetrical filter, an improvement and alternative over previous work on Type-II empirically-based interpolating filters. Given image training patches, the work discusses the enforcement of our filter properties while simultaneously drawing information from the training set. Additionally, we describe the implementation of the designed filter, its application, and related considerations. Finally, advantages of the proposed algorithm are analyzed.

A model for image patch-based algorithms

We explore a new technique for video frame rate up-conversion (FRUC). A noniterative multilayer motion estimation algorithm is investigated, based on spatio-temporal smoothness constraints. Our algorithms performance is at par with complicated iterative algorithms based on motion segmentation. For regions in the interpolated frame which cannot be motion compensated, we use an exemplar based video inpainting algorithm. To our knowledge, this is the first use of inpainting to fill in holes for FRUC.