Zhen James Xiang

Screening Tests for Lasso Problems

This paper is a survey of dictionary screening for the lasso problem. The lasso problem seeks a sparse linear combination of the columns of a dictionary to best match a given target vector. This sparse representation has proven useful in a variety of subsequent processing and decision tasks. For a given target vector, dictionary screening quickly identifies a subset of dictionary columns that will receive zero weight in a solution of the corresponding lasso problem. These columns can be removed from the dictionary prior to solving the lasso problem without impacting the optimality of the solution obtained. This has two potential advantages: it reduces the size of the dictionary, allowing the lasso problem to be solved with less resources, and it may speed up obtaining a solution. Using a geometrically intuitive framework, we provide basic insights for understanding useful lasso screening tests and their limitations. We also provide illustrative numerical studies on several datasets.

Lasso screening with a small regularization parameter

Screening for lasso problems is a means of quickly reducing the size of the dictionary needed to solve a given instance without impacting the optimality of the solution obtained. We investigate a sequential screening scheme using a selected sequence of regularization parameter values decreasing to the given target value. Using analytical and empirical means we give insight on how the values of this sequence should be chosen and show that well designed sequential screening yields significant improvement in dictionary reduction and computational efficiency for lightly regularized lasso problems.

Tradeoffs in improved screening of lasso problems

Recently, methods of screening the lasso problem have been developed that use the target vector x to quickly identify a subset of columns of the dictionary that will receive zero weight in the solution. Current classes of screening tests are based on bounding the dual lasso solution within a sphere or the intersection of a sphere and a half space. Stronger tests are possible but are more complex and incur a higher computational cost. To investigate this, we determine the optimal screening test when the dual lasso solution is bounded within the intersection of a sphere and two half spaces, and empirically investigate the trade-off that this test makes between screening power and computational efficiency. We also compare its performance both in terms of rejection power and efficiency to existing test classes. The new test always has better rejection, and for an interesting range of regularization parameters, offers better computational efficiency.

Edge-Preserving Image Regularization Based on Morphological Wavelets and Dyadic Trees

Despite the tremendous success of wavelet-based image regularization, we still lack a comprehensive understanding of the exact factor that controls edge preservation and a principled method to determine the wavelet decomposition structure for dimensions greater than 1. We address these issues from a machine learning perspective by using tree classifiers to underpin a new image regularizer that measures the complexity of an image based on the complexity of the dyadic-tree representations of its sublevel sets. By penalizing unbalanced dyadic trees less, the regularizer preserves sharp edges. The main contribution of this paper is the connection of concepts from structured dyadic-tree complexity measures, wavelet shrinkage, morphological wavelets, and smoothness regularization in Besov space into a single coherent image regularization framework. Using the new regularizer, we also provide a theoretical basis for the data-driven selection of an optimal dyadic wavelet decomposition structure. As a specific application example, we give a practical regularized image denoising algorithm that uses this regularizer and the optimal dyadic wavelet decomposition structure.

Morphological wavelets and the complexity of dyadic trees

In this paper we reveal a connection between the coefficients of the morphological wavelet transform and complexity measures of dyadic tree representations of level sets. This leads to better understanding of the edge preserving property that has been discovered in both areas. As an immediate application, we examine a depth-adaptive soft thresholding scheme on morphological wavelet coefficients in which the threshold decays geometrically as the resolution increases. A greater decay rate gives greater preference towards unbalanced trees and this can control edge enhancement in denoised signals.

Sparse boosting

We propose a boosting algorithm that seeks to minimize the AdaBoost exponential loss of a composite classifier using only a sparse set of base classifiers. The proposed algorithm is computationally efficient and in test examples produces composite classifiers that are sparser and generalize as well those produced by Adaboost. The algorithm can be viewed as a coordinate descent method for the l1-regularized Adaboost exponential loss function.

Morphological wavelet transform with adaptive dyadic structures

We propose a two component method for denoising multidimensional signals, e.g. images. The first component uses a dynamic programing algorithm of complexity O (N log N) to find an optimal dyadic tree representation of a given multidimensional signal of N samples. The second component takes a signal with given dyadic tree representation and formulates the denoising problem for this signal as a Second Order Cone Program of size O (N). To solve the overall denoising problem, we apply these two algorithms iteratively to search for a jointly optimal denoised signal and dyadic tree representation. Experiments on images confirm that the approach yields denoised signals with improved PSNR and edge preservation.

Learning a wavelet tree for multichannel image denoising

We propose a new multichannel image denoising algorithm. To exploit important inter-channel dependencies, we first use dynamic programming to learn an explicit dyadic tree representation of the common structure of the channels. Based on this dyadic tree, optimal Haar wavelet thresholding is then applied to denoise the image. In addition to the original channels, the algorithm can employ multiple derived channels to improve tree learning. Experimental results confirm that the approach improves multichannel image denoising performance both in PSNR and in edge preservation.