Yuyuan Ouyang

An Accelerated Linearized Alternating Direction Method of Multipliers

We present a novel framework, namely, accelerated alternating direction method of multipliers (AADMM), for acceleration of linearized ADMM. The basic idea of AADMM is to incorporate a multistep acceleration scheme into linearized ADMM. We demonstrate that for solving a class of convex composite optimization with linear constraints, the rate of convergence of AADMM is better than that of linearized ADMM, in terms of their dependence on the Lipschitz constant of the smooth component. Moreover, AADMM is capable of dealing with the situation when the feasible region is unbounded, as long as the corresponding saddle point problem has a solution. A backtracking algorithm is also proposed for practical performance.

Optimal Primal-Dual Methods for a Class of Saddle Point Problems

We present a novel accelerated primal-dual (APD) method for solving a class of deterministic and stochastic saddle point problems (SPPs). The basic idea of this algorithm is to incorporate a multistep acceleration scheme into the primal-dual method without smoothing the objective function. For deterministic SPP, the APD method achieves the same optimal rate of convergence as Nesterovs smoothing technique. Our stochastic APD method exhibits an optimal rate of convergence for stochastic SPP not only in terms of its dependence on the number of the iteration, but also on a variety of problem parameters. To the best of our knowledge, this is the first time that such an optimal algorithm has been developed for stochastic SPP in the literature. Furthermore, for both deterministic and stochastic SPP, the developed APD algorithms can deal with the situation when the feasible region is unbounded, as long as a saddle point exists. In the unbounded case, we incorporate the modified termination criterion introduced b...

Accelerated schemes for a class of variational inequalities

We propose a novel stochastic method, namely the stochastic accelerated mirror-prox (SAMP) method, for solving a class of monotone stochastic variational inequalities (SVI). The main idea of the proposed algorithm is to incorporate a multi-step acceleration scheme into the stochastic mirror-prox method. The developed SAMP method computes weak solutions with the optimal iteration complexity for SVIs. In particular, if the operator in SVI consists of the stochastic gradient of a smooth function, the iteration complexity of the SAMP method can be accelerated in terms of their dependence on the Lipschitz constant of the smooth function. For SVIs with bounded feasible sets, the bound of the iteration complexity of the SAMP method depends on the diameter of the feasible set. For unbounded SVIs, we adopt the modified gap function introduced by Monteiro and Svaiter for solving monotone inclusion, and show that the iteration complexity of the SAMP method depends on the distance from the initial point to the set of strong solutions. It is worth noting that our study also significantly improves a few existing complexity results for solving deterministic variational inequality problems. We demonstrate the advantages of the SAMP method over some existing algorithms through our preliminary numerical experiments.

Vectorial total variation regularisation of orientation distribution functions in diffusion weighted MRI

We propose a model for simultaneous Orientation Distribution Function (ODF) reconstruction and regularisation. The ODFs are represented by real spherical harmonic functions, and we propose to solve the spherical harmonic coefficients of the ODFs, with spatial regularisation by minimising the Vectorial Total Variation (VTV) of the coefficients. The proposed model also incorporates angular regularisation of the ODFs using Laplace-Beltrami operator on the unit sphere. A modified primal-dual hybrid gradient algorithm is applied to solve the model efficiently. The experimental results indicate better directional structures of reconstructed ODFs.

An enhanced approach for simultaneous image reconstruction and sensitivity map estimation in partially parallel imaging

We develop a variational model and a faster and robust numerical algorithm for simultaneous sensitivity map estimation and image reconstruction in partially parallel MR imaging with significantly under-sampled data. The proposed model uses a maximum likelihood approach to minimizing the residue of data fitting in the presence of independent Gaussian noise. The usage of maximum likelihood estimation dramatically reduces the sensitivity to the selection of model parameter, and increases the accuracy and robustness of the algorithm. Moreover, variable splitting based on the specific structure of the objective function, and alternating direction method of multipliers (ADMM) are used to accelerate the computation. The preliminary results indicate that the proposed method resulted in fast and robust reconstruction.

A spatial regularization framework of orientation diffusion functions using total variation and wavelet

We introduce a variational framework and a numerical method for simultaneous reconstruction and regularization of orientation distribution functions (ODF). The regularization is performed both angularly and spatially. The spatial regularization is based on the sparsity of MR images in finite difference domain and wavelet domain. The angular regularization is performed using Laplace-Beltrami operator on the unit sphere. The modified primal-dual hybrid gradient scheme is applied to solve the model efficiently. We apply the framework on two ODF reconstruction models. The experimental results indicate that with spatial and angular regularization in the process of reconstruction, we can get better directional structures of reconstructed ODFs.

Stochastic Accelerated Alternating Direction Method of Multipliers with Importance Sampling

In this paper, we incorporate importance sampling strategy into accelerated framework of stochastic alternating direction method of multipliers for solving a class of stochastic composite problems with linear equality constraint. The rates of convergence for primal residual and feasibility violation are established. Moreover, the estimation of variance of stochastic gradient is improved due to the use of important sampling. The proposed algorithm is capable of dealing with the situation, where the feasible set is unbounded. The experimental results indicate the effectiveness of the proposed method.