Hung M. Phan

Full length article: The rate of linear convergence of the Douglas-Rachford algorithm for subspaces is the cosine of the Friedrichs angle

The Douglas-Rachford splitting algorithm is a classical optimization method that has found many applications. When specialized to two normal cone operators, it yields an algorithm for finding a point in the intersection of two convex sets. This method for solving feasibility problems has attracted a lot of attention due to its good performance even in nonconvex settings. In this paper, we consider the Douglas-Rachford algorithm for finding a point in the intersection of two subspaces. We prove that the method converges strongly to the projection of the starting point onto the intersection. Moreover, if the sum of the two subspaces is closed, then the convergence is linear with the rate being the cosine of the Friedrichs angle between the subspaces. Our results improve upon existing results in three ways: First, we identify the location of the limit and thus reveal the method as a best approximation algorithm; second, we quantify the rate of convergence, and third, we carry out our analysis in general (possibly infinite-dimensional) Hilbert space. We also provide various examples as well as a comparison with the classical method of alternating projections.

Linear convergence of the Douglas–Rachford method for two closed sets

In this paper, we investigate the Douglas–Rachford method (DR) for two closed (possibly nonconvex) sets in Euclidean spaces. We show that under certain regularity conditions, the DR converges locally with -linear rate. In convex settings, we prove that the linear convergence is global. Our study recovers recent results on the same topic.

Restricted Normal Cones and Sparsity Optimization with Affine Constraints

The problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an NP-complete problem that is typically solved numerically via convex heuristics or nicely behaved nonconvex relaxations. In this paper we consider the elementary method of alternating projections (MAP) for solving the sparsity optimization problem without employing convex heuristics. In a parallel paper we recently introduced the restricted normal cone which generalizes the classical Mordukhovich normal cone and reconciles some fundamental gaps in the theory of sufficient conditions for local linear convergence of the MAP algorithm. We use the restricted normal cone together with the notion of superregularity, which is inherently satisfied for the affine sparse optimization problem, to obtain local linear convergence results with estimates for the radius of convergence of the MAP algorithm applied to sparsity optimization with an affine constraint.

Variational Analysis of Marginal Functions with Applications to Bilevel Programming

This paper pursues a twofold goal. First goal is to derive new results on generalized differentiation in variational analysis focusing mainly on a broad class of intrinsically nondifferentiable marginal/value functions. Then the results established in this direction are applied to deriving necessary optimality conditions for the optimistic version of bilevel programs, which occupy a remarkable place in optimization theory and its various applications. We obtain new sets of optimality conditions in both smooth and nonsmooth settings of finite-dimensional and infinite-dimensional spaces.

On Slater's condition and finite convergence of the Douglas---Rachford algorithm for solving convex feasibility problems in Euclidean spaces

The Douglas–Rachford algorithm is a classical and very successful method for solving optimization and feasibility problems. In this paper, we provide novel conditions sufficient for finite convergence in the context of convex feasibility problems. Our analysis builds upon, and considerably extends, pioneering work by Spingarn. Specifically, we obtain finite convergence in the presence of Slater’s condition in the affine-polyhedral and in a hyperplanar-epigraphical case. Various examples illustrate our results. Numerical experiments demonstrate the competitiveness of the Douglas–Rachford algorithm for solving linear equations with a positivity constraint when compared to the method of alternating projections and the method of reflection–projection.

The Method of Alternating Relaxed Projections for Two Nonconvex Sets

The Method of Alternating Projections (MAP), a classical algorithm for solving feasibility problems, has recently been intensely studied for nonconvex sets. However, intrinsically available are only local convergence results: convergence occurs if the starting point is not too far away from solutions to avoid getting trapped in certain regions. Instead of taking full projection steps, it can be advantageous to underrelax, i.e., to move only part way towards the constraint set, in order to enlarge the regions of convergence.In this paper, we thus systematically study the Method of Alternating Relaxed Projections (MARP) for two (possibly nonconvex) sets. Complementing our recent work on MAP, we establish local linear convergence results for the MARP. Several examples illustrate our analysis.

Tangential extremal principles for finite and infinite systems of sets II: applications to semi-infinite and multiobjective optimization

This paper contains selected applications of the new tangential extremal principles and related results developed in Mordukhovich and Phan (Math Program 2011) to calculus rules for infinite intersections of sets and optimality conditions for problems of semi-infinite programming and multiobjective optimization with countable constraints.

Linear convergence of the generalized Douglas–Rachford algorithm for feasibility problems

In this paper, we study the generalized Douglas–Rachford algorithm and its cyclic variants which include many projection-type methods such as the classical Douglas–Rachford algorithm and the alternating projection algorithm. Specifically, we establish several local linear convergence results for the algorithm in solving feasibility problems with finitely many closed possibly nonconvex sets under different assumptions. Our findings not only relax some regularity conditions but also improve linear convergence rates in the literature. In the presence of convexity, the linear convergence is global.

Stadium Norm and Douglas-Rachford Splitting: A New Approach to Road Design Optimization

The basic optimization problem of road design is quite challenging due to an objective function that is the sum of nonsmooth functions and the presence of set constraints. In this paper, we model and solve this problem by employing the Douglas-Rachford splitting algorithm. This requires a careful study of new proximity operators related to minimizing area and to the stadium norm. We compare our algorithm to a state-of-the-art projection algorithm. Our numerical results illustrate the potential of this algorithm to significantly reduce cost in road design.

Rated extremal principles for finite and infinite systems

In this article we introduce new notions of local extremality for finite and infinite systems of closed sets and establish the corresponding extremal principles for them, here called rated extremal principles. These developments are in the core geometric theory of variational analysis. We present their applications to calculus and optimality conditions for problems with infinitely many constraints.