Defeng Sun

A new look at smoothing Newton methods for nonlinear complementarity problems and box constrained variational inequalities

Abstract.In this paper we take a new look at smoothing Newton methods for solving the nonlinear complementarity problem (NCP) and the box constrained variational inequalities (BVI). Instead of using an infinite sequence of smoothing approximation functions, we use a single smoothing approximation function and Robinson’s normal equation to reformulate NCP and BVI as an equivalent nonsmooth equation H(u,x)=0, where H:ℜ2n→ℜ2n, u∈ℜn is a parameter variable and x∈ℜn is the original variable. The central idea of our smoothing Newton methods is that we construct a sequence {zk=(uk,xk)} such that the mapping H(·) is continuously differentiable at each zk and may be non-differentiable at the limiting point of {zk}. We prove that three most often used Gabriel-Moré smoothing functions can generate strongly semismooth functions, which play a fundamental role in establishing superlinear and quadratic convergence of our new smoothing Newton methods. We do not require any function value of F or its derivative value outside the feasible region while at each step we only solve a linear system of equations and if we choose a certain smoothing function only a reduced form needs to be solved. Preliminary numerical results show that the proposed methods for particularly chosen smoothing functions are very promising.

Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities

The smoothing Newton method for solving a system of nonsmooth equations F(x) = 0, which may arise from the nonlinear complementarity problem, the variational inequality problem or other problems, can be regarded as a variant of the smoothing method. At the kth step, the nonsmooth function F is approximated by a smooth function f(.,∈ κ ), and the derivative of f(.,∈ κ ) at x k is used as the Newton iterative matrix. The merits of smoothing methods and smoothing Newton methods are global convergence and convenience in handling. In this paper, we show that the smoothing Newton method is also superlinearly convergent if F is semismooth at the solution and f satisfies a Jacobian consistency property. We show that most common smooth functions, such as the Gabriel-More function, have this property. As an application, we show that for box constrained variational inequalities if the involved function is P- uniform, the iteration sequence generated by the smoothing Newton method will converge to the unique solution of the problem globally and superlinearly (quadratically).

A Quadratically Convergent Newton Method for Computing the Nearest Correlation Matrix

The nearest correlation matrix problem is to find a correlation matrix which is closest to a given symmetric matrix in the Frobenius norm. The well-studied dual approach is to reformulate this problem as an unconstrained continuously differentiable convex optimization problem. Gradient methods and quasi-Newton methods such as BFGS have been used directly to obtain globally convergent methods. Since the objective function in the dual approach is not twice continuously differentiable, these methods converge at best linearly. In this paper, we investigate a Newton-type method for the nearest correlation matrix problem. Based on recent developments on strongly semismooth matrix valued functions, we prove the quadratic convergence of the proposed Newton method. Numerical experiments confirm the fast convergence and the high efficiency of the method.

Semismooth Matrix-Valued Functions

Matrix-valued functions play an important role in the development of algorithms for semidefinite programming problems. This paper studies generalized differential properties of such functions related to nonsmooth-smoothing Newton methods. The first part of this paper discusses basic properties such as the generalized derivative, Rademachers theorem, B-derivative, directional derivative, and semismoothness. The second part shows that the matrix absolute-value function, the matrix semidefinite-projection function, and the matrix projective residual function are strongly semismooth.

Hankel Matrix Rank Minimization with Applications to System Identification and Realization

We introduce a flexible optimization framework for nuclear norm minimization of matrices with linear structure, including Hankel, Toeplitz, and moment structures and catalog applications from diverse fields under this framework. We discuss various first-order methods for solving the resulting optimization problem, including alternating direction methods of multipliers, proximal point algorithms, and gradient projection methods. We perform computational experiments to compare these methods on system identification problems and system realization problems. For the system identification problem, the gradient projection method (accelerated by Nesterovs extrapolation techniques) and the proximal point algorithm usually outperform other first-order methods in terms of CPU time on both real and simulated data, for small and large regularization parameters, respectively, while for the system realization problem, the alternating direction method of multipliers, as applied to a certain primal reformulation, usuall...

Complementarity Functions and Numerical Experiments on Some Smoothing Newton Methods for Second-Order-Cone Complementarity Problems

Two results on the second-order-cone complementarity problem are presented. We show that the squared smoothing function is strongly semismooth. Under monotonicity and strict feasibility we provide a new proof, based on a penalized natural complementarity function, for the solution set of the second-order-cone complementarity problem being bounded. Numerical results of squared smoothing Newton algorithms are reported.

On NCP-Functions

In this paper we reformulate several NCP-functions for the nonlinear complementarity problem (NCP) from their merit function forms and study some important properties of these NCP-functions. We point out that some of these NCP-functions have all the nice properties investigated by Chen, Chen and Kanzow [2] for a modified Fischer-Burmeister function, while some other NCP-functions may lose one or several of these properties. We also provide a modified normal map and a smoothing technique to overcome the limitation of these NCP-functions. A numerical comparison for the behaviour of various NCP-functions is provided.

The Strong Second-Order Sufficient Condition and Constraint Nondegeneracy in Nonlinear Semidefinite Programming and Their Implications

For a locally optimal solution to the nonlinear semidefinite programming problem, under Robinsons constraint qualification, the following conditions are proved to be equivalent: the strong second-order sufficient condition and constraint nondegeneracy; the nonsingularity of Clarkes Jacobian of the Karush-Kuhn-Tucker system; the strong regularity of the Karush-Kuhn-Tucker point; and others.

Semismooth homeomorphisms and strong stability of semidefinite and Lorentz complementarity problems

Based on an inverse function theorem for a system of semismooth equations, this paper establishes several necessary and sufficient conditions for an isolated solution of a complementarity problem defined on the cone of symmetric positive semidefinite matrices to be strongly regular/stable. We show further that for a parametric complementarity problem of this kind, if a solution corresponding to a base parameter is strongly stable, then a semismooth implicit solution function exists whose directional derivatives can be computed by solving certain affine problems on the critical cone at the base solution. Similar results are also derived for a complementarity problem defined on the Lorentz cone. The analysis relies on some new properties of the directional derivatives of the projector onto the semidefinite cone and the Lorentz cone.

Löwner's Operator and Spectral Functions in Euclidean Jordan Algebras

We study analyticity, differentiability, and semismoothness of Lowners operator and spectral functions under the framework of Euclidean Jordan algebras. In particular, we show that many optimization-related classical results in the symmetric matrix space can be generalized within this framework. For example, the metric projection operator over any symmetric cone defined in a Euclidean Jordan algebra is shown to be strongly semismooth. The research also raises several open questions, whose answers would be of strong interest for optimization research.