On the global convergent of an inexact quasi-Newton conditional gradient method for constrained nonlinear systems
aa r X i v : . [ m a t h . O C ] J un On the global convergent of an inexact quasi-Newton conditionalgradient method for constrained nonlinear systems
M.L.N. Gon¸calves ∗ F.R. Oliveira ∗ June 6, 2018
Abstract
In this paper, we propose a globally convergent method for solving constrained nonlinear sys-tems. The method combines an efficient Newton conditional gradient method with a derivative-free and nonmonotone linesearch strategy. The global convergence analysis of the proposedmethod is established under suitable conditions, and some preliminary numerical experimentsare given to illustrate its performance.
Keywords: constrained nonlinear systems; inexact quasi-Newton method; conditional gradientmethod; Newton conditional gradient method; nonmonotone and derivative-free linesearch; globalconvergence.
Let F : Ω → R n be a continuously differentiable nonlinear function and Ω ⊂ R n be an open set.Consider the problem of finding a vector x ∈ Ω such that F ( x ) = 0 . (1)Among various methods for solving unconstrained nonlinear system (1), the Newton method isregarded as one of the most effective. Basically, it generates a sequence { x k } in such a way that x k +1 = x k + s k , ∀ k ≥ , where the Newton direction s k is computed by solving the linear system F ′ ( x k ) s k = − F ( x k ) . (2) ∗ IME, Universidade Federal de Goi´as, Goiˆania, GO 74001-970, Brazil. (E-mails: [email protected] and [email protected] ). The work of these authors was supported in part by CAPES, FAPEG/CNPq/PRONEM-201710267000532, and CNPq Grants 406975/2016-7 and 302666/2017-6.
1e refer the reader to [1, 6, 11, 14] where convergence results of the Newton method and its variantshave been discussed.Consider now the constrained nonlinear system F ( x ) = 0 , x ∈ C, (3)where C ⊂ Ω is a nonempty convex compact set. Various numerical methods for solving (3)have been recently proposed and studied in the literature. Many of them are combinations ofNewton methods with some strategies taking into account the constraint set. Strategies based onprojections, trust region, active set and gradient methods have been used; see, e.g., [3, 5, 10, 18,19, 20, 24, 25, 26, 31, 32, 33].A Newton conditional gradient (Newton-CondG) method was proposed in [15] (see [16] for itsinexact version) to compute approximate solutions of (3). Briefly speaking, the latter methodconsists of computing a Newton step and later applying a conditional gradient (CondG) procedurein order to get the Newton iterative back to the feasible set. In general, the CondG method and itsvariants require, at each iteration, to minimize a linear function over the constraint set, which, ingeneral, is significantly simpler than the projection step arising in many proximal-gradient methods.Moreover, depending on the application, linear optimization oracles may provide solutions withspecific characteristics leading to important properties such as sparsity and low-rank; see, e.g.,[13, 17] for a discussion on this subject. As shown in [15, 16], the Newton-CondG method as wellas its inexact version performed well and compared favorably with other methods. However, noglobalization strategy was considered in [15, 16] and hence only local convergence analyses of thesemethods were presented.Therefore, the aim of this article is to propose and analyze a version global of the methodin [16]. It is worth pointing out that, in many cases, the strategy of globalization may becomethe methods more robustness. Usually, the global convergence of the methods for solving (1) isobtained by ensuring the decreasing of the merit function f ( x ) = 12 k F ( x ) k . (4)See, for example, [9, 20, 25, 26, 28]. However, for the inexact quasi-Newton method, the direction s k , which is an approximate solution of (2) with F ′ ( x k ) replaced by an approximation of it, maynot be a descent direction of (4). Hence, in this case, only nonmonotone globalization strategycan be considered. Almost all of these strategies are based on approximate norm descent conditionproposed in [22]. This condition can be described as follows: a sequence of feasible iterates { x k } isgenerated in such a way that the following nonmonotone condition is satisfied k F ( x k +1 ) k ≤ (1 + η k ) k F ( x k ) k , ∀ k ≥ , (5)where { η k } is a positive sequence such that ∞ X k =0 η k ≤ η < ∞ . (6)2ased on this condition, Morini proposed in [28] (see also [25]) a more general criterion, whichreplaced (5) by the following inequalities: k F ( x k + π ( s k , λ k )) k ≤ (1 − α (1 + λ k )) k F ( x k ) k , (7)or k F ( x k + π ( s k , λ k )) k ≤ (1 + η k − αλ k ) k F ( x k ) k , (8)with η k as in (6), λ k ∈ (0 , α ∈ (0 , , and π ( s k , λ k ) is a suitable direction. We mention thatthe global method to be proposed here is based on the latter globalization criterion. In order toillustrate the robustness and efficiency of the new method, we report some preliminary numericalexperiments on a set of box-constrained nonlinear systems and compare its performance with thelocal FD-INL-CondG method in [16] and the constrained dogleg method [4].The paper is organized as follows. Section 2 presents the global inexact quasi-Newton condi-tional gradient method as well as its analysis of global convergence. Some preliminary numericalexperiments for the proposed method are reported in Section 3. Notation:
Throughout this paper, the Jacobian matrix of F at x ∈ Ω is denoted by F ′ ( x ). Theinner product and its associated Euclidean norm in R n be denoted by h· , ·i and k · k , respectively.The i -th component of a vector x is indicated by ( x ) i . Our goal in this section is to present as well as analyze a new iterative method, namely the globalinexact quasi-Newton conditional gradient (GIQN-CondG) method, for solving (3).
This subsection describes the GIQN-CondG method, which is obtained basically by combining theinexact Newton-like conditional gradient method proposed in [16] with a strategy of globalizationsimilar to the one in [28]. As already mentioned, in many cases, the strategy of globalization maybecome the methods more robustness.The GIQN-CondG method is formally described as follows.
GIQN-CondG method(S.0) (Initialization) Let x ∈ C , α , σ ∈ (0 , η k satisfying (6) and { θ j } ⊂ [0 , ∞ ) be given, andset k = 0. (S.1) (Termination criterion) If F ( x k ) = 0 , then stop .3 S.2) (Computation of the approximate quasi-Newton direction) Choose an invertible approxi-mation M k of F ′ ( x k ). For the residual r k ∈ R n compute a duple ( s k , y k ) ∈ R n × R n suchthat M k s k = − F ( x k ) + r k , y k = x k + s k . (9) (S.3) (CondG procedure) If y k ∈ C , set ˜ s k = s k ; otherwise, let˜ s k = CondG( y k , x k , θ k k s k k ) − x k . (10) (S.4) (Backtracking process) Set s + = ˜ s k . If k ˜ s k k 6 = 0 set s − = − ˜ s k else s − = − s k .(S.4.1) Set λ = 1.(S.4.2) Repeat(S.4.2.1) If π ( s k , λ ) := λs + satisfies (7), go to (S.5) .Else if π ( s k , λ ) := λs − satisfies x k + π ( s k , λ ) ∈ C and (7), go to (S.5) .(S.4.2.2) If k s + k 6 = 0 and π ( s k , λ ) := λs + satisfies (8), go to (S.5) .Else if π ( s k , λ ) := λs − satisfies x k + π ( s k , λ ) ∈ C and (8), go to (S.5) .(S.4.2.3) Set λ = σλ. (S.5) (Computation of new iterative) Set λ k = λ , p k = π ( s k , λ k ), x k +1 = x k + p k . (S.6) (Update) Set k ← k + 1 , and go to (S.1) . end Let us now describe the CondG procedure.
CondG procedure z = CondG( y, x, ε ) P0.
Set z = x and t = 1. P1.
Use the linear optimization (LO) oracle to compute an optimal solution u t of g ∗ t = min u ∈ C {h z t − y, u − z t i} . (11) P2. If g ∗ t ≥ − ε , set z = z t and stop the procedure; otherwise, compute α t ∈ (0 ,
1] and z t +1 as α t := min (cid:26) , − g ∗ t k u t − z t k (cid:27) , z t +1 = z t + α t ( u t − z t ) . P3.
Set t ← t + 1, and go to P1 . 4 nd procedureRemark 1. i) There are different choices for, or way to build, the matrix M k and the residual r k in(S.2), which originate variations of the GIQN-CondG method. For example, by taking r k = 0 and M k = F ′ ( x k ) (resp. M k = F ′ ( x ) ), we obtain a globalized version of the Newton (resp. modifiedNewton) conditional gradient method proposed in [15] (resp. [16]). We refer the reader to [7, 8, 29]for some derivative-free approaches for building M k . ii) Note that, the CondG procedure in (S.3)is used in order to obtain an approximate projection of the inexact quasi-Newton iteration y k tothe feasible set C , and as a consequence, a possible feasible direction ˜ s k . More discussions of thisspecialized CondG procedure can be found in [16, Remark 1]. iii) The Backtracking process given in(S.4) is well-defined, since its repeat-loop in (S.4.2) terminates in a finite number of steps. Indeed,as F is a continuous function and η k is a positive scalar for every k , then there exists a smallenough scalar ˆ λ > such that the following inequality is satisfied ( F ( x k + λs )) i ≤ (1 + η k − αλ ) ( F ( x k )) i , for λ ∈ (0 , ˆ λ ) and i = 1 , . . . , n . Consequently, condition (8) trivially holds. Moreover, since s − may not be a feasible search direction, it is necessary to check the feasibility of the new iteratein this case. iv) The GIQN-CondG method is closely related to the quasi-Newton method in [25].However, they differ mainly in two respects. First, our approach computes an inexact projection bythe CondG procedure, whereas the method in [25] requires, in each iteration, two exact projections.As already mentioned, in many applications, computing the projection step may be more difficultthan solving (11) . Second, in [25], the linear system (9) is solved exactly (i.e., r k = 0 for every k ≥ ), which may be expensive and difficult for medium and large scale problems. In this subsection, we present global convergence results for the GIQN-CondG method. Specifically,we show that the sequence {k F ( x k ) k} is convergent and, under stronger assumptions, it convergesto zero. Moreover, the global convergence of the sequence { x k } is also established.The following lemma guarantees that the approximate norm descent condition (5) is satisfiedfor every k and establishes some upper bounds for k F ( x k ) k . Lemma 1.
Let { x k } and { λ k } be generated sequences by GIQN-CondG method.i) For all k ≥ , condition (5) holds and k F ( x k +1 ) k ≤ e η k F ( x ) k , (12) αλ k k F ( x k ) k ≤ (1 + η k ) k F ( x k ) k − k F ( x k +1 ) k . (13)5 i) Let { k m } , with m ≥ and k ≥ , be the indices of the iterates satisfying (7) , i.e., k F ( x k m ) k ≤ (1 − α (1 + λ k m − )) k F ( x k m − ) k . (14) Then, k F ( x k m ) k ≤ (1 − α ) m e η k F ( x ) k . (15) Proof.
See proofs of [28, Theorem 4.2] and [25, Lemma 3.1] for itens (i) and (ii), respectively.The next lemma presents a basic property of the CondG procedure, whose proof can be foundin [15, Lemma 4].
Lemma 2.
For any y, ˜ y ∈ R n , x, ˜ x ∈ C and µ ≥ , we have k CondG ( y, x, µ ) − CondG (˜ y, ˜ x, k ≤ k y − ˜ y k + p µ. The following assumption is needed in order to investigate the global convergence of the se-quences { x k } and {k F ( x k ) k} . Assumption 1.
Approximation M k of F ′ ( x k ) is invertible for every k ≥ . Moreover, assume that M k and the residual r k satisfy k M k − k ≤ c , k r k k ≤ c k F ( x k ) k , ∀ k ≥ , (16) for some scalars c > and c ≥ . Remark 2. i) It is easy to see that the first equality in (9) and Assumption 1 imply k s k k ≤ c (1 + c ) k F ( x k ) k . ii) See, for example, [2, 21] for more details in how to built matrices M k such that the Assumption 1trivially holds. Assumption 1 is essential to provide estimaties for { ˜ s k } and { p k } , which will be useful in theglobal analysis of GIQN-CondG method. Lemma 3.
Let { x k } , {k F ( x k ) k} and { λ k } be generated sequences by GIQN-CondG method. As-sume that Assumption 1 holds and { θ k } ⊂ [0 , β / where β ≥ . Then, for every k ≥ ,i) k ˜ s k k ≤ c (1 + β )(1 + c ) k F ( x k ) k ; ii) k p k k ≤ c (1 + β )(1 + c ) λ k k F ( x k ) k . roof. i) First of all, if ˜ s k = s k , then from Remark 2(i) follows that k ˜ s k k ≤ c (1 + c ) k F ( x k ) k , which, combined with the fact that β ≥
0, implies the inequality of item (i). On the other hand, if˜ s k = 0, the desired inequality trivially holds. Finally, let us consider the case where0 = ˜ s k = CondG( y k , x k , θ k k s k k ) − x k . Using the fact that CondG( x, x,
0) = x for all x ∈ C , Lemma 2 and the second equality in (9), weobtain k ˜ s k k = k CondG( y k , x k , θ k k s k k ) − CondG( x k , x k , k ≤ k y k − x k k + p θ k k s k k ≤ (1 + β ) k s k k , where the last inequality follows from √ θ k ≤ β . Hence, from Remark 2(i), we conclude the proveof the item.ii) It follows from GIQN-CondG method that k p k k = k π ( s k , λ k ) k = λ k k ˜ s k k , which, combined with item (i), proves the inequality of item (ii).The next theorem discusses the global convergence of the sequences {k F ( x k ) k} , { λ k k F ( x k ) k} and { x k } as well as the case in which the GIQN-CondG method fails to solve (3). Theorem 4.
Let { x k } , {k F ( x k ) k} and { λ k } be generated sequences by GIQN-CondG method.Then,i) The sequence {k F ( x k ) k} is convergent;ii) The sequence { λ k k F ( x k ) k} is convergent and such that lim k →∞ λ k k F ( x k ) k = 0; (17) iii) If (7) is satisfied for infinitely many k , then lim k →∞ k F ( x k ) k = 0 . Now, if k F ( x k ) k ≤k F ( x k +1 ) k for all k sufficiently large, then lim k →∞ λ k = 0 and lim k →∞ k F ( x k ) k 6 = 0 ;iv) If in addition Assumption 1 holds and { θ k } ⊂ [0 , β / where β ≥ , then the sequence { x k } is convergent.Proof. The proofs of the items (i), (ii) and (iii) follows the same pattern as proofs of items (i), (ii)and (iii) of [25, Theorem 3.2]. 7 v) Our goal is to prove that { x k } is a Cauchy sequence and hence it converges. Before, let us firstprove that P ∞ k =0 λ k k F ( x k ) k is a convergent series. It follows from (13) that ∞ X k =0 λ k k F ( x k ) k ≤ ∞ X k =0 (cid:18) (1 + η k ) α k F ( x k ) k − α k F ( x k +1 ) k (cid:19) = ∞ X k =0 α ( k F ( x k ) k − k F ( x k +1 ) k ) + ∞ X k =0 η k α k F ( x k +1 ) k≤ α k F ( x ) k + ∞ X k =0 η k α k F ( x k +1 ) k , which, combined with (6) and (12), yields ∞ X k =0 λ k k F ( x k ) k ≤ α k F ( x ) k + ∞ X k =0 η k α e η k F ( x ) k ≤ (cid:18) α + ηα e η (cid:19) k F ( x ) k . Since λ k k F ( x k ) k is positive for every k , we conclude that P ∞ k =0 λ k k F ( x k ) k is convergent. Hence,from Lemma 3, we obtain ∞ X k =0 k p k k ≤ c (1 + β )(1 + c ) ∞ X k =0 λ k k F ( x k ) k < ∞ . On the other hand, let m ≥ l and consider k x m − x l k = k p l + p l +1 + . . . + p m − k ≤ ∞ X k = l k p k k = ∞ X k =0 k p k k − l − X k =0 k p k k . (18)Taking the limit in (18) as l goes to infinity, we have k x m − x l k tends to zero. This implies thatfor every ε >
0, there exists l sufficiently large such that k x m − x l k ≤ ε , for all m ≥ l . Therefore, { x k } is a Cauchy sequence and the proof of the item is complete.For the last two results we will assume that the Jacobian F ′ is Lipschitz continuous. Assumption 2.
Assume that the Jacobian F ′ of F satisfies k F ′ ( x ) − F ′ ( y ) k ≤ L k x − y k , ∀ x, y ∈ C. We now prove that, under additional assumptions, the {k F ( x k ) k} converges to zero. Theorem 5.
Let { x k } be sequence generated by GIQN-CondG method. Assume that Assumptions 1and 2 hold. If for all k sufficiently large the step ˜ s k satisfies k F ′ ( x k )˜ s k + F ( x k ) k ≤ δ k F ( x k ) k , ≤ δ < − α, (19) then lim k →∞ k F ( x k ) k = 0 . roof. It follows from Lemma 3 that k ˜ s k k ≤ c (1 + β )(1 + c ) k F ( x k ) k . (20)Let us now prove that (7) holds for infinitely many k . Since k ˜ s k k 6 = 0 (see (19)), we have F ( x k + λ k ˜ s k ) = F ( x k ) + Z F ′ ( x k + tλ k ˜ s k ) λ k ˜ s k dt = (1 − λ k ) F ( x k ) + λ k ( F ′ ( x k )˜ s k + F ( x k )) + Z ( F ′ ( x k + tλ k ˜ s k ) − F ′ ( x k )) λ k ˜ s k dt. Using (19), (20) and Assumption 2, we obtain k F ( x k + λ k ˜ s k ) k ≤ (1 − λ k ) k F ( x k ) k + λ k δ k F ( x k ) k + L λ k k ˜ s k k ≤ (1 − λ k + λ k δ ) k F ( x k ) k + L c (1 + β )(1 + c )] λ k k F ( x k ) k , which, combined with the fact that λ k ∈ (0 , k F ( x k + λ k ˜ s k ) k ≤ (1 − λ k + λ k δ ) k F ( x k ) k + L c (1 + β )(1 + c )] λ k k F ( x k ) k = (cid:18) − λ k + λ k δ + L c (1 + β )(1 + c )] λ k k F ( x k ) k (cid:19) k F ( x k ) k . As consequence of (17), we conclude that there exists a ¯ k such that ( L/ c (1+ β )(1+ c )] λ k k F ( x k ) k <α for k ≥ ¯ k . Hence, condition (7) holds for k ≥ ¯ k if1 − λ k + λ k δ + α ≤ − α (1 + λ k ) . or, equivalently, λ k (1 − α − δ ) ≥ α. Therefore, since (19) implies 0 < α/ (1 − α − δ ) <
1, we conclude, from steps (S.4.1) and (S.4.2)of the GIQN-CondG method, that condition (7) holds for every k ≥ ¯ k , and hence the statement ofthe lemma trivially follows from Theorem 4(iii).Note that, the first equation in (9) and second inequality in (16) imply that k M k s k + F ( x k ) k ≤ c k F ( x k ) k , for every k ≥
0. Hence, condition (19) trivially holds if ˜ s k = s k and M k = F ′ ( x k ) forall k sufficiently large, and c < − α . In the next corollary, we give conditions in which (19) alsoholds when M k is only an approximate of F ′ ( x k ).9 orollary 6. Let { x k } be sequence generated by GIQN-CondG method. Assume that Assumptions 1and 2 hold. If for all k sufficiently large the steps s k and ˜ s k satisfy ˜ s k = s k and k F ′ ( x k ) M − k k ≤ ρ, k I − F ′ ( x k ) M − k k ≤ υ, υ + ρc < − α, (21) where ρ > , υ ≥ and c is given in Assumption 1, then lim k →∞ k F ( x k ) k = 0 . Proof.
By the first equality in (9), ˜ s k = s k , (21) and Assumption 1 follow that k F ′ ( x k )˜ s k + F ( x k ) k = k − F ′ ( x k ) M − k ( F ( x k ) − r k ) + F ( x k ) k≤ k ( I − F ′ ( x k ) M − k ) F ( x k ) k + k F ′ ( x k ) M − k r k k≤ ( υ + ρc ) k F ( x k ) k . Hence, the statement of the corollary now follows from Theorem 5 with δ = υ + ρc . This section reports results of some preliminary numerical experiments obtained by applying theGIQN-CondG method to solve 17 test problems of the form (1) with C = { x ∈ R n : l ≤ x ≤ u } , where l, u ∈ R n , see Table 1. We tested the following variants of the GIQN-CondG method whichdiffer in the way that the approximation matrices M k ’s are built. In the FD-GIQN-CondG method,the matrices M k ’s were approximated by finite differences, whereas in the BSU-GIQN-CondG andBPU-GIQN-CondG methods, we used the Broyden-Schubert Update [8, 29] and the Bogle-PerkinsUpdate [7], respectively. For the latter two methods, we also used the strategy of periodically (i.e., k = 0 and mod( k − ,
5) = 0) approximating by finite differences the matrices M k ’s. We comparethe performance of above variants with the local FD-INL-CondG method [16] and the constrainedDogleg solver (CoDoSol), which is a MATLAB package based on the constrained Dogleg method[4], and available on the web site http://codosol.de.unifi.it . In the latter two methods, the Jacobianmatrices were approximated by finite differences. The parameters of the CoDoSol were selectedas recommended by the authors, see [4, Subsection 4.1]. All numerical results were obtain usingMATLAB R2016a on a 2.5GHz Intel(R) i5 with 6GB of RAM and Windows 7 ultimate operationsystem.For all methods, the starting points were defined as x ( γ ) = l + 0 . γ ( u − l ), where γ ≥ k F ( x k ) k ∞ ≤ − , and a failure wasdeclared if either no progress was detected or the total number of iterations exceeded 300. Inthe variants of the GIQN-CondG method, the initialization data were α = 10 − , σ = 0 . η k =0 . k (100 + k F ( x ) k ) and θ k = 10 − , for every k ≥
0, and the linear systems in (9) were solvedby direct methods, i.e., r k = 0 for all k ≥
0. The CondG procedure stopped when either thestopping criterion given in P2 is satisfied or the maximum of 300 iterations are performed. Notethat, in this application, subproblem (11) has a closed-form solution, i.e., if ( z t ) i − ( y ) i ≥
0, then10 u t ) i = ( l ) i ; otherwise ( u t ) i = ( u ) i . The parameters of the FD-INL-CondG method were chosen asthe corresponding one of its global version (i.e., GIQN-CondG method).Tables 2 and 3 display all numerical results obtained. The methods were compared on the totalnumber of iterates (It), number of F-evaluation (Fe) and CPU time in seconds (Time). The symbol“ ∗ ” indicates a failure, whereas k F k ∞ and ζ ( q ) are the infinity norm of F at the final iterate x k and ζ · q , respectively. In Table 2, the number of F-evaluations of the FD-INL-CondG methodwas omitted in all cases, because it is always equal to the number of iterations plus one.From Table 2, in terms of amount of problems solved, we can see that the FD-GIQN-CondGmethod was more robust than the FD-INL-CondG method and CoDoSol. This because the FD-GIQN-CondG method solved 47 problems of a total of 51, whereas the FD-INL-CondG method andCoDoSol sucessfully ended in 42 problems. Regarding to the number of iterations, we observe thatthe FD-GIQN-CondG and FD-INL-CondG methods had similar performance and, in general, theyrequired less iterations than CoDoSol. Similar efficiency can also be observed for the number ofF-evaluations of the FD-GIQN-CondG method and CoDoSol. The CPU times of the three methodswere practically the same.Comparing the methods in which F ′ is not evaluated at each iteration, we can observe, fromTable 3, that the BSU-GIQN-CondG and BPU-GIQN-CondG methods were similar in terms ofrobustness and efficiency. Note also that the slower convergence rates of the BSU-GIQN-CondGand BPU-GIQN-CondG methods are compensated by their smaller CPU times per iteration. Such abehavior is due to the fact that quasi-Newton approximations of M ′ k s are computationally cheaper.As a summary of the previous discussion, we can say that the GIQN-CondG method seems tobe a robust and efficient tool for solving box-constrained systems of nonlinear equations. References [1]
Argyros, I. K., and Hilout, S.
Estimating upper bounds on the limit points of majorizing sequencesfor newton’s method.
Numer. Algorithms 62 , 1 (2013), 115–132.[2]
Barzilai, J., and Borwein, J. M.
Two-point step size gradient methods.
IMA J. Numer. Anal. 8 ,1 (1988), 141–148.[3]
Bellavia, S., Macconi, M., and Morini, B.
An affine scaling trust-region approach to bound-constrained nonlinear systems.
Appl. Num. Math. 44 , 3 (2003), 257 – 280.[4]
Bellavia, S., Macconi, M., and Pieraccini, S.
Constrained dogleg methods for nonlinear systemswith simple bounds.
Comput. Optim. Appl. 53 , 3 (2012), 771–794.[5]
Bellavia, S., and Morini, B.
Subspace trust-region methods for large bound-constrained nonlinearequations.
SIAM J. Numer. Anal. 44 , 4 (2006), 1535–1555.[6]
Birgin, E. G., Kreji´c, N., and Mart´ınez, J. M.
Globally convergent inexact quasi-newton methodsfor solving nonlinear systems.
Numer. Algorithms 32 , 2 (2003), 249–260.
Problem Name and source n BoxPb 1 Effati-Grosan problem 2 [30] 2 [ − , R = 0 .
935 [30] 2 [0 , , π ]Pb 4 Brown’s almost linear system [12, 14.1.5] 5 [ − , − , − , − , − , − , − , − , − , − , − , − , − , . Bogle, I. D. L., and Perkins, J. D.
A new sparsity preserving quasi-Newton update for solvingnonlinear equations.
SIAM J. Sci. Statist. Comput. 11 , 4 (1990), 621–630.[8]
Broyden, C. G.
The convergence of an algorithm for solving sparse nonlinear systems.
Math. Comp.25 (1971), 285–294.[9]
Cruz, W. L., and Raydan, M.
Nonmonotone spectral methods for large-scale nonlinear systems.
Optim. Methods Softw. 18 , 5 (2003), 583–599.[10]
Echebest, N., Schuverdt, M. L., and Vignau, R. P.
A derivative-free method for solving box-constrained underdetermined nonlinear systems of equations.
Appl. Math. Comput. 219 , 6 (2012),3198–3208.[11]
Ferreira, O. P., and Gonc¸alves, M. L. N.
Local convergence analysis of inexact Newton-likemethods under majorant condition.
Comput. Optim. Appl. 48 , 1 (2011), 1–21.[12]
Floudas, C. A., et al.
Handbook of Test Problems in Local and Global Optimization, in: NonconvexOptimization and Its Applications, Volume 33, Kluwer Academic, Dordrecht, 1999 .[13]
Freund, R., and Grigas, P.
New analysis and results for the Frank-Wolfe method.
Math. Program. (2014), 1–32.[14]
Gonc¸alves, M. L. N.
Inexact Gauss-Newton like methods for injective-overdetermined systems ofequations under a majorant condition.
Numer. Algorithms 72 , 2 (2016), 377–392. Gonc¸alves, M. L. N., and Melo, J. G.
A Newton conditional gradient method for constrainednonlinear systems.
J. Comput. Appl. Math. 311 (2017), 473–483.[16]
Gonc¸alves, M. L. N., and Oliveira, F. R.
An inexact Newton-like condi-tional gradient method for constrained nonlinear systems.
Appl. Num. Math. (2018),https://doi.org/10.1016/j.apnum.2018.05.006 .[17]
Jaggi, M.
Revisiting Frank-Wolfe: Projection-free sparse convex optimization. In
Proceedings of the30th International Conference on Machine Learning (ICML-13) (2013), vol. 28, pp. 427–435.[18]
Kanzow, C.
An active set-type Newton method for constrained nonlinear systems. In
Complementar-ity: Applications, Algorithms and Extensions , M. C. Ferris, O. L. Mangasarian, and J.-S. Pang, Eds.,vol. 50 of
Appl. Optim.
Springer US, 2001, pp. 179–200.[19]
Kozakevich, D. N., Martinez, J. M., and Santos, S. A.
Solving nonlinear systems of equationswith simple constraints.
Comput. Appl. Math. 16 (1997), 215–235.[20]
La Cruz, W.
A projected derivative-free algorithm for nonlinear equations with convex constraints.
Optim. Methods Softw. 29 , 1 (2014), 24–41.[21]
La Cruz, W., Mart´ınez, J. M., and Raydan, M.
Spectral residual method without gradientinformation for solving large-scale nonlinear systems of equations.
Math. Comp. 75 , 255 (2006), 1429–1448.[22]
Li, D. H., and Fukushima, M.
A derivative-free line search and global convergence of Broyden-likemethod for nonlinear equations.
Optim. Methods Softw. 13 , 3 (2000), 181–201.[23]
Lukˇsan, L., and Vlˇcek, J.
Sparse and partially separable test problems for unconstrained andequality constrained optimization.
Technical Report N. 767, Institute of Computer Science, Academyof Sciences of the Czech Republic (1999).[24]
Macconi, M., Morini, B., and Porcelli, M.
Trust-region quadratic methods for nonlinear systemsof mixed equalities and inequalities.
Appl. Num. Math. 59 , 5 (2009), 859 – 876.[25]
Marini, L., Morini, B., and Porcelli, M.
Quasi-newton methods for constrainednonlinear systems: complexity analysis and applications.
Comput. Optim. Appl. (2018),https://doi.org/10.1007/s10589–018–9980–7.[26]
Martinez, M. J.
Quasi-inexact-Newton methods with global convergent for solving constrained non-linear systems.
Nonlinear Anal. 30 , 1 (1997), 1–7.[27]
Mor´e, J. J., Garbow, B. S., and Hillstrom, K. E.
Testing unconstrained optimization software.
ACM Trans. Math. Softw. 7 , 1 (1981), 17–41.[28]
Morini, B., Porcelli, M., and Toint, P. L.
Approximate norm descent methods for constrainednonlinear systems.
Math. Comput. 87 , 311 (2018), 1327–1351.[29]
Schubert, L. K.
Modification of a quasi-Newton method for nonlinear equations with a sparseJacobian.
Math. Comp. 24 (1970), 27–30. Tsoulos, I. G., and Stavrakoudis, A.
On locating all roots of systems of nonlinear equations insidebounded domain using global optimization methods.
Nonlinear Anal. Real World Appl. 11 , 4 (2010),2465–2471.[31]
Wang, P., and Zhu, D.
An inexact derivative-free levenberg–marquardt method for linear inequalityconstrained nonlinear systems under local error bound conditions.
Appl. Math. Comput. 282 (2016),32–52.[32]
Zhang, Y., and Zhu, D.-t.
Inexact Newton method via Lanczos decomposed technique for solvingbox-constrained nonlinear systems.
Appl. Math. Mech. 31 , 12 (2010), 1593–1602.[33]
Zhu, D.
An affine scaling trust-region algorithm with interior backtracking technique for solvingbound-constrained nonlinear systems.
J. Comput. App. Math. 184 , 2 (2005), 343 – 361.
FD-GIQN-CondG FD-INL-CondG CoDoSolProblem γ It Fe Time/ k F k ∞ It Time/ k F k ∞ It Fe Time/ k F k ∞ Pb 1 1 * * *2 29 31 7 . − . −
7) * 6 8 3 . − . − . − . −
7) 19 7 . − . −
7) 7 9 1 . − . − . − . −
12) 13 1 . . −
12) 13 14 2 . − . − . − . −
8) 23 2 . − . −
8) 23 24 4 . − . − . − . −
8) 33 2 . − . −
8) 33 34 1 . − . − . − . −
11) 3 4 . − . −
11) 3 4 1 . − . − . − . − . −
16) 5 8 4 . − . − . − . −
16) 3 1 . − . . − . − . . − . − . −
11) 11 3 . − . −
8) 4 5 5 . − . − . − . −
8) * 6 8 1 . − . − . − . −
7) * *1 * * *2 * * *Pb 6 0 18 19 2 . . −
7) * *1 18 19 1 . . −
8) 25 1 . . −
7) 17 18 1 . . − . . −
8) 14 1 . . −
7) 16 19 1 . . − . − . −
8) 13 4 . − . −
8) 16 17 4 . − . − . − . −
7) 10 1 . − . −
7) 12 13 1 . − . − . − . −
8) 11 1 . − . −
8) 13 14 1 . − . − . − . −
13) 34 5 . − . −
13) *2 18 19 1 . − . −
8) 18 2 . − . −
8) *3 9 10 7 . − . −
14) 9 1 . − . −
14) 10 11 9 . − . − . − . −
10) 13 2 . − . −
10) *2 9 10 2 . − . −
8) 9 1 . − . −
8) 10 12 1 . − . − . − . −
12) 7 6 . − . −
12) 7 8 7 . − . − . − . −
13) 15 2 . − . −
13) 21 28 3 . − . − . − . −
8) 5 6 . − . −
8) 10 14 1 . − . − . − . −
9) 8 1 . − . −
9) 10 12 1 . − . − . − . −
7) 27 3 . − . −
7) 32 33 2 . . − . − . −
7) 26 3 . − . −
7) 31 32 5 . − . − . − . −
7) 25 2 . − . −
7) 29 30 4 . − . − . − . −
7) 14 2 . − . −
7) 15 16 4 . − . − . − . −
7) 13 1 . − . −
7) 14 15 1 . − . − . − . −
7) 13 1 . − . −
7) 14 15 1 . − . − . . −
8) 10 1 . . −
8) 16 17 3 . . − . . −
10) 10 1 . . −
10) 16 17 2 . . − . . −
8) 9 1 . . −
8) 15 16 2 . . − . . −
8) 10 1 . . −
8) 17 18 3 . . − . . −
11) 10 1 . . −
11) 16 17 3 . . − . . −
8) 9 1 . . −
8) 16 17 2 . . − . . −
8) 15 2 . . −
8) 19 20 2 . . − . . −
7) 13 1 . . −
7) 16 17 2 . . − . . −
11) 11 1 . . −
11) 13 14 1 . . . − . −
7) 1 7 . − . −
7) 10 11 7 . . − . . −
8) 2 1 . . −
8) 10 11 7 . . − . − . −
7) 1 6 . − . −
7) 9 10 6 . . − . . −
8) * *1 14 15 4 . . −
8) 14 4 . . −
8) 7 8 2 . . − . . −
8) 15 7 . . −
10) 17 18 5 . . − BSU-GIQN-CondG BPU-GIQN-CondGProblem γ It Fe Time/ k F k ∞ It Fe Time/ k F k ∞ Pb 1 1 12 13 2 . − . −
9) 32 33 2 . − . − . − . −
7) 64 66 7 . − . − . − . −
9) 12 13 1 . − . − . − . −
11) 16 17 1 . − . − . − . −
7) 31 32 1 . − . − . − . −
8) 44 45 1 . − . − . − . −
7) 3 4 8 . − . − . − . −
10) 6 7 3 . − . − . − . −
10) 6 7 6 . − . − . − . −
7) 10 11 1 . − . − . − . −
9) 10 11 2 . − . − . − . −
7) 15 16 8 . − . − . − . −
9) 104 105 3 . − . − . − . − . − . −
9) 22 23 5 . − . − . − . −
7) 27 28 1 . − . − . − . −
7) 35 36 2 . − . − . − . −
10) 27 28 2 . − . − . − . −
10) 78 79 2 . − . − . − . −
12) 12 13 4 . − . − . − . −
7) 92 93 6 . − . − . − . − . − . −
11) 13 14 4 . − . − . − . −
8) 7 8 5 . − . − . − . −
8) 20 21 7 . − . − . − . −
7) 37 38 2 . − . − . − . −
7) 37 38 1 . − . − . − . −
7) 33 34 1 . − . − . − . −
7) 20 21 2 . − . − . − . −
7) 18 19 1 . − . − . − . −
7) 18 19 7 . − . − . . −
8) 13 14 9 . − . − . − . −
7) 12 13 6 . − . − . − . −
9) 12 13 6 . − . − . . −
7) 13 14 1 . . − . − . −
8) 13 14 8 . − . − . − . −
7) 13 14 8 . − . − . . −
8) 20 21 9 . − . − . − . −
7) 17 18 8 . − . − . − . −
8) 14 15 6 . − . − . − . −
7) 1 2 8 . − . − . . −
8) 2 3 1 . . − . − . −
7) 1 2 6 . − . − . . −
10) 11 12 1 . . − . . −
7) 25 26 3 . . −7)2 * *