Finite Difference Weerakoon-Fernando Method to solve nonlinear equations without using derivatives
aa r X i v : . [ m a t h . NA ] F e b Finite Difference Weerakoon-Fernando Method to solvenonlinear equations without using derivatives
S.L. Heenatigala , Sunethra Weerakoon , T. G. I. Fernando Department of Mathematics , & Department of Computer Science University of Sri Jayewardenepura, Gangodawila, Nugegoda, Sri Lanka
Abstract
This research was mainly conducted to explore the possibility of formulating an efficient algorithm tofind roots of nonlinear equations without using the derivative of the function. The Weerakoon-Fernandomethod had been taken as the base in this project to find a new method without the derivative sinceWeerakoon-Fernando method gives 3rd order convergence. After several unsuccessful attempts we wereable to formulate the Finite Difference Weerakoon-Fernando Method (FDWFM) presented here. We no-ticed that the FDWFM approaches the root faster than any other existing method in the absence of thederivatives as an example, the popular nonlinear equation solver such as secant method (order of conver-gence is 1.618) in the absence of the derivative. And the FDWFM had three function evaluations andsecant method had two function evaluations. By implementing FDWFM on nonlinear equations withcomplex roots and also on systems of nonlinear equations, we received very encouraging results. Whenapplying the FDWFM to systems of nonlinear equations, we resolved the involvement of the Jacobianproblem by following the procedure in the Broyden's method. The computational order of convergenceof the FDWFM was close to 2.5 for all these cases. This will undoubtedly provide scientists the efficientnumerical algorithm, that doesn’t need the derivative of the function to solve nonlinear equations, that theywere searching for over centuries.Keywords :- Weerakoon-Fernando Method, Nonlinear equations, Absence of the derivative, Iterative meth-ods, Broyden's method, Order of convergence. 1onlinear Equations Without Using Derivatives
Even though the Newton's method is popular in finding a single root of a nonlinear equation, the Weerakoon-Fernando method Fernando, (1998) and Weerakoon and Fernando, (2000) is better because its order ofconvergence is higher. Both the Newton's method and the Weerakoon-Fernando method require the pres-ence of the derivative of the function in the iterative equation. However, it so happens that some modellingsituations do not provide value of the derivative baring the use of most computer algorithms using suchdata. The secant method is obtained by replacing the derivative of Newton's method by a difference ap-proximation. But the order of convergence of the secant method is 1.618 and the number of functionevaluations required to carry out one iteration is two. However, this speed seems to be not adequate whenconsidering the requirements of the current world with the high demand for more efficient algorithms dueto the rapid development of technology. Thus we were compelled to search for a method to fulfill the cur-rent needs by replacing the derivatives in the fast converging Weerakoon-Fernando method by appropriatedifference approximations due to the non-availability of powerful numerical methods that don’t requirederivatives to approximate roots of nonlinear equation.This paper describes how to construct a best suitable method leading to an efficient algorithm without thederivatives to numerically solve nonlinear equations while optimizing the high order of convergence andthe number of function evaluations. Then it describes how we checked that the method is in fact accept-able for nonlinear equations with complex roots. Finally it explains how that method was adapted even forsystems of nonlinear equations, producing excellent results.
Let the iterative sequence: { x n : n = 1 , , . . . } that converges to x ∗ be generated by a numerical scheme.If there exists a constant c ≥ , an integer n ≥ and ρ ≥ such that for all n > n , the inequality belowholds for any vector norm k . k . (cid:13)(cid:13) x n +1 − x ∗ (cid:13)(cid:13) ≤ c k x n − x ∗ k ρ (2.1)onlinear Equations Without Using DerivativesThen the iterative scheme is said to be converge to x ∗ with ρ th order convergence. Let x ∗ be a root of the equation F ( x ) = 0 and suppose that x n − , x n and x n +1 be consecutive iteratescloser to the root x ∗ , generated by an iterative scheme. Then the Computational Order of Convergence(COC) ρ of the iterative scheme or the numerical algorithm can be approximated by ρ ≈ ln k x n +1 − x ∗ k k x n − x ∗ k ln k x n − x ∗ k k x n − − x ∗ k (2.2) Fernando, (1998) and Weerakoon and Fernando, (2000) introduced a third order convergent Weerakoon-Fernando Method (WFM) to solve nonlinear equations. The local model M n ( x ) of WFM is given by theequation. M n ( x ) = f ( x n ) + 12 ( x − x n ) h f ′ ( x n ) + f ′ ( x ) i (3.1)When x is taken as the next iterate x n +1 and the root, we get the following formula x n +1 = x n − f ( x n ) f ′ ( x n ) + f ′ ( x n +1 ) (3.2)here n = 0 , , , . . . x n +1 = x n − f ( x n ) f ′ ( x n ) + f ′ (cid:0) x ∗ n +1 (cid:1) (3.3)onlinear Equations Without Using Derivativeswhere x ∗ n +1 = x n − f ( x n ) f ′ ( x n ) (3.4)here n = 0 , , , . . . In Weerakoon-Fernando formula replace the variable x with the complex variable z in both sides then thefollowing is the complex form of the WFM. z n +1 = z n − f ( z n ) f ′ ( z n ) + f ′ (cid:0) z ∗ n +1 (cid:1) (3.5)where z ∗ n +1 = z n − f ( z n ) f ′ ( z n ) (3.6)here n = 0 , , , . . . Nishani, (2015) and Nishani, Weerakoon, Fernando and Liyanage, (2018) introduced a third order conver-gent Weerakoon-Fernando Method (WFM) to solve systems of nonlinear equations. When F is a vectorvalued function with non-zero derivatives defined on the set D ⊂ ℜ n and x , x ∈ D , the extension of theWFM to solve systems of nonlinear equations can be given as follows. x n +1 = x n − (cid:2) J ( F ( x n )) + J (cid:0) F (cid:0) x λn +1 (cid:1)(cid:1)(cid:3) − ( F ( x n )) (3.7)where x λn +1 = x n − [ J ( F ( x n ))] − ( F ( x n )) (3.8)here n = 0 , , , . . . and x n is the n th iterate and J is the Jacobian matrix of F .onlinear Equations Without Using Derivatives When we use the secant method for systems of nonlinear equations we face a special problem with theJacobian. In secant method for systems, a vector is present in the denominator. To overcome the problemof having to take the inverse of a vector we follow the most popular secant approximation proposed byC. Broyden The algorithm is analogous to Newton's method, but it replaces the analytic Jacobian by thefollowing approximation. This method is called the Broyden's method Atkinson, (1988).ALGORITHEM
Broyden's method .Given F : ℜ n → ℜ n , x ∈ ℜ n , A ∈ ℜ n × n s k - Initial Step, y k - Yield of current Step, x k +1 - Next iterationFOR k = 0 to as k = x k +1 − x k y k = F (cid:0) x k +1 (cid:1) − F ( x k ) for A k s k = y k A k s k = − F ( x k ) x k +1 = x k + s k y k = F (cid:0) x k +1 (cid:1) − F ( x k ) A k +1 = A k + y k − A k s k s tk s k s tk END FORHere the final step is used to replace the analytic Jacobian by a matrix.
Here we use both backward and forward difference approximations for the derivatives in the same for-mula appropriately to get an acceptable result.onlinear Equations Without Using DerivativesConsider the Weerakoon-Fernando Method x n +1 = x n − f ( x n ) f ′ ( x n ) + f ′ ( x n +1 ) (4.1)Substituting the forward difference approximation for f ′ ( x n ) ≈ f ( x n +1 ) − f ( x n ) x n +1 − x n (4.2)and the backward difference approximation for f ′ ( x n +1 ) ≈ f ( x n +1 ) − f ( x n ) x n +1 − x n (4.3)in equation (4 . , we get x n +1 = x n − f ( x n ) f ( x n +1 ) − f ( x n ) x n +1 − x n + f ( x n +1 ) − f ( x n ) x n +1 − x n (4.4) x n +1 = x n − f ( x n ) ( x n +1 − x n )2 ( f ( x n +1 ) − f ( x n )) (4.5)Thus the new iterative formula without derivative terms is x n +1 = x n − f ( x n ) ( x n +1 − x n ) f ( x n +1 ) − f ( x n ) (4.6)But it is an implicit method requiring x n +1 term at the ( n + 1) th iterative step to calculate the ( n + 1) th iterate itself. The secant method can be used to replace the term x n +1 on the RHS of the above equation toovercome this difficulty. Thus the Finite Difference Weerakoon-Fernando Method (FDWFM) is obtainedas x n +1 = x n − f ( x n ) (cid:0) x ∗ n +1 − x n (cid:1) f (cid:0) x ∗ n +1 (cid:1) − f ( x n ) (4.7)where x ∗ n +1 = x n − f ( x n ) ( x n − x n − ) f ( x n ) − f ( x n − ) (4.8) n = 1 , , . . . onlinear Equations Without Using Derivatives In the Finite Difference Weerakoon-Fernando formula replace the variable x with the complex variable zin both sides to get the following form: z n +1 = z n − f ( z n ) (cid:0) z ∗ n +1 − z n (cid:1) f (cid:0) z ∗ n +1 (cid:1) − f ( z n ) (4.9)where z ∗ n +1 = z n − f ( z n ) ( z n − z n − ) f ( z n ) − f ( z n − ) (4.10)Here n = 0 , , , . . . Choose an initial estimate x ∈ ℜ n and a non-singular initial Matrix A ∈ ℜ n × n . Set k := 0 and repeatthe following sequence of steps until (cid:13)(cid:13) F ( x k ) (cid:13)(cid:13) < tolerance
1. Solve A k I k = F (cid:0) X ∗ k +1 (cid:1) − F ( X k ) f or I k & X ∗ k +1 from Broyden's method2. X k +1 = X k + I k Y k = F (cid:0) X k +1 (cid:1) − F ( X k ) A k +1 = A k + ( Y k − A k I k ) I tk I tk I k Results are given in Table 1, Table 2, Table 3, Table 4, Table 5, Table 6 and Table 7. on li n ea r E qu a ti on s W it hou t U s i ng D e r i v a ti v e s Table 1: Comparison of FDWFM with secant method & Newton's for uni-variate nonlinear equations with real rootsFunction X0 X1 i COC NFC RootSCT NM FDWFM SCT NM FDWFM SCT NM FDWFM x + 5 x + 4 cosx + e x sin x − x + 1 x − e x − x + 2 cosx − x (1 − x ) − xe x − sin x + 3 cosx + 5 x sin x + e x cosxsinx − -2 -1 11 9 5 1.432 1.99 2.49479 12 11 12 -1.207647 e x +7 x − − x − on li n ea r E qu a ti on s W it hou t U s i ng D e r i v a ti v e s Table 2: Comparison of FDWFM with secant method & Newton's method for nonlinear equations with complex rootsFunction X0 X1 i COC NFC RootSCT NM FDWFM SCT NM FDWFM SCT NM FDWFM z + 1 z + e z − z − sin z − z − ze z − sinz + 3 cosz + 1 -2,1 -1,1.5 16 13 7 1.657 1.991 2.42055 17 15 15 1.1199- 0.7028i z − z + 3 z (1 − z ) + 1 z − z + 7 z − z + 4 z − z + 1 z sin z + e z coszsinz + 10 on li n ea r E qu a ti on s W it hou t U s i ng D e r i v a ti v e s Table 3: Comparison of FDWFM with Broyden's method for systems of two nonlinear equationsFunction X0 i COC RootBMFSM FDWFM BMFSM FDWFM1 x + y − x + y − (1,3) 12 10 1.678007 2.6416 0.000000642.999981362 x + y − x − xy + 35 (2,3) 13 12 1.712084 2.5687 1.883640562.715947453 x − x + y + 8 xy + x − y + 8 (0,0) 11 8 1.687346 2.8984 0.999999271.000000484 x − cosysinx + 0 . y (0,-0.5) 11 7 1.78364 2.2331 0.53038868-1.011737345 x + y − e x − + y − (0.5,0.5) 15 10 1.69356 2.2420 0.999987771.000014976 − x − x + 2 y − x − + ( y −
6) 2 − (-5,-5) 24 15 1.537854 2.3251 1.5469463610.96999487 cosy + 7 sinx − x cosx − siny − y (0,0) 10 9 1.62944 2.4499 0.526517020.50792810FDWFM - Finite Difference Weerakoon-Fernando Method BMFSM - Broyden's method for secant methodi - Number of iterations to approximate the root COC - Computational Order of Convergence on li n ea r E qu a ti on s W it hou t U s i ng D e r i v a ti v e s Table 4: Comparison of FDWFM with Broyden's method for systems of three nonlinear equationsFunction X0 i COC RootBMFSM FDWFM BMFSM FDWFM1 x + y + z x − y + z x + y − z (1,1,1) 15 14 1.71123 2.3883 0.002702710.002702710.002702712 x cos (? yz ) − / x −
81 ( y + 0 . + sin ? ( z ) + 1 . e − ( xy ) + 20 z + (10 π − / (1,1,-1) 16 13 1.84657 2.81152 0.707121210.01416426-0.522997713 x + e x − + ( y + z ) − e y − /x + z − y + sin (? y −
2) + z − (1.4,2.2,3.1) 9 8 1.877212 2.79400 0.999964912.000093962.999809414 x + y − z − x + 10 y − z − y − z + 22 (3,3,2) 14 12 1.722963 2.94914 1.036404521.085703430.93119446FDWFM - Finite Difference Weerakoon-Fernando Method BMFSM - Broyden's method for secant methodi - Number of iterations to approximate the root COC - Computational Order of Convergence on li n ea r E qu a ti on s W it hou t U s i ng D e r i v a ti v e s Table 5: Comparison of FDWFM with Newton's method and Broyden's method for systems of four nonlinear equations f = x + x − , f = x x + x x , f = x x + x x − / , f = x x + x x Initial guessX0 i COC RootBMFSM NM FDWFM BMFSM NM FDWFM(10,10,2,-1) 10 8 7 1.528 1.789 2.534 1.0000,1.0000,0.57735,-0.57735(9.449645,8.198130,1.958279,-2.2299584) 11 8 7 1.687 2.187 2.574 1.0000,1.0000,0.57735,-0.57735(10,10,-1,2) 10 8 7 1.432 1.789 2.341 1.0000,1.0000,-0.57735,0.57735FDWFM-Finite Difference Weerakoon-Fernando Method NM-Newton’s methodBMFSM-Broyden’s method for secant method i-Number of iterations to approximate the rootCOC-Computational Order of Convergence on li n ea r E qu a ti on s W it hou t U s i ng D e r i v a ti v e s Table 6: Comparison of FDWFM with Newton's method and Broyden's method for systems of six nonlinear equations f = x + x − , f = x + x − , f = x x + x x , f = x x + x x , f = x x x + x x x , f = x x x + x x x Initial guessX0 i COC RootBMFSM NM FDWFM BMFSM NM FDWFM(3,5,4,6,5.5,2,1,-4) 9 7 6 1.632 2.277 2.435 0.5368,0.9170,0.8436,0.3987,0.0000,-0.0000(2.5257,5.0538,5.8289,2.1629,2.4797,-4.9408) 9 7 6 1.714 2.321 2.498 0.5039,0.8519,0.8637,0.5236,-0.0000,0.0000(2.4711,4.3696,6.2511,1.4369,1,9453,-4.4211) 9 7 6 1.786 2.625 2.542 0.3676,0.9499,0.9299,0.3123,0.0000,0.0000FDWFM-Finite Difference Weerakoon-Fernando Method NM-Newton’s methodBMFSM-Broyden’s method for secant method i-Number of iterations to approximate the rootCOC-Computational Order of Convergence on li n ea r E qu a ti on s W it hou t U s i ng D e r i v a ti v e s Table 7: Comparison of FDWFM with Newton's method and Broyden's method for systems of ten nonlinear equations f = x − . − . x x x , f = x − . − . x x x , f = x − . − . x x x , f = x − . − . x x x , f = x − . − . x x x , f = x − . − . x x x , f = x − . − . x x x , f = x − . − . x x x , f = x − . − . x x x , f = x − . − . x x x Initial guessX0 i COC RootBMFSM NM FDWFM BMFSM NM FDWFM(1,1,1,1,1,1,1,1,1,1) 8 5 4 1.234 1.799 2.365 0.2578,0.3810,0.2878,0.20060.4452,0.1491,0.4320,0.07340.3459,0.4273(-0.3956,-1.3108,-0.3927,4.8163,-3.4359,3.555,1.4476,-1.2372,-3.0907,-0.7174) 12 10 8 1.435 1.827 2.481 0.3452,1.2453,1,0342,1,4352,1.4533,1.4563,1.8453,1.7453,1.3435,1.8464(1.625,1.780,1.0811,1.9293,1.7757,1.4867,1.4358,1.44671.3063,1.5085) 10 8 6 1.654 1.958 2.499 1.8430,1.9683,1.6191,2.0850,2.5636,2.4194,2.7151,2.1386,2.5682,2.1907FDWFM-Finite Difference Weerakoon-Fernando Method NM-Newton’s methodBMFSM-Broyden’s method for secant method i-Number of iterations to approximate the rootCOC-Computational Order of Convergenceonlinear Equations Without Using DerivativesTable 1 shows the result for comparison of FDWFM with secant method and Newton’s for uni-variatenonlinear equations with real roots. Here computer order of convergence for FDWFM is near 2.4. It ishigher than Secant method and Newton method. Then we applied the method for complex roots. Whenthe method was applied to nonlinear equations for complex roots it also gives the same satisfactory results.Then the method was extended to systems of nonlinear equations. When we apply the method to systemsof non-linear equations there were some difficulties. In Improved Newton’s method it was necessary toobtain Jacobian matrices to solve systems of nonlinear equations but for this resulting method the Jacobianmatrix becomes a vector. So the challenge was to find the inverse of the vector. So Broyden’s method wasused to overcome that difficulty. After following the technique used in Broyden’s method resulting for-mula for systems of nonlinear equations with two three four six and ten variables, the results were againencouraging just like for the one variable case. So the objective was achieved there as well. However,when we follow Broyden’s method we cannot do away with the derivative part.
For all nonlinear equations we have considered, the new algorithm FDWFM seems to be more efficientthan any other algorithm without the derivatives to numerically solve them. This method gives a compu-tational order of convergence higher than any other existing method in the absence of the derivatives. Itreturns the same order of convergence for nonlinear equations with complex roots as well as for systemsof nonlinear equations. Apparently, the FDWFM had three function evaluations and secant method hadtwo function evaluations. But according to the computed results (Table 1 and Table 2) most of the results,total number of function evaluations required is less than or equal that secant method. Thus FDWFMcan be considered as a superior method giving faster convergence to find the roots of nonlinear equationsin the absence of the derivative for uni-variate nonlinear equations with complex roots as well as for themultivariate systems of nonlinear equations. Since the proposed method is even faster than the universallyaccepted second order Newton’s method while meeting the requirement of not having the derivative ofthe function as well, this algorithm will undoubtedly be very useful to the scientific and the industrialcommunity.onlinear Equations Without Using Derivatives
1. Atkinson K. (1988).
An Introduction to Numerical Analysis , 2nd edition, University of lowa.2. Burden R.L. & Faires J.D. (2005).
Numerical Analysis , 8th edition, Bob Pirtle, USA.3. Dennis J.E. & Robert B.S. (1987)
Numerical Methods for Unconstrained Optimization and Nonlin-ear Equation , Society for Industrial and Applied Mathematics, USA.4. Fernando T. G. I. (1998).
Improved Newton’s Method for Solving Nonlinear Equations
Third order convergence ofImproved Newton’s method for systems of nonlinear equations , 502/E1, Proceedings of the annualsessions of Sri Lanka Association for the Advancement of Science, Sri Lanka.6. Nishani H.P.S. (2015).
A Variant of Newton's Method with Accelerated Third Order Convergence forSystems of Nonlinear Equations , M. Sc in Industrial Mathematics thesis, University of Sri Jayewar-denepura.7. Nishani H.P.S., Weerakoon S., Fernando T.G.I. & Liyanage L.M. (2018).