Inexact Double Step Length Method for Solving Systems of Nonlinear Equations

In this paper, a single direction with double step length method for solving systems of nonlinear equations is presented. Main idea used in the algorithm is to approximate the Jacobian via acceleration parameter. Furthermore, the two step lengths are calculated using inexact line search procedure. This method is matrix-free, and so is advantageous when solving large-scale problems. The proposed method is proven to be globally convergent under appropriate conditions. The preliminary numerical results reported in this paper using a large-scale benchmark test problems show that the proposed method is practically quite effective


Introduction
Consider the systems of nonlinear equations: where F : R n → R n is nonlinear map. Among various methods for solving nonlinear equations (1), Newton's method is quite welcome due to its nice properties such as the rapid convergence rate, the decreasing of the function value sequence [14]. The iterative formula of a Newton method is given by where, α k is a step length to be computed by a line search technique [4,5,7], x k+1 represents a new iterative point, x k is the previous iteration, while d k is the search direction to be calculated by solving the following linear system of equations, where F ′ (x k ) is the Jacobian matrix of F (x k ) at x k . A basic requirement of the line search is to sufficiently decrease the function values i.e. to establish f (x) = 1 2 ||F (x)|| 2 (5) where, || · || to stand for the Euclidian norm. Then the nonlinear equations problem (1) is equivalent to the following global optimization problem minf (x), x ∈ R n . and condition (4) is equivalent to Furthermore, the search direction d k is generally required to satisfy the descent condition It is vital to mention that due to the well known shortcomings of Newton method, a double step length has been proposed in [13,16] and the iterative procedure is given as: r where x k+1 represents a new iterative point, x k is the previous iterative point, α k and β k denote the step lengths, while c k and d k are search directions respectively. The step lengths α k and β k can also be computed either exact or in exact. The ideal line search rule is the exact one [1] that satisfies In fact, the exact step length is difficult or even impossible to seek in practical computation. Therefore the most frequently used line search in practice is inexact line search [3,9,11]. Brown and Saad [5] proposed the following line search rule to obtain the step length α k where σ ∈ (0, 1) and the search direction d k can be obtained in several ways (see [2,6,7,8] and references therein). From the technique in (9), it is easy to see that the Jacobian matrix must be computed at every iteration, which will increase the computing difficulty, especially for the large-scale problems or when the matrix is expensive to calculate. Considering these points, a new backtracking inexact technique is presented by Yuan * and Lu [12] in order to obtain the step length α k : where δ ∈ (0, 1). The global convergence and the superlinear convergence of this method is established. The numerical results showed that the line search technique (10) is more effective than the normal methods.
In this article, we introduce the derivative-free line search proposed by Li and Fukushima [10] in order to compute our step lengths α k and β k .
Let ω 1 > 0, ω 2 > 0 and q, r ∈ (0, 1) be constants and let {η k } be a given positive sequence such that Let i k be the smallest non negative integer i such that (12) holds for α = r i and β = q i . Let α k = r i k and β k = q i k . An iterative method that generates a sequence {x k } satisfying (4) or (6) is called a norm descent method [16]. If d k is a descent direction of f at x k , then inequality (6) holds for all α k > 0 sufficiently small. Accordingly, the related iterative method is a norm descent method. In particular, Newton's method with line search is norm descent. For a quasi-Newton method, however, d k may not be a descent direction of f at x k even if B k is symmetric and positive definite. To globalize a quasi-Newton method, Li and Fukushima [10] proposed an approximately norm descent line search technique and established global and superlinear convergence of a Gauss-Newton based BFGS method for solving symmetric nonlinear equations. The method in [10] is not norm descent. In addition, the global convergence theorem is established under the assumption that F ′ (x k ) is uniformly nonsingular. The drawback of the technique (3) is the need to compute the Jacobian matrix F ′ (x k ) at every iteration, which will increase the computing difficulty, due to the first-order derivative of the system because sometimes they are not even available or could not be obtained exactly [14] especially for the large-scale problems. In this case Newton's method cannot be applied directly [7]. Therefore, motivated by [13] the purpose of this article is to develop a derivative-free method with decent direction for solving system of nonlinear equations via Where I is an identity matrix. The proposed method has a norm descent property without computing the Jacobian matrix with less number of iterations and CPU time that is globally convergent. We organized the paper as follows; In the next section, we present the proposed method, convergence results are presented in section 3. Some numerical results are reported in section 4. Finally we made conclusions in section 5.

Derivation of the Method
In this section we compute the two step lengths α k and β k in (7) using inexact line search procedure. This is made possible by making the two directions in (7) to be equal, i.e c k = d k . In order to incorporate more information of the iterates at each iteration and to improve good direction towards the solution, we suggest a new direction to be defined as: where γ k > 0 is an acceleration parameter. By putting (14) in to (7) we obtained We now proceed to obtain the proposed acceleration parameter. Therefore, we start from where the parameter ξ fulfills the Bearing in mind that the distance between x k and x k+1 is small enough, we can take δ = 1 in (17) and get ξ = x k+1 . Thus we have Now from (16) and (18) its not difficult to verify that: , and multiplying y T k to the both side of (19) the acceleration parameter yields: From (14) and (15) we have the general scheme as: Now we describe the algorithm of the proposed method as follows: . STEP 5: Compute step the lengths α k and β k (using (12)).

Convergence Analysis
In this section we present the global convergence of our method (IDS). To begin with, let us defined the level set In order to analyze the convergence of algorithm 1 we need the following assumption: (1) There exists x * ∈ R n such that F (x * ) = 0.
(2) F is continuously differentiable in some neighborhood say N of x * containing Ω.
(3) The Jacobian of F is bounded and positive definite on N. i.e there exists a positive constants M > m > 0 such that and From the level set we have: Remarks: Assumption 1 implies that there exists a constants M > m > 0 such that In particular ∀x ∈ N we have where x * stands for the unique solution of (1) in N. Since γ k I approximates F ′ (x k ) along direction s k , we can contemplate another assumption where ϵ ∈ (0, 1) is a small quantity [12]. Lemma 1. Let assumption 2 holds and {x k } be generated by algorithm 1.
proof. from (14), we have by chauchy swatz we have, Hence for ϵ ∈ (0, 1) this lemma is true. By lemma 1, we can deduce that the norm function f (x k ) is a descent along d k , which means that ∥F (x k+1 )∥ ≤ ∥F (x k )∥ is true. Lemma 2. Let assumption 2 hold and {x k } be generated by algorithm 1. Then {x k } ⊂ Ω. proof. By lemma 1 we have ∥F (x k+1 )∥ ≤ ∥F (x k )∥. Moreover, we have for all k.
This implies that {x k } ⊂ Ω. Lemma 3.(see [12]) Suppose that assumption 1 holds {x k } is generated by algorithm 1. Then there exists a constant m > 0 such that for all k.
Lemma 4.(see [7]) Suppose that assumption 1 holds and {x k } is generated by algorithm 1. Then we have and lim Lemma 5. Suppose that assumption 1 holds and {x k } is generated by algorithm 1. Then there exists some positive constants m 2 such that for all k > 0, (37) , we have (36). The proof is completed.
We can deduce that for all k (36) hold. Now we are going to establish the following global convergence theorem to show that under some suitable conditions, there exist an accumulation point of {x k } which is a solution of problem (1). Theorem 1. Suppose that assumption 1 holds and {x k } is generated by algorithm 1. Assume further for all k > 0, where c is some positive constant. Then from (38) and (40) we have on the other hand from (14) we have, The proof is completed.

Numerical Results
In this section,the performance of our method for solving non linear equation (1) is compared with a derivative-free CG method and its global convergence for solving symmetric nonlinear equations [2]. It should be noted here that: Step length Method (IDS) stands for our method and we set the following: (ii) A derivative-free CG (DFCG) is the method proposed by [2] and we set the following: The employed computational codes was written in Matlab 7.9.0 (R2009b) and run on a personal computer 2.00 GHz CPU processor and 3 GB RAM memory. We stopped the iteration if the total number of iterations exceeds 1000 or ∥F (x k )∥ ≤ 10 −3 . We have tried the two methods on eight test problems with different initial points and dimension (n values). problems 1-7 are from [2] and problem 8 was arbitrarily constructed by us. Problem 1: Problem 2: Problem 3:

Problem 4:
Problem 5: Problem 6: Problem 7: Problem 8: i = 1, 2, ..., n − 1.  The numerical results of the two methods are reported in Tables 1,where "iter" and "Time" stand for the total number of all iterations and the CPU time in seconds, respectively, while ∥F (x k )∥ is the norm of the residual at the stopping point. From Tables 1, we can easily observe that both of these methods attempt to solve the systems of nonlinear equations (1), but the better efficiency and effectiveness of our proposed algorithm was clear for it solves where DFCG fails. This is quite evident for instance with problem 6. In particular, the IDS method considerably outperforms the DFCG for almost all the tested problems, as it has the least number of iterations and CPU time, which are even much less than the CPU for the DFCG method. This is apparently due to the computation of double step length in each iteration of the IDS as well as the approximation of the Jacobian through the acceleration parameter. Figures (1-2) show the performance of our method relative to the number of iterations and CPU time, which were evaluated using the profiles of Dolan and Moré [15]. That is, for each method, we plot the fraction P (τ ) of the problems for which the method is within a factor τ of the best time. The top curve is the method that solved the most problems in a time that was within a factor τ of the best time.

Conclusion
In this paper we present an Inexact Double Step length (IDS) method for solving systems of nonlinear equations and compare its performance with that of a derivative-free conjugate gradient (DFCG) method for symmetric nonlinear equations [2] by doing some numerical experiments. We however proved the global convergence of our proposed method by using a derivative-free line search, and the numerical results show that our method is very efficient.