A weighted full-Newton step primal-dual interior point algorithm for convex quadratic optimization

In this paper, a new weighted short-step primal-dual interior point algorithm for convex quadratic optimization (CQO) problems is presented. The algorithm uses at each interior point iteration only full-Newton steps and the strategy of the central path to obtain an ε-approximate solution of CQO. This algorithm yields the best currently wellknown theoretical iteration bound, namely, O( √ n log ε ) which is as good as the bound for the linear optimization analogue.


Introduction
Consider the quadratic optimization (QO) problem in standard format: x T Qx : Ax = b, x ≥ 0 } and its dual problem where Q is a given (n × n) real symmetric matrix, A is a given (m × n) real matrix with rank The QO problems have many important applications in optimization and mathematical programming problems.
There are a variety of solution approaches for CQO which have been studied intensively.Among them, the interior-point methods (IPMs) gained more attention than others methods.Feasible primal-dual path-following methods are the most attractive methods of IPMs [7,9].Their derived algorithms achieved important results such as polynomial complexity and numerical efficiency.These algorithms trace approximately the so-called central-path which is a curve that lies in the feasible region of the considered problem and they reach an optimal solution of it.However, in practice these methods don't always find a strictly feasible centered point to starting their derived algorithms.So, it is worth analyzing other cases when the starting points are not centered.Thus leads to the concept of Target-Following IPMs introduced by Jansen et al., [6] as a generalization of the classical path-following methods.These methods are based on the observation that with every algorithm which follows the central-path we associate a target sequence on the central-path.Weighted path-following methods can be viewed as a particular case of it.These methods were studied extensively by many authors [3,4,5,7,8] for Linear optimization (LO) and linear complementarity problem (LCP).Recently, Achache and Khebchache [1], introduced a new weighted method for monotone LCP where the complexity of the corresponding short-step algorithm is O( √ n log n ϵ ).Motivated by their work, we propose a new weighted primal-dual path-following algorithm for solving CQO.The algorithm uses at each interior point iteration only weighted full-Newton steps and the strategy of the central path to get an ϵ−approximate solution of CQO.We prove that the shortstep algorithm has the following iteration bound O( √ n log n ϵ ) which is as good as the bound for LO [3,7,8], CQO [1,3] and LCP [2,7], analogue.The algorithm has advantages that no line searches is needed and it can start with a suitable starting point not necessarily centered.
The rest of the paper is built as follows.In Section 2, the weighted-path and the search direction are presented.The generic weighted primal-dual path-following algorithm for CQO is also stated.In Section 3, the analysis of the algorithm and the iteration bound with short-step method are presented.Finally, a conclusion and future remarks follow in Section 4.
The notation used in this paper is as follows.R n denotes the space of ndimensional real vectors and denote the Euclidean and the maximum norms for a vector u, respectively.Let and f (x), be two positive real valued functions, then g for some positive constant k.Finally, the vector of all ones and the identity matrix are denoted by e and I, respectively.

The weighted-path and the search direction
Throughout the paper, we make the following assumptions for QO.Assumption 1. Interior Point Condition (IPC).There exists a triplet of vectors (x 0 , y 0 , z 0 ) such that: Finding an approximate solution of (P) and (D) is equivalent to solving the following system of optimality conditions for (P) and (D): The basic idea behind weighted primal-dual interior-point algorithm is to replace the third equation (complementarity condition) in ( 1) by the parametrized equation xz = w with w is a positive vector in R n .Thus, we consider the following perturbed system: ( Under Assumption 1 and Assumption 2, the system (2) has a unique solution denoted by (x(w), y(w), z(w)) for all w > 0 [2].The set is called the weighted-path of problems (P) and (D).If w goes to zero, then the limit of the weighted-path exists and since the limit point satisfies the complementarity condition, the limit yields an optimal solution for CQO.This limiting property of the weighted-path leads to the main idea of the iterative primal-dual methods for solving (2).

Remark 2.1
If w = µe with µ > 0, then the weighted-path coincides with the classical centralpath.
M. ACHACHE Now, we proceed to describe a weighted full-Newton step produced by the algorithm for a given w > 0. Applying Newton's method for (2) for a given feasible point (x, y, z) then the Newton direction (∆x, ∆y, ∆z) at this point is the unique solution of the following linear system of equations: where X :=diag(x), Z :=diag(z).
Again under our assumptions and the fact that rank(A) = m, the system (3) has a unique solution (∆x, ∆y, ∆z).Hence, a new weighted full-Newton iteration is constructed according to: x + := x + ∆x; y + := y + ∆y; and z To simplify the matters, we define the vectors: The vector d uses to scale the vectors x and z to the same vector v as and as well as for the original directions to the scaling directions: It follows that: and since Q is a semidefinite matrix.Hence, by using ( 5), ( 6) and (7), the system (3) becomes: where and Ā = DAD and Q = DQD with D :=diag(d).
In the next sub-section, we describe the generic feasible weighted primal-dual path-following algorithm to solve CQO.

The Algorithm
Similar to LO case, we define for any positive vector v and in view of ( 9), a normbased proximity measure as follows: One can easily verify that Hence the value δ(v; w) is to measure the distance of a point (x, y, z) to the weighted-path (x(w), y(w), z(w)).
Let denote another measure σ C (w) as follows The role of σ C (w) is to measure the closeness of w to the central path. Here, and likewise max(w) = max i (w i ).
Note that in (11), σ C (w) ≥ 1, with equality if w is on the central-path.Now we are ready to describe the generic weighted path-following interior-point algorithm for CQO as follows.

Complexity analysis
In the next lemma, we state some useful technical results that will be used later in the analysis of the algorithm.Lemma 3.1 Let (d x , d z ) be a solution of ( 8) and suppose w > 0. If δ := δ(v; w).Then, one has and Proof: Since 0 ≤ d T x d z , the statement in (12) follows immediately from the following equality: For ( 13), (see Lemma C.4 in [7]), since This completes the proof. 2 The following lemma shows that the feasibility of the weighted full-Newton step under the condition δ := δ(v; w) < 1.

Lemma 3.2
Let (x, z) be a strictly feasible primal-dual point.Then x + = x + ∆x > 0 and z + = y + ∆z > 0 if and only if w + d x d z > 0.
If the full-Newton step is strictly feasible x + > 0 and z + > 0 then x + z + > 0 and so w + d x d z > 0.
To show that x + and z + are positive, we introduce a step length α ∈ [0, 1] and we define So x 0 = x, x 1 = x + and similar notations for z, hence x 0 z 0 = xz > 0. We have, Now by using (6), we get We assume that w + d x d z > 0, we deduce that w + ∆x∆z > 0 which equivalent to ∆x∆z > −w.Substitution we obtain Since xz and w are positive it follows that x α z α > 0 for α ∈ [0, 1] .Hence, none of the entries of x α and z α vanish for α ∈ [0, 1] .Since x 0 and z 0 are positive, this implies that x α > 0 and z α > 0 for α ∈ [0, 1].Hence, by continuity argument, the vectors x α and z α must be positive which proves that x + and z + are positive.This completes the proof. 2
Proof: In Lemma 3.2, we have seen that: We have Now, according to (13), Lemma 3.1, it follows that: This completes the proof. 2 For convenience, we may write ) .
Proof: It follows straightforwardly from Lemma 3.3 and since In the next lemma, we show the influence of a weighted full-Newton step on the proximity measure. Then Proof: By definition, we have, w+dxdz , then by Lemmas 3.1 and 3.4, we have, This completes the proof.
M. ACHACHE Using Lemma 3.5 and (11), we have, n σC (w) , and observe that σ C (w) ≥ 1, and for n ≥ 3, then θ ≤ Note that, in all the iterates produced by Algorithm 2.1, we have σ C (w) = σ C (w 0 ).Thus, we deduce from Lemma 3.6 that for the default θ = 1 2 √ n σC (w 0 ) , the conditions x, y > 0 and δ(v + ; w + ) ≤ 1 √ 2 are maintained during the algorithm.Thus, confirms that Algorithm 2.1, is well-defined.The upper bound of the duality gap after a weighted full-Newton step is presented in the following lemma.
This completes the proof. 2 The following lemma gives an upper bound for the total number of iterations produced by Algorithm 2.1.

Conclusion and future remarks
In this paper, we have presented a weighted full-Newton step path-following method for CQO.At each interior point iteration, only full-Newton steps are used.The favorable polynomial complexity bound for the algorithm with short-step method is deserved, namely, O( √ n log n ϵ ) which is as good as LO case.Finally, the numerical implementation of this algorithm remains to be investigated.

x
+ z + = w + d x d z .Hence e T (x + z + ) = e T w + e T d x d z = e T w + d T x d z .