Quadratic third-order tensor optimization problem with quadratic constraints

Quadratically constrained quadratic programs (QQPs) problems play an important modeling role in many diverse problems. These problems are in general NP hard and numerically intractable. Semideﬁnite programming (SDP) relaxations often provide good approximate solutions to these hard problems. For several special cases of QQP, e.g., convex programs and trust region subproblems, SDP relaxation provides the exact optimal value, i.e., there is a zero duality gap. However, this is not true for the general QQP, or even the QQP with two convex constraints, but a nonconvex objective. In this paper, we consider a certain QQP where the variable is neither vector nor matrix but a third-order tensor. This problem can be viewed as a generalization of the ordinary QQP with vector or matrix as it’s variable. Under some mild conditions, we ﬁrst show that SDP relaxation provides exact optimal solutions for the original problem. Then we focus on two classes of homogeneous quadratic tensor programming problems which have no requirements on the constraints number. For one, we provide an easily implemental polynomial time algorithm to approximately solve the problem and discuss the approximation ratio. For the other, we show there is no gap between the SDP relaxation and itself.


Introduction
A tensor is a multidimensional array.More formally, an N -way or N th-order tensor is an element of the tensor product of N vector spaces, each of which has its own coordinate system.A first-order tensor is a vector, a second-order tensor is a matrix, a third-order tensor has three indices and tensors of order three or higher are called higher-order tensors.Tensors have been widely utilized in Electrical Engineering since the nineties [16], and in particular in Antenna Array Processing [4] or Telecommunications [9] [13].
In this paper, we mainly consider the following nonconvex quadratic optimization problem of the form where R n1×n2×n3 is a third-order tensor.Problem of the above type is actually an extension of the quadratic matrix programming (QMP) problem [6,2] and quadratically constrained quadratic programs (QQPs) [5,15,3] if we let n 3 = 1 and n 2 , n 3 = 1 respectively.We call problem (1.1) a quadratic third-order tensor programming (QTTP) problem.Tight semidefinite programming (SDP) relaxation is known to hold for only a few classes of nonconvex QQPs such as trust region problem [5].Many extensions of this problem were considered in [12,15] while these results cannot be extended to QQPs involving two constraints [17].As a generalization of QQPs with vector variables, Beck [2] proved stronger results to QMP problems.Now after extending "matrix" to the "third-order tensor" situation, we will establish the same tight SDP relaxation results for QTTP problems with at most n 2 constraints (see section 2).However if QTTP problem (1.1) is homogeneous, still we can get the tightness result in spite of the constraints number under some conditions (see section 4).
Tensors can be multiplied together, though obviously the notation and symbols for this are much more complex than for matrices.For a full treatment of tensor multiplication see, e.g., Bader and Kolda [1].In [8], they consider the tensor nmode product, i.e., multiplying a tensor by a matrix (or a vector) in mode n: here tensor X ∈ R I1×•••×IN and matrix U ∈ R J×In .Using this definition, we seek to find a third-order tensor X which solves the following optimization problem: where A ∈ R n1×n1 and X , B ∈ R n1×n2×n3 are two third-order tensors.The norm of a tensor X is the square root of the sum of the squares of all its elements, i.e., This is analogous to the matrix Frobenius norm, which is denoted ∥A∥ for a matrix 2) can be rewritten as a nonconstraint QTTP problem: The remaining of the paper is organized as follows: In section 2, we homogenize the QTTP problem and change it to a separable QMP problem and show that the optimal values can be obtained exactly by its SDP relaxation under some mild conditions.In section 3, we concentrate on the homogeneous QTTP problem.Although it is still NP-hard, we can approximately solve it in polynomial time.
In section 4, we present a special class of homogeneous QTTP problems with nonpositive off-diagonal elements and show that their optimal values can be obtained exactly by SDP relaxations and more precisely be solved without gap.
Throughout this paper, we use the following notations: Vectors are denoted by lowercase letters, e.g., x, matrices by uppercase letters, e.g., X, and third-order tensors by calligraphic capitals, e.g., X .Let x i , X ij and X ijk denote the ith, ijth and ijkth component of x, X and X respectively.For matrix A and B, A ≽ 0 means that A is a positive semidefinite matrix, rank(A) denotes the rank of A, TrA denotes the trace of A, A T denotes the transpose matrix of A, and A • B stands for the standard matrix inner product, i.e., A • B = TrA T B. For random variable ξ, Eξ denotes the expectation of ξ. S n (S n + ) is the set of all real symmetric n × n matrices(positive semidefinite matrices).0 n1×n2 is the n 1 × n 2 matrix of zeros, I r is the r × r identity matrix, and E r st is the r × r matrix with 1 at the stth component and 0 elsewhere.The value of the optimal objective function of an optimization problem We construct a similar scheme provided by A. Beck [2] to obtain a SDP relaxation for (QTTP).By solving the relaxed problem, in this section, we will show that under some mild conditions, tightness of (QTTP) is hold with at most n 2 constraints.
As the method used to homogenize a quadratic vector function, we consider the following homogenization procedure to QTT function: Let like (QTTP).The homogenized QTT function is denoted by f H : R (n1+n2)×n2×n3 → R and given by The homogenous function f H satisfies the following easily verifiable properties, which will become useful in what follows , we call Z ∈ R n2×n2×n3 the identity three order tensor and denote it by I, then f (X ) = f H (X , I); (2). 2) Using the above homogenization procedure for QTT functions, we are able to prove (see Proposition 2.1 below) that (QTTP) is equivalent to some homogeneous QTTP problem.
L. YANG, Q. YANG AND X. ZHAO Proposition 2.1 Consider the following homogenized version of (QTTP): where g stl = ∑ k Z skl Z tkl and δ st is the Kronecker delta, then we have: 1.If (QTTP) is solvable and X * is an optimal solution, then (HQTP) is solvable and (X * ; I) is an optimal solution, where I ∈ R n2×n2×n3 is the identity three order tensor defined in property 2.1.2. If (HQTP) is solvable and (X * ; Z * ) is an optimal solution, then (QTTP) is solvable and D * is an optimal solution, where D * is defined as (2.2).
Proof.Through property 2.1, we note that f (D) = f H (X ; Z) for every feasible solution (X ; Z) of (HQTP), since the system of equalities 1.For every feasible solution (X ; Z) of (HQTP), combined with property 2.1 we have Meanwhile shows that D is a feasible solution of (QTTP), which means f 0 (D) ≥ f 0 (X * ).
From the above equalities, we argue that (X * ; I) is an optimal solution of (HQTP).2. Similar to the above method, for every X in the feasible set of (QTTP), we have , which shows that D * is an optimal solution of (QTTP).
2 Notice that (QTTP) and (HQTP) have the same value of the optimal objective function via the proof of Proposition 2.1, i.e. ( For convenience, we adopt the following matrix notation: ) .
Then we can rewrite f H (X ; Z) as It holds that (HQTP) is equivalent to with Now after some constructions, (QTTP) has been transferred to problem (2.4), where the objective function is in the form of the sum of n 3 individual functions without crossed variables.In fact, problem (2.4) can be viewed as a special kind of separable QMP problems.Based on this characterization, making use of the rank reduction algorithm of [11], we can deduce the tightness between (QTTP) and its SDP relaxation. Let which can be equivalently written as Omitting constraints {rank(U l ) ≤ n 2 } n3 l=1 , we finally arrive at the SDP relaxation of (HQTP): The SDP problem (RHQTP) has a dual, which is given by For the sake of completeness, we present the constructive rank reduction procedure [2,11] again (Proposition 2.2 below) to reach a rank constraint of the optimal solution of (RHQTP).
) are optimal solutions of (RHQTP) and (DHQTP), respectively (existence can be ensured by assuming that both the primal and dual SDPs have interior points in their feasible regions, respectively).Notice that they comply with the complementary conditions and Consider the following system of linear equations: unknowns in the entries of ∆ l ; therefore the system (2.7) has m + n3n2(n2+1)

2
, then there is a nonzero solution of the system of linear (2.7), say, Thus it is easily seen that the matrices l=1 is a feasible solution of (RHQTP).Meanwhile, it is easy to prove that (2.5) (2.6) and (2.7), together with Therefore the complementary conditions are satisfied, which means ( Ū l ) n3 l=1 is still an optimal solution of (RHQTP).Note that i.e. sum of the ranks of the optimal solutions reduce at least one.Repeat the above rank-deduction procedure, finally we will get one optimal solution (U * l ) n3 l=1 of (RHQTP) such that which completes the proof. 2 Using Proposition 2.2, we are now able to show that (QTTP) with at most n 2 constraints possess a tight SDP relaxation under some mild conditions.Theorem 2.1 Suppose that (RHQTP) and its dual (DHQTP) are solvable.If the rank of each optimal solutions of (RHQTP) is no less than n 2 and m ≤ n 2 , then, (QTTP) is solvable and v min (QT T P ) = v min (RHQT P ).

This implies rank(U
As a result, (RHQTP) is equivalent to problem (2.4).Hence by (2.3), we have which completes the proof. 2

Approximation algorithms for the homogeneous QTTP problem
The focus of this section, in particular, is on nonconvex QTTPs which involve positive semidefinite matrices in constraints, and the objective function and constrains both are homogeneous.Through a simple randomized polynomial-time procedure proposed by AMC So et al [14], we can extract a feasible solution of (QTTP) from the optimal solution of its SDP relaxation.And the approximation ratio is Ω( ) which depends on the magnitude of n 2 .
Consider the following optimization problems: where A 0 , A p ∈ S n1 and α p ∈ R for p = 1, • • • , m. we can see that (HQ-1) is (QTTP)'s simplification by omitting the monomial term and the constant term.
QUADRATIC THIRD-ORDER TENSOR OPTIMIZATION 139 Unfortunately, (HQ-1) itself is still NP-hard which has been proved in [10] as n 2 = n 3 = 1.However, it is possible to approximately solve the problem by deriving a feasible solution from its SDP relaxation.And we see that the approximating procedure can be completed in polynomial time.Similar to the argument in section 2, we can rewrite (HQ-1) with another equivalent form: where , and its natural SDP relaxation is given by: Now we introduce a low-rank approximate solution to a system of linear equations in symmetric positive semidefinite matrices, which will be used to derive an approximate solution to our homogeneous QTTP problem (HQ-1).

Lemma 3.1 [14] Let
R n×n be symmetric positive semidefinite matrices, where n < √ 2m.Then, for any d ≥ 1, there exists an X 0 ≽ 0 with rank(X 0 ) ≤ d such that: where: and Moreover, there exists an efficient randomized algorithm for finding such an X 0 .
Our main result in this section is the following: l=1 be an optimal solution to (3.2).We can extract a feasible solution X 0 of (HQ-1) from (U * l ) n3 l=1 such that where: f or n 2 > 12 log(4mh) Proof.Using Proposition 2.2, we know there exists an optimal solution Through the technique used by AMC So et al [14], we construct the following matrices where P * l ∈ R n1×r l with r l = rank(U * l ) < √ 2m, and ξ jl ∈ R r l for ξ jl i are i.i.d.Gaussian random variables with mean 0 and variance 1/n 2 , here then by proof of Lemma 3.1 [14] and some calculations, we assert that Xl , combined with (3.4), easily we get Obviously X l is a feasible solution of problem (3.1).Meanwhile where the second equality follows from (3.3).So there exists certain X l such that Now we transfer matrix sequence {X l } n3 l=1 to a third-order tensor X 0 .Actually the construction process of {X l } n3 l=1 states that here we let and This implies, together with (3.5), that which completes the proof of the theorem. 2 We remark that the constructive proof of Theorem 3.1 actually provides a polynomial-time algorithm to generate an appropriate feasible solution of the homogeneous QTTP problem (HQ-1), and the approximation ratio is Ω( ) which depends on the comparison of n 2 and m.

A class of homogeneous QTTP problems whose SDP relaxations admit no gap with the true optimal values
For the homogeneous QTTP problem, in this section, after making some assumptions to the coefficient matrices in the objective function and constrains, we can see that its optimal value is the same as the bound obtained by its SDP relaxation and the optimal solution for the original problem can be obtained from the optimal solution of its SDP relaxation.Consider a homogeneous QTTP problem of the form: where α p ∈ R and A 0 , A p ∈ S n1 for p = 1, • • • , m.Unlike in (HQ-1), here A 1 , • • • , A m may be indefinite.Stretch the three-order tensor X to n 2 n 3 vectors defined by: . By using the correspondence (4.1), we can represent (HQ-2) as the separable indefinite homogeneous quadratic optimization problem: 2) The so-called SDP relaxation of (4.2) is min Using transfer strategy (4.1), compress n 2 n 3 vectors {x * kl } kl to a third-order tensor X * .Then X * is an optimal solution of (HQ-2) and v min (HQ − 2) = v min (4.3).

Proof. By definition, we first observe that
The above inequality based on the nonpositivity of all offdiagonal elements of A p (0 ≤ p ≤ m) and the fact that all the 2 × 2 principle submatrices of positive semidefinite matrices {Y * kl } kl are nonnegative.Let p = 1, • • • , m, combined with the optimality of (Y * kl ) kl , (4.4) makes it clear that (x * kl ) kl is a feasible solution of (4.2).Therefore, we obtain that Hence (x * kl ) kl is an optimal solution of problem (4.2) which means that X * is an optimal solution of (HQ-2) and v min (HQ − 2) = v min (4.3).

2
The assumption that all off-diagonal elements of A p (0 ≤ p ≤ m) are nonpositive plays an important role in the proof of Theorem 4.1.This however, does not necessarily require that A p ij (i ̸ = j), are of the same sign.In fact, A p (0 ≤ p ≤ m) can be relaxed to be uniformly almost OD-nonpositive.To make this clear, we introduce the following definition.

Definition 4.1 [7] A family of symmetric matrices
As a result, Theorem 4.1 can be extended to the following.√ Y * kl n1n1 ) T for 1 ≤ k ≤ n 2 and 1 ≤ l ≤ n 3 .Using transfer strategy (4.1), compress n 2 n 3 vectors {x * kl } kl to a third-order tensor X * .Then X * is an optimal solution of (HQ-2) and v min (HQ − 2) = v min (4.5).