Interpolation problem for periodically correlated stochastic sequences with missing observations

The problem of mean square optimal estimation of linear functionals which depend on the unobserved values of a periodically correlated stochastic sequence is considered. The estimates are based on observations of the sequence with a noise. Formulas for calculation the mean square errors and the spectral characteristics of the optimal estimates of functionals are derived in the case of spectral certainty, where the spectral densities of the sequences are exactly known. Formulas that determine the least favorable spectral densities and the minimax spectral characteristics are proposed in the case of spectral uncertainty, where the spectral densities of the sequences are not exactly known while some classes of admissible spectral densities are specified.


Introduction
W.R. Bennett [5] in 1958 introduced cyclostationarity as a phenomenon describing signals in channels of communication. Studying the statistical characteristics of information transmission, he calls the group of telegraph signals a cyclostationary process, that is the process whose group of statistics changes periodically with time. W.A. Gardner and L. E. Franks [17] highlighted the similarity of cyclostationary processes, which form a subclass of nonstationary processes, with stationary processes. W.A. Gardner [18], W. A. Gardner, A. Napolitano and L. Paura [19] presented bibliography of works in which properties and applications of cyclostationary processes were studied. Recent developments and applications of cyclostationary signal analysis are reviewed in the papers by A. Napolitano [77], [78]. Note that in different sources cyclostationary processes are called periodically stationary, periodically nonstationary, periodically correlated. We will use the term periodically correlated processes.
E.G. Gladyshev [20] was the first who analysed the spectral properties and representations of periodically correlated sequences based on its connection with the vector valued stationary sequences. He formulated the necessary and sufficient conditions for determining the periodically correlated sequence in terms of the correlation function. A. Makagon [50], [51] presented a detailed spectral analysis of periodically correlated sequences. The main ideas of the research of periodically correlated sequences are outlined in the book by H. L. Hurd and A. Miamee [24]. IRYNA GOLICHENKO AND MIKHAIL MOKLYACHUK 633 In this paper we deal with the problem of optimal linear estimation of the functional A s ζ which depends on the unobserved values of a periodically correlated stochastic sequence ζ(j). Estimates are based on observations of the sequence ζ(j) + θ(j) at points j ∈ Z \ S, where S = ∪ s−1 l=0 {M l + 1, . . . , M l + N l+1 }. θ(j) is an uncorrelated with ζ(j) periodically correlated stochastic sequence. Formulas for calculation the mean square errors and the spectral characteristics of the optimal estimates of the functional A s ζ are proposed in the case of spectral certainty where the spectral densities are exactly known. Formulas that determine the least favorable spectral densities and minimax spectral characteristics are proposed in the case of spectral uncertainty where the spectral densities are not exactly known while some classes of admissible spectral densities are given.
The paper is organized as follows. The spectral properties of periodically correlated stochastic sequences and their correlation functions are described in Section 2. Relations of periodically correlated stochastic sequences with multidimensional stationary sequences are discussed in this section.
In section 3 we consider the problem of mean square optimal linear estimation of the the functional which depends on the unknown values of a T -dimensional stationary stochastic sequence ⃗ ξ(j), based on observations of the sequence ⃗ ξ(j) + ⃗ η(j) at points j ∈ Z \ S, where S = ∪ s−1 l=0 {M l + 1, . . . , M l + N l+1 }. Formulas for calculation the mean square error and the spectral characteristic of the optimal estimate of the functional A s ⃗ ξ are proposed in the case where spectral density matrices of the sequences ⃗ ξ(j) and ⃗ η(j) are exactly known.
In section 4 we consider the problem of mean square optimal linear estimation of the the functional which depends on the unknown values of T-PC stochastic sequence ζ(j), based on observations of the sequence ζ(j) + θ(j) at points j ∈ Z \ S, where S = ∪ s−1 l=0 {M l + 1, . . . , M l + N l+1 }. In section 5 we consider the problem of optimal estimation for the linear functional which depends on the unknown values of T -PC sequence ζ(j) from observations of the sequence ζ(j) + θ(j) at points j ∈ Z \ S, where the number of missed observations at each of the intervals is a multiple of the period T . In sections 4 and 5 the estimation problem is investigated in the case of spectral certainty, where the spectral densities of observed sequences are exactly known.
In section 6 we describe the minimax approach to the problem of estimation of the linear functionals. In this case we find the estimate which minimizes the mean square error for all spectral densities from the given set of admissible densities simultaneously.
In section 7 the least favorable spectral densities and the minimax (robust) spectral characteristics of the optimal estimate of A s ⃗ ζ are found for the class D − 0 of admissible spectral densities. In section 8 the least favorable spectral densities and the minimax (robust) spectral characteristics of the optimal estimate of A s ⃗ ζ are found for the class D − G of admissible spectral densities.

Periodically correlated and multidimensional stationary sequences
The term periodically correlated process was introduced by E. G. Gladyshev [20] while W. R. Bennett [5] called random and periodic processes cyclostationary process.
Periodically correlated sequences are stochastic sequences that have periodic structure (see the book by H. L. Hurd and A. Miamee [24]).
If this is the case, we denote R(n) = {R νµ (n)} T ν,µ=1 and call it the covariance matrix of T-variate stochastic sequence ⃗ ξ(n).
The sequence ⃗ ξ(n) is called generating sequence of the sequence ζ(n).
is stationary.

Hilbert space projection method of linear interpolation
Let ⃗ ξ(j) and ⃗ η(j) be uncorrelated T-variate stationary stochastic sequences with the spectral density matrices } T ν,µ=1 , respectively. Consider the problem of optimal linear estimation of the functional that depends on the unknown values of the sequence ⃗ ξ(j), based on observations of the sequence ⃗ ξ(j) + ⃗ η(j) at Let the spectral densities f ⃗ ξ (λ) and f ⃗ η (λ) satisfy the minimality condition Condition (4) is necessary and sufficient in order that the error-free interpolation of the unknown values of the sequence ⃗ ξ(j) + ⃗ η(j) is impossible [86]. Denote by L 2 (f ) the Hilbert space of vector valued functions ⃗ b(λ) = {b ν (λ)} T ν=1 that are square integrable with respect to a measure with the density

The mean square error
The spectral characteristic ⃗ h(f ⃗ ξ , f ⃗ η ) of the optimal linear estimate of A s ⃗ ξ minimizes the mean square error With the help of the Hilbert space projection method proposed by A. N. Kolmogorov [29] we can find a solution of the optimization problem (7). The optimal linear estimate A s ⃗ ξ is a projection of the functional A s ⃗ ξ on the Condition 2) gives us the possibility to derive the formula for the spectral characteristic of the optimal estimate where

Condition 1) is satisfied when the system of equalities
with elements which are the Fourier coefficients of the matrix functions Making use of the introduced operators, relation (9) can be written in the form of the equation where the k l -th component of the vector ⃗ c s is calculated by the formula The mean-square error of the optimal estimate A ⃗ ξ is calculated by the formula (6) and is of the form where ⟨a, b⟩ denotes the scalar product, R s is the linear operator determined by T ρ × T ρ matrix composed with See [66] for more details.
The following statement holds true.
of the optimal linear estimate of the functional A s ⃗ ξ based on observations of the sequence ⃗ ξ(j) + ⃗ η(j) at points j ∈ Z \ S, are calculated by formulas (8) and (11).
In the case of observations without noise we have the following corollary.
, which satisfies the minimality condition The spectral characteristic ⃗ h(f ⃗ ξ ) and the mean square error ∆(f ⃗ ξ ) of the optimal linear estimate of the functional A s ⃗ ξ based on observations of the sequence ⃗ ξ(j) at points j ∈ Z \ S, are calculated by formulas where The k l -th component of the vector ⃗ c s is calculated by the formula The spectral characteristic ⃗ h(f ⃗ ξ ) and the mean square error ∆(f ⃗ ξ ) of the optimal linear estimate of the functional For more details see [12], [65].

Consider the problem of estimation of the functional
) based on observations of ⃗ ξ(n), n ∈ Z \ {1, 3}. Here ⃗ a(1) = (1, 1), ⃗ a(3) = (−1, −1). In this case the spectral density matrix of ⃗ ξ(n) is ) satisfies the minimality condition (12). The matrix B 2 and its inverse B −1 2 , the vector of unknown coefficients ⃗ c 2 are of the form The spectral characteristic can be calculated by (13) ⃗ Then the optimal linear estimate of A 2 ⃗ ξ determined by (5) is of the form The mean square error of A 2 ⃗ ξ determined by (14) is

Interpolation of T-PC stochastic sequences
Let ζ(j) and θ(j) be uncorrelated T-PC stochastic sequences. Consider the problem of optimal linear estimation of the functional that depends on the unobserved values of T-PC stochastic sequence ζ(j), based on observations of the sequence ζ(j) + θ(j) at points j ∈ Z \ S, where S = ∪ s−1 l=0 {M l + 1, . . . , M l + N l+1 }. Using the Gladyshev relation (2) of PC and multivariate stationary sequences the problem of estimation of the functional A s ζ may be reduced to the problem of estimation of the functional A s ⃗ ξ since where ⃗ a ⊤ (j) = (a 1 (j), . . . , a T (j)) , a ν (j) = a(j)e 2πijν/T , ν = 1, . . . , T, is a T-variate stationary stochastic sequence that generates the PC sequence ζ(j). For the interpolation problem for PC sequences the following results hold true.

Corollary 2
The optimal linear estimate ζ(1) of the unknown value ζ(1), based on observations of the sequence ζ(j) + θ(j) at points j ∈ Z \ S is defined by the formula (15). The spectral characteristic ⃗ h(f ⃗ ξ , f ⃗ η ) and the mean square of the optimal linear estimate ζ(1) are calculated by formulas (8) and (11) In the case of observations without noise we have the following corollary.

Corollary 3
Let ζ(j) be a T-PC stochastic sequence. Then the optimal linear estimate of the functional A s ζ based on observations of the sequence ζ(j) at points j ∈ Z \ S, is given by where ⃗ ξ(j) is generating sequence of ζ(j). The spectral characteristic ⃗ h(f of A s ζ are calculated by formulas (13) and (14), where ⃗ a(j) = (a 1 (j), . . . , a T (j)) ⊤ , a ν (j) = a(j)e 2πijν/T , ν = 1, . . . , T .

Interpolation of T-PC stochastic sequences with special sets of missed observations
Consider the problem of optimal estimation for the linear functional and the number of observations at each of the intervals is a multiple of T and coefficients a(j), j ∈ S are of the form

Using Proposition 2.2, the linear functional A s ζ can be written as follows
where ⃗ a(j) = (a 1 (j), . . . , a T (j)) ⊤ , a ν (j) = a(ν +jT ) = a(j)e 2πijν/T , Taking into account the definition of the functional A s ⃗ ζ and Theorem 1 we can verify that the following statements hold true.

Theorem 3 Let ζ(j) and θ(j) be uncorrelated T-PC stochastic sequences with the spectral density matrices
of T-variate stationary sequences ⃗ ζ(j) and ⃗ θ(j), respectively. Assume that f ⃗ ζ (λ) and f ⃗ θ (λ) satisfy the minimality condition (4). Then the optimal linear estimate of A s ⃗ ζ based on observations of ⃗ ζ(j) + ⃗ θ(j) at points j ∈ Z \S, is given by are orthogonal random measures of the sequences ⃗ ζ(j) where

Corollary 5
Let ζ(j) be a T-PC stochastic sequence with the spectral density matrix f ⃗ ζ (λ) of T-variate stationary sequence ⃗ ζ(j). Assume that f ⃗ ζ (λ) satisfies the minimality condition (12). Then the optimal linear estimate of A s ⃗ ζ based on observations of ⃗ ζ(j) at points j ∈ Z \S, is given by The spectral characteristic ⃗ h(f ⃗ ζ ) and the mean square unknown coefficients ⃗ c(j),j ∈S are determined from the relation
Formulas (19)-(22) may be applied for finding the spectral characteristic and the mean square error of the optimal linear estimate of the functional A s ⃗ ζ only under the condition that the spectral density matrices f (λ) and g(λ) are exactly known. If the density matrices are not known exactly while a set D = D f × D g of possible spectral densities is given, the minimax (robust) approach to estimation of functionals from unknown values of stationary sequences is reasonable. In this case we find the estimate which minimizes the mean square error for all spectral densities from the given set simultaneously.

Definition 3
For a given class of pairs of spectral densities D = D f × D g the spectral density matrices f 0 (λ) ∈ D f , g 0 (λ) ∈ D g are called the least favorable in D for the optimal linear estimation of the functional A s ⃗ ζ if

Definition 4
For a given class of pairs of spectral densities D = D f × D g the spectral characteristic ⃗ h 0 (λ) of the optimal linear estimate of the functional A s ⃗ ζ is called minimax (robust) if

INTERPOLATION PROBLEM FOR PC SEQUENCES WITH MISSING OBSERVATIONS
Taking into consideration these definitions and the obtained relations we can verify that the following lemmas hold true.

Lemma 1
The spectral density matrices f 0 (λ) ∈ D f , g 0 (λ) ∈ D g , that satisfy condition (4), are the least favorable in D for the optimal linear estimation of A s ⃗ ζ, if the Fourier coefficients of the matrix functions The minimax spectral characteristic

Lemma 2
The spectral density matrix f 0 (λ) ∈ D f , that satisfies condition (12), is the least favorable in D f for the optimal linear estimation of A s ⃗ ζ based on observations of the sequence ⃗ ζ(j) at points j ∈ Z \S, if the Fourier coefficients of the matrix function (f 0 (λ)) −1 define the matrix B ζ s 0 , that determine a solution of the constrained optimization problem max The least favorable spectral densities f 0 (λ) ∈ D f , g 0 (λ) ∈ D g and the minimax spectral characteristic ⃗ h 0 = ⃗ h(f 0 , g 0 ) form a saddle point of the function ∆( ⃗ h; f, g) on the set H D × D. The saddle point inequalities The linear functional ∆( ⃗ h(f 0 , g 0 ); f, g) is calculated by the formula The constrained optimization problem (23) is equivalent to the unconstrained optimization problem [83]: where δ((f, g)|D f × D g ) is the indicator function of the set D = D f × D g . A solution of the problem (24) is characterized by the condition 0 ∈ ∂∆ D (f 0 , g 0 ), where ∂∆ D (f 0 , g 0 ) is the subdifferential of the convex functional ∆ D (f, g) at point (f 0 , g 0 ) [85]. The form of the functional ∆( ⃗ h(f 0 , g 0 ); f, g) admits finding the derivatives and differentials of the functional in the space L 1 × L 1 . Therefore the complexity of the optimization problem (24) is determined by the complexity of calculating of subdifferentials of the indicator functions δ((f, g)|D f × D g ) of the sets D f × D g [25].
Taking into consideration the introduced definitions and the derived relations we can verify that the following lemma holds true.

Lemma 3
Let (f 0 , g 0 ) be a solution to the optimization problem (24). The spectral densities f 0 (λ), g 0 (λ) are the least favorable in the class D = D f × D g and the spectral characteristic ⃗ h 0 = ⃗ h(f 0 , g 0 ) is the minimax of the optimal linear estimate of the functional In the case of estimation of the functional based on observations without noise we have the following statement.

Lemma 4
Let f 0 (λ) satisfies the condition (12) and be a solution of the constrained optimization problem dλ.
Then f 0 (λ) is the least favorable spectral density matrix for the optimal linear estimation of A s ⃗ ζ based on observations of the sequence ⃗ ζ(j) at points j ∈ Z \S. The minimax spectral characteristic ⃗ h 0 = ⃗ h(f 0 ) is given by (13)

0
Let ζ(j), j ∈ Z be T -PC sequence and let ⃗ ζ(j) be T -variate stationary sequence, obtained by T -blocking (3) of T -PC sequence ζ(j). Assume that the number of missed observations of the functional A s ⃗ ζ at each of the intervals is a multiple of the period T and the number of observations at each of the intervals is a multiple of T and coefficients a(j), j ∈ S are of the form (17). Consider the problem of minimax estimation of the functional A s ⃗ ζ from observations of the sequence ⃗ ζ(j) at points j ∈ Z \S without noise, under the condition that the spectral density matrix f (λ) of T -variate stationary sequence ⃗ ζ(j) belongs to the set where P = {p νµ } T ν,µ=1 is a given positive definite matrix and detP ̸ = 0. With the help of Lemma 4 and the method of Lagrange multipliers we can find that a solution f 0 (λ) of the constrained optimization problem (25) satisfy the following relation: where ⃗ α = (α 1 , . . . , α T ) ⊤ is a vector of Lagrange multipliers, is positive definite and has nonzero determinant. Then the least favorable in the class D − 0 spectral density for the optimal linear estimate of A s ⃗ ζ is given by the formula The minimax spectral characteristic ⃗ h(f 0 ) is given by (28).The greatest value of the mean square error of A s ⃗ ζ is calculated by the formula
The least favorable spectral density in the class D − 0 for the optimal linear estimate of A 2 ⃗ ζ by (29) is of the form ) .
The minimax spectral characteristic, calculated by (28), is given by the formula The greatest value of the mean square error of A 2 ⃗ ζ takes value ∆(f 0 ) = 10 9 .
of the vector stochastic autoregression sequence of the order G G ∑ g=0 Q(g) ⃗ ζ(l − g) = ⃗ ε l .