Interpolation Problem for Multidimensional Stationary Processes with Missing Observations

The problem of the mean-square optimal linear estimation of linear functionals which depend on the unknown values of a multidimensional continuous time stationary stochastic process is considered. Estimates are based on observations of the process with an additive stationary stochastic noise process at points which do not belong to some finite intervals of a real line. The problem is investigated in the case of spectral certainty, where the spectral densities of the processes are exactly known. Formulas for calculating the mean-square errors and spectral characteristics of the optimal linear estimates of functionals are proposed under the condition of spectral certainty. The minimax (robust) method of estimation is applied in the case of spectral uncertainty, where spectral densities of the processes are not known exactly while some sets of admissible spectral densities of the processes are given. Formulas that determine the least favorable spectral densities and the minimax spectral characteristics of the optimal estimates of functionals are proposed for some special sets of admissible spectral densities.


Introduction
The problem of estimation of the unknown values of stochastic processes is of constant interest in the theory and applications of stochastic processes.The description of the problem and effective methods of solution of the interpolation, extrapolation and filtering problems for stationary stochastic sequences and processes were were developed by Kolmogorov [17], Wiener [40] and Yaglom [41,42].Further development of the methods of solution and analysis of the problems can be found in the works by Rozanov [37] and Hannan [12], Box et.al [3], Brockwell and Davis [4].The basic assumption of most of the methods of solution of the problem of estimation of the unobserved values of stochastic processes is that the spectral densities of the considered stochastic processes are exactly known.However, in practice these methods can not be applied since the complete information on the spectral densities is impossible in most cases.In order to solve the problem, the parametric or nonparametric estimates of the unknown spectral densities are found.Then, one of the traditional estimation methods is applied, provided that the selected spectral densities are true.This procedure can result in significant increase in the value of the error of estimate as Vastola and Poor [39] have demonstrated with the help of some examples.To avoid this effect one can search estimates which are optimal for all densities from a certain class of admissible spectral densities.These estimates are called minimax since they minimize the maximum value of 119 the error.The paper by Grenander [11] should be marked as the first one where this approach to extrapolation problem for stationary processes was proposed.Several models of spectral uncertainty and the minimax-robust methods of data processing can be found in the survey paper by Kassam and Poor [16].In their papers, Franke [7,8], Franke and Poor [9] presented results of the investigation of the minimax extrapolation and filtering problems for stationary sequences with the help of convex optimization methods.This approach makes it possible to find relations that determine the least favorable spectral densities for different classes of admissible spectral densities.The papers by Moklyachuk [23,24] is dedicated to the investigation of the problems of the optimal linear estimation of functionals which depend on the unknown values of stationary sequences and processes.The problem of estimation of functionals which depend on the unknown values of multivariate stationary stochastic processes is the aim of the papers by Moklyachuk and Masyutka [26] - [28].In the book by Moklyachuk and Golichenko [25] results of investigation of the interpolation, extrapolation and filtering problems for periodically correlated stochastic processes are described.Estimation problems for functionals which depend on the unknown values of the stochastic processes with stationary increments were investigated by Luz and Moklyachuk [18]- [21].The problem of interpolation, extrapolation and filtering for stationary processes with missing observations were investigated by Moklyachuk and Sidei [29] - [32].The prediction and interpolation problems for stationary stochastic sequences with missing observations were investigated in papers by Bondon [1,2], Cheng, Miamee and Pourahmadi [5], Cheng and Pourahmadi [6], Kasahara, Pourahmadi and Inoue [15], Pourahmadi, Inoue and Kasahara [34], Pelagatti [33].The problem of interpolation of stationary sequences was investigated in the paper of Salehi [38].
In this paper we deal with the problem of the mean-square optimal linear estimation of the functional which depends on the unknown values of a multidimensional continuous time stationary stochastic process ⃗ ξ(t) from observations of the process with an additive stationary stochastic noise process ⃗ ξ(t) + ⃗ η(t) at points t ∈ R\S, The case of spectral certainty as well as the case of spectral uncertainty are considered.
In the case of spectral certainty, where the spectral densities of the processes are exactly known, formulas for calculating the spectral characteristic and the mean square error of the optimal linear estimate of the functional are derived.In the case of spectral uncertainty, where the spectral densities are not exactly known while a set of admissible spectral densities is given, the minimax method is applied.This method allows us to find estimates that minimize the maximum values of the mean-square errors of estimates for all spectral density matrices from a given class of admissible spectral density matrices.Formulas that determine the least favorable spectral densities and the minimax-robust spectral characteristics of the optimal estimates of the functional are proposed for some specific classes of admissible spectral densities.
2. Hilbert space projection approach to interpolation problem for stationary processes with missing observations , t ∈ R, be uncorrelated mean square continuous multidimensional stationary stochastic processes with zero first moments, E ⃗ ξ(t) = ⃗ 0, E⃗ η(t) = ⃗ 0, and correlation functions which admit the spectral decomposition (see Gikhman and Skorokhod [10]) where F (λ) = {f kl (λ)} T k,l=1 and G(λ) = {g kl (λ)} T k,l=1 are spectral density matrices of the processes ⃗ ξ(t) and ⃗ η(t) respectively which satisfy the minimality condition is a nontrivial function of the exponential type.Under this condition the errorfree interpolation of the process ⃗ ξ(t) + ⃗ η(t) is impossible (see Rozanov [37]).The stationary processes ⃗ ξ(t) and ⃗ η(t) admit the spectral decompositions (see Gikhman and Skorokhod [10], Karhunen [14]) where Z ξ (dλ) and Z η (dλ) are vector valued orthogonal stochastic measures such that Consider the problem of the mean-square optimal linear estimation of the functional We will assume that the function ⃗ a(t) satisfies the following conditions ( This condition ensures that the functional A s ⃗ ξ has a finite second moment.Making use of the spectral decomposition (2) of the stationary process ⃗ ξ(t), we can represent the functional A s ⃗ ξ in the following form Denote by Âs ⃗ ξ the optimal linear estimate of the functional and denote by the mean-square error of the estimate Âs ⃗ ξ.Since the spectral densities of the stationary processes ⃗ ξ(t) and ⃗ η(t) are suppose to be known, we can use the method of orthogonal projections in Hilbert spaces proposed by Kolmogorov [17] to find the estimate Âs ⃗ ξ.
Consider values ξ k (t), k = 1, . . ., T, t ∈ R and η k (t), k = 1, . . ., T, t ∈ R as elements of the Hilbert space H = L 2 (Ω, F, P ) generated by random variables ξ with zero mathematical expectations, Eξ = 0, finite variations, E|ξ| 2 < ∞, and inner product (ξ, η) = Eξη.Denote by H s (ξ + η) the closed linear subspace generated by Denote by L 2 (F + G) the Hilbert space of vector-valued functions ⃗ a(λ and denote by L s 2 (F + G) the subspace of the space L 2 (F + G) generated by the functions The mean-square optimal linear estimate Âs ⃗ ξ of the functional A s ⃗ ξ can be represented in the form where is the spectral characteristic of the estimate.The mean-square error ∆(h; F, G) of the estimate Âs ⃗ ξ is given by the formula According to the Hilbert space projection method the optimal estimate of the functional A s ⃗ ξ is a projection of the element A s ⃗ ξ of the space H on the subspace H s (ξ + η).It can be found from the following conditions: It follows from the second condition that the spectral characteristic h(λ) = {h k (λ)} T k=1 of the optimal linear estimate Âs ⃗ ξ for any t ∈ R\S satisfies the equation The last relation is equivalent to equations Define the function and its Fourier transform and the spectral characteristic of the estimate Âs ⃗ ξ is of the form It follows from the first condition, Âs ⃗ ξ ∈ H s (ξ + η), which determine the optimal linear estimate of the functional A s ⃗ ξ, that the following relation holds true The last relation can be represented in terms of linear operators in the space L 2 (S).Let us define operators Making use of the introduced operators, relation ( 6) can be represented in the form Suppose that the operator B s is invertible (see the paper by Salehi [38] for more details).Then the function ⃗ c(t) can be calculated by the formula Consequently, the spectral characteristic h(λ) of the estimate Âs ⃗ ξ is calculated by the formula The mean-square error of the estimate Âs ⃗ ξ can be calculated by the formula where ⟨a(t), b(t)⟩ is the inner product in the space L 2 (S).
Let us summarize the obtained results and present them in the form of a theorem.

Theorem 2.1
Let { ⃗ ξ(t), t ∈ R} and {⃗ η(t), t ∈ R} be uncorrelated multidimensional stationary stochastic processes with the spectral density matrices F (λ) and G(λ) which satisfy the minimality condition (1).Suppose that condition (3) is satisfied and the operator B s is invertible.The spectral characteristic h(λ) and the mean-square error ∆(F, G) of the optimal linear estimate of the functional A s ⃗ ξ which depends on the unknown values of the process ⃗ ξ(t) based on observations of the process ⃗ ξ(t) + ⃗ η(t) at points t ∈ R\S can be calculated by formulas ( 8), (9).
Consider the case where the stationary process ⃗ ξ(t) is observed without noise.In this case the spectral characteristic of the estimate Âs ⃗ ξ is of the form the relation ( 7) is of the form ⃗ a(t) = (B s c)(t), t ∈ S. (11) Suppose that the operator B s is invertible.Then the unknown function ⃗ c(t) can be calculated by the formula and the spectral characteristic of the estimate Âs ⃗ ξ is of the form The mean-square error of the estimate can be calculated by the formula The following theorem holds true.

Theorem 2.2
Let { ⃗ ξ(t), t ∈ R} be a multidimensional stationary stochastic process with the spectral density matrix F (λ), which satisfies the minimality condition INTERPOLATION OF MULTIDIMENSIONAL PROCESS WITH MISSING OBSERVATIONS for a nonzero vector-valued function of the exponential type b , can be calculated by formulas (12), (13).

Minimax approach to interpolation problem for stationary processes with missing observations
The derived formulas can be applied only in the case of spectral certainty, where the spectral density matrices of the considered processes are exactly known.However, in practice, we do not have exact values of the spectral density matrices while, instead, we have a class of admissible spectral density matrices, where the spectral density matrices of the processes belong to.In this case, the minimax method can be applied to estimate the value of the functional.This method allows us to find estimates that minimize the maximum values of the mean-square errors of the estimates for all spectral density matrices from a given class of admissible spectral density matrices.
Definition 3.1.For a given class of spectral densities are called the least favorable in the class D for the optimal linear interpolation of the functional A s ⃗ ξ if the following relation holds true Definition 3.2.For a given class of spectral densities D = D F × D G the spectral characteristic h 0 (λ) of the optimal linear interpolation of the functional A s ⃗ ξ is called minimax-robust if there are satisfied conditions From the introduced definitions and formulas derived in previous section we can obtain the following statements.

Lemma 3.1
The spectral densities F 0 (λ) ∈ D F , G 0 (λ) ∈ D G satisfying the minimality condition (1) are the least favorable in the class D = D F × D G for the optimal linear interpolation of the functional A s ⃗ ξ, if the Fourier coefficients of the functions The minimax spectral characteristic Let the spectral density F 0 (λ) ∈ D F satisfy the minimality condition (14).The spectral density F 0 (λ) ∈ D F is the least favorable in the class D F for the optimal linear interpolation of the functional A s ⃗ ξ from observations of the process ⃗ ξ(t) at points t ∈ R\S if the Fourier coefficients of the function (F 0 (λ)) −1 define the operator B 0 s which determines a solution of the constrained optimization problem The minimax spectral characteristic h 0 = h(F 0 ) is determined by the formula (12 For more detailed analysis of properties of the least favorable spectral densities and the minimax-robust spectral characteristics we observe that the least favorable spectral densities F 0 (λ), G 0 (λ) and the minimax spectral characteristic h 0 = h(F 0 , G 0 ) form a saddle point of the function ∆ (h; F, G) on the set H D × D. The saddle point inequalities The linear functional The constrained optimization problem ( 17) is equivalent to the unconstrained optimization problem (see Pshenichnyj [35]) where , which is the necessary and sufficient condition under which the pair (F 0 , G 0 ) belongs to the set of minimums of the convex functional ∆(h(F 0 , G 0 ); F, G).This condition makes it possible to find the least favourable spectral densities in some special classes of spectral densities D (see books by Ioffe and Tihomirov [13], Pshenichnyj [35], Rockafellar [36]).Note, that the form of the functional ∆(h(F 0 , G 0 ); F, G) is convenient for application of the Lagrange method of indefinite multipliers for finding solution to the problem (18).Making use of the method of Lagrange multipliers and the form of subdifferentials of the indicator functions we describe relations that determine the least favourable spectral densities in some special classes of spectral densities (see books by Moklyachuk [22,23], Moklyachuk and Masyutka [28] for additional details).
Taking into consideration the introduced definitions and the derived relations we can verify that the following lemma holds true.

INTERPOLATION OF MULTIDIMENSIONAL PROCESS WITH MISSING OBSERVATIONS
Lemma 3.2 Let (F 0 , G 0 ) be a solution of the optimization problem (18).The spectral densities F 0 (λ), G 0 (λ) are the least favorable in the class D = D F × D G and the spectral characteristic h 0 = h(F 0 , G 0 ) is the minimax of the optimal linear estimate of the functional

Least favorable spectral densities in the class
Consider the problem of mean-square optimal interpolation of the functional A s ⃗ ξ in the case where the spectral density matrices of the processes are not exactly known while the admissible spectral density matrices are from the class D = D 0 × D ε , where Here G 1 (λ) is a known and fixed spectral density matrix while W (λ) is an unknown spectral density matrix, p, q, p k , q k , k = 1, T are fixed numbers, P, Q, B 1 , B 2 are fixed positive definite Hermitian matrices.The class D ε describes "ε-contamination" model of stochastic processes.
From the condition 0 ∈ ∂∆ D (F 0 , G 0 ) we find the following equations which determine the least favourable spectral densities for these sets of admissible spectral densities.

Theorem 4.1
The least favorable spectral densities F 0 (λ), G 0 (λ) in the classes D 0 × D ε for the optimal linear interpolation of the functional A s ⃗ ξ are determined by relations (19), (20) for the first pair D 1 0 × D 1 ε of sets of admissible spectral densities; (21), (22) for the second pair D 2 0 × D 2 ε of sets of admissible spectral densities; ( 23), (24) for the third pair D 3 0 × D 3 ε of sets of admissible spectral densities; (25), (26) for the fourth pair D 4 0 × D 4 ε of sets of admissible spectral densities; the minimality condition (1); the constrained optimization problem (15) and restrictions on densities from the corresponding classes D 0 × D ε .The minimax-robust spectral characteristic of the optimal estimate of the functional A s ⃗ ξ is determined by the formula (8).

Corrolary 4.1
The least favorable spectral densities F 0 (λ) in the classes D k 0 , k = 1, 2, 3, 4, for the optimal linear interpolation of the functional A s ⃗ ξ, which depends on the unknown values of the process ⃗ ξ(t) based on observations of the process ⃗ ξ(t) at points t ∈ R\S, are determined by the following equations, respectively, the minimality condition (14); the constrained optimization problem ( 16) and restrictions on densities from the corresponding classes D k 0 , k = 1, 2, 3, 4. The minimax spectral characteristic of the optimal estimate of the functional A s ⃗ ξ is determined by the formula (12).

Conclusions
In this article we propose methods of the mean-square optimal linear interpolation of functionals which depend on the unknown values of a multidimensional continuous time stationary stochastic process based on observations of the process with an additive stationary stochastic noise process.The case of spectral certainty as well as the case of spectral uncertainty are considered.Under the condition of spectral certainty, where the spectral density matrices of the stationary processes are exactly known, we derive formulas for calculating the spectral characteristic and the mean-square error of the optimal estimate of the functional.The corresponding results are derived in the case of observations of the process without noise.In the case of spectral uncertainty, where the spectral density matrices of the stationary processes are not exactly known while some sets of admissible spectral density matrices are given, we apply the minimax-robust method of estimation.This method allows us to find estimates that minimize the maximum values of the mean-square errors of the estimates for all spectral density matrices from a given class of admissible spectral density matrices and derive relations which determine the least favourable spectral density matrices.These least favourable spectral density matrices are solutions of the optimization problem (18) which is characterized by the condition 0 ∈ ∂∆ D (F 0 , G 0 ), where ∂∆ D (F 0 , G 0 ) is the subdifferential of the convex functional ∆ D (F, G) at point (F 0 , G 0 ).The form of the functional ∆(h(F 0 , G 0 ); F, G) is convenient for application of the Lagrange method of indefinite multipliers for finding solution to the problem (18).Making use of the method of Lagrange multipliers and the form of subdifferentials of the indicator functions we describe relations that determine the least favourable spectral densities in some special classes of spectral densities.
Stat., Optim.Inf.Comput.Vol. 7, March 2019 O. MASYUTKA, M. MOKLYACHUK AND M. SIDEI 125 Corrolary 3.1 March 2019 122 INTERPOLATION OF MULTIDIMENSIONAL PROCESS WITH MISSING OBSERVATIONS It follows from the relation (4) that the function ⃗ c(t) is nonzero only on the set S. Hence, the function C s (λ) is of the form dλ, t ∈ R. Stat., Optim.Inf.Comput.Vol. 7, e itλ dt.Suppose that condition (3) is satisfied and the operator B s is invertible.The spectral characteristic h(λ) and the mean-square error ∆(F ) of the optimal linear estimate of the functional A s ⃗ ξ which depends on the unknown values of the process ⃗ s are Lagrange multipliers, δ kl are Kronecker symbols, γ k (λ) ≤ 0 and γ k