Filtering Problem for Functionals of Stationary Sequences

The problem of the mean-square optimal linear estimation of functionals which depend on the unknown values of a stationary stochastic sequence from observations of the sequence with noise is considered. In the case of spectral certainty, where the spectral densities of the sequences are exactly known, we propose formulas for calculating the spectral characteristic and value of the mean-square error of the estimate, which are determined using the Fourier coefficients of some functions from the spectral densities. The minimax-robust method of estimation is applied in the case of spectral uncertainty, where the spectral densities are not exactly known, but a class of admissible spectral densities is given. Formulas for determining the least favorable spectral densities and the minimax-robust spectral characteristics of the optimal estimates of the functionals are proposed for some specific classes of admissible spectral densities.


Introduction
A great number of scientific papers are dedicated to investigation of properties of stationary stochastic processes.Extensive expositions of the theory of stationary stochastic processes can be found, for example, in the books by Gikhman and Skorokhod [9], Hannan [12], Rozanov [35], Yaglom [38,39].The developed theory of stationary processes has a lot of theoretical and practical applications in engineering, econometrics, finance etc. See, for example, the book by Box, Jenkins and Reinsel [3].An important problem of the theory is the problem of estimation (extrapolation, interpolation and filtering) of the unobserved values of a stationary process based on observations of the process or on observations of another process which is stationary related with the process under investigation.Constructive methods of extrapolation, interpolation and filtering of stationary processes were developed by Kolmogorov [17], Wiener [37], Yaglom [38,39].For some recent results see papers by Bondon [1], [2], Kasahara, Pourahmadi and Inoue [14], [32], Frank and Klotz [6], Klotz and Mädler [16], Lindquist [18].
The crucial assumption of most of the methods developed for estimation of the unobserved values of stochastic processes is that the spectral densities of the involved stochastic processes are exactly known.However, in practice complete information on the spectral densities is impossible in most cases.In this situation one finds parametric or nonparametric estimate of the unknown spectral density and then apply one of the traditional estimation methods provided that the estimated density is the true one.This procedure can result in significant increasing of the value of error as Vastola and Poor [36] have demonstrated with the help of some examples.To avoid this effect one can search estimates which are optimal for all densities from a certain class of admissible spectral densities.These 69 estimates are called minimax since they minimize the maximum value of the error.Grenander [11] was the first one who applied this method to the extrapolation problem for stationary processes.Franke [7], Franke and Poor [8] investigated the minimax interpolation and extrapolation problem for stationary sequences by using convex optimization techniques.For more details see a survey paper by Kassam and Poor [15].
In the papers by Moklyachuk [23] - [28], Moklyachuk and Sidei [31] extrapolation, interpolation and filtering problems for stationary stochastic processes and sequences are investigated.The corresponding problems for vector-valued stationary sequences and processes are investigated by Moklyachuk and Masyutka [29], [30].In the article by Dubovets'ka and Moklyachuk [4], [5] and in the book by Golichenko and Moklyachuk [10] the minimax estimation problems are investigated for another generalization of stationary processes -periodically correlated stochastic sequences and stochastic processes.
Particularly, in the paper [23] the minimax-robust filtering problem for stationary stochastic sequences from observations with noise is considered.The problem is solved under the condition that the spectral densities f (λ) and f (λ) + g(λ) admit the canonical factorizations.An estimate is found as a solution to the minimization problem by using convex optimization methods.Formulas are proposed that determine the least favorable spectral densities and the minimax-robust spectral characteristics of the optimal estimates for some special classes of admissible spectral densities.
In this paper we deal with the problem of the mean-square optimal linear estimation of the functionals which depend on the unknown values of a stationary stochastic sequence ξ(k) with the spectral density f (λ) from observations of the sequence ξ(k) + η(k) at time k = 0, −1, −2, . .., where η(k) is an uncorrelated with the sequence ξ(k) stationary stochastic sequence with the spectral density g(λ).In contrast to the paper [23], in this paper we propose solution of the problem using the method of projection in the Hilbert space of random variables with finite second moments proposed by Kolmogorov [17].The derived spectral characteristics are determined with the help of operators in the space ℓ 2 constructed from the Fourier coefficients of the functions f (λ)+g(λ) .In the case of spectral uncertainty, where the spectral densities are not exactly known while a set of admissible spectral densities is specified, formulas for determination the least favorable spectral densities and the minimax-robust spectral characteristics of the optimal estimates of the functionals Aξ and A N ξ are proposed for some specific classes of admissible spectral densities.Such approach to filtering problems for stochastic sequence with stationary nth increments was applied in the papers by Luz and Moklyachuk [20], [21].The method of projection in the Hilbert space is also applied to extrapolation and interpolation problems for stochastic sequences with stationary nth increments in papers by Luz and Moklyachuk [19], [22].

Classical filtering problem
Consider a stationary stochastic sequence {ξ(m) : m ∈ Z} with an absolutely continuous spectral function F (λ) which has the spectral density function f (λ).Let {η(m) : m ∈ Z} be a stationary stochastic sequence which is uncorrelated with {ξ(m) : m ∈ Z} and has an absolutely continuous spectral function G(λ) with the spectral density function g(λ).The stationary stochastic sequences {ξ(m) : m ∈ Z} and {η(m) : m ∈ Z} admit the following spectral representations [9]: where Z ξ (λ) and Z η (λ) are two random processes with uncorrelated increments defined on [−π, π) that correspond to the spectral functions F (λ) and G(λ).
We will suppose that the spectral density functions f (λ) and g(λ) satisfy the minimality condition FILTERING PROBLEM FOR FUNCTIONALS OF STATIONARY SEQUENCES Suppose that we have observations of the sequence ξ(m) + η(m) at time m = 0, −1, −2, . ... The problem is to find the mean-square optimal linear estimates of the functionals which depend on the unknown values of the sequence ξ(k), k ≤ 0.
Define by Aξ and A N ξ the mean-square optimal linear estimates of the values of the functionals Aξ and A N ξ based on observations of the sequence ξ(m) + η(m) at time m = 0, −1, −2, . ... The mean-square errors of the estimates Aξ and A N ξ are determined as ∆(f, g, Aξ) To find the mean-square optimal estimate of the functional Aξ we use the method of projection in Hilbert space proposed by Kolmogorov [17].
Suppose that the coefficients {a(k) : k ≥ 0} defining the functional Aξ satisfy the following conditions: The first condition ensures that the functional Aξ has a finite second moment.The second condition ensures that a linear operator A in the space ℓ 2 to be defined below is compact.
Denote by H 0 (ξ + η) the closed linear subspace generated by elements {ξ(m) + η(m) : m = 0, −1, −2, . ..} in the space H = L 2 (Ω, F, P) of random variables with 0 mathematical expectations, Eγ = 0, and finite variations, E|γ| 2 < ∞.Denote by L 0 2 (f + g) a subspace in the space L 2 (f + g), which is generated by functions The estimate Aξ of the functional Aξ has the representation where h(λ) is the spectral characteristic of the estimate Aξ.The optimal estimate Aξ is a projection of the element Aξ of the space H on the subspace H 0 (ξ + η).It is determined by the following conditions: Condition 2) implies that the spectral characteristic h(λ) of the optimal estimate Aξ satisfies the following equalities for all k ≤ 0 which leads to the condition It follows from the last equality that the spectral characteristic of the estimate Aξ satisfies the equality where {c(k) : k ≥ 0} are coefficients we have to find.Equality (4) implies that the spectral characteristic of the estimate Aξ can be presented in the following form .
Let us find the coefficients {c(k) : k ≥ 0}.Condition 1) implies that the Fourier coefficients of the function h(λ) equal to zero if l > 0. Therefore the following relations hold true: ) e −iλl dλ = 0, l ≥ 1.
We can write these relations in the form Let us introduce the Fourier coefficients of the functions Using the introduced notations we can verify that equality (5) is equivalent to the following system of equations: which admits the matrix representation Ra = Pc, where c = (c(0), c(1), c(2), . ..) ′ , a = (a(0), a(1), a(2), . ..) ′ , P, R are linear operators in the space ℓ 2 defined by matrices with coefficients where (P −1 Ra) k is the kth element of the vector P −1 Ra.
The derived expressions allow us to conclude that the spectral characteristic h(λ) of the optimal estimate Aξ can be calculated by the formula The value of the mean-square error of the estimate Aξ of the functional Aξ can be calculated by the formula where Q is a linear operator in the space ℓ 2 defined by the matrix We should note that it might be difficult to find the inverse operator P −1 if the operator P is defined using the Fourier coefficients of the function . However, the operator P −1 can be easily found if the sequence ξ(m) + η(m) admits the canonical moving average representation.In this case the function f (λ) + g(λ) admits the canonical factorization [12] Lemma 1 Suppose that canonical factorization (8) holds true.Let the linear operators Ψ and Θ in the space ℓ 2 be defined by the matrices with elements Then the linear operator P in the space ℓ 2 admits the factorization P = Ψ ′ Ψ and the inverse operator P −1 admits the factorization Proof.Factorization (8) implies Thus, In the case i ≥ j we have And in the case i < j we have which proves the factorization P = Ψ ′ Ψ. Factorization (P) −1 = ΘΘ ′ comes from the relation ΨΘ = ΘΨ = I which has to be proved.If follows from factorization (8) that which leads to the following equalities finishing the proof of the lemma: Summing up the relations described above, we can formulate the following theorem.∑ ∞ k=0 a(k)ξ(−k) which depends on the unknown values of the stationary sequence ξ(k), k ≤ 0, based on observations of the sequence ξ(m) + η(m) at time m = 0, −1, −2, . . . is calculated by the formula (3).The spectral characteristic h(λ) and the value of the mean-square error ∆(f, g; Aξ) of the optimal estimate Aξ are calculated by formulas ( 6) and ( 7) respectively.In the case where the spectral density f (λ) + g(λ) of the sequence ξ(m) + η(m) admits canonical factorization (8), the operator P −1 in formulas (6), ( 7) can be presented as (P) −1 = ΘΘ ′ .
To derive the estimate of the functional which depends on the unknown values of the stationary stochastic sequence ξ(k) at time k = 0, −1, −2, . . ., −N we can use theorem 1.Let us consider the vector a N = (a(0), a(1), . . ., a(N ), 0, . ..) ′ to find the spectral characteristic h N (λ) and the mean-square error ∆(f, g; A N ξ) of the estimate of the functional A N ξ.In this case the estimate A N ξ does not depend on the elements (R) l,m of the operator R for m > N .Define a linear operator R N in the space ℓ 2 by the matrix with elements Then the spectral characteristic of the optimal estimate is calculated by the formula where The value of the mean-square error of the optimal estimate A N ξ is calculated by the formula where Q N is a linear operator in the space ℓ 2 which is defined by the matrix with elements The following theorem holds true.
FILTERING PROBLEM FOR FUNCTIONALS OF STATIONARY SEQUENCES Theorem 2 allows us to find an estimate of the unknown ξ(p) at point p, p ≤ 0, based on observations of the sequence ξ(m) + η(m) at time m = 0, −1, −2, . ... Consider the vector a N with 1 at the coordinate (−p) and 0 at the rest of coordinates, and put it in (6).We get that the spectral characteristic φ m (λ) of the optimal estimate can be presented by the formula where r p = (R 0,−p , R 1,−p , . ..).The value of the mean-square error of the estimate ξ(p) can be calculated by the formula Thus, the following corollary holds true.
Let us find the mean-square optimal estimate ξ(0) of the value ξ(0) of the stochastic sequence ξ(m) based on observations of the sequence ξ(m) + η(m) at time m = 0, −1, −2, . ... This problem is known as the smoothing of stochastic sequence.

Denote by r(
. They satisfy the property r(k) = r(−k), k ∈ Z, where r(k) is the complex conjugate to r(k).Denote by {V k,j : k, j ≥ 0} elements of the matrix which define the operator V = (P) −1 .Then where δ kj is the Kronecker delta.It follows from relations (14) and ( 16) that the spectral characteristic φ(λ) of the estimate ξ(0) of the element ξ(0) is of the form where Y is a linear operator in the space ℓ 2 defined by the matrix with elements (Y) k,l = P k,−l , k ≥ 0, l ≥ 0. (YVr 0 ) k is the kth element of the vector YVr 0 .The optimal estimate of the element ξ(0) is calculated by the formula The value of the mean-square error of the estimate is calculated by the formula where {g(k) : k ∈ Z} are the Fourier coefficients of the spectral density g(λ).
Formulas ( 6) and ( 7) for calculating the spectral characteristic and the value of the mean-square error of the estimate Aξ of the functional Aξ have been found by the method of orthogonal projection in Hilbert space.Let us show that these formulas are equivalent to the formulas obtained in the paper [23].Suppose that canonical factorization (8) holds true as well as the canonical factorization Let G be a linear operator in the space ℓ 2 determined by the matrix with elements Lemma 2 Suppose the functions (f (λ) + g(λ)) −1 and f (λ) admit canonical factorizations ( 8) and ( 19) respectively.Let the linear operators Ψ and Φ in the space ℓ 2 be determined by matrices with elements (Ψ) k,j = ψ(k − j) and admits the canonical factorization where b) the linear operator Υ in the space ℓ 2 defined by the matrix with elements (Υ) k,j = υ(k − j) for 0 ≤ j ≤ k, (Υ) k,j = 0 for 0 ≤ k < j can be presented as Υ = ΨΦ = ΦΨ.

Proof. Statement a) follows from the equalities
Statement b) follows from the equalities holding true for i ≥ j:

Lemma 3
Suppose that canonical factorizations ( 8), ( 19) hold true and let the linear operators Φ and Υ be defined in the same way as in lemma 2. Define the linear operator T in the space ℓ 2 by the matrix with elements (T) l,k = s(l − k), l, k ≥ 0, where the coefficients s(k), k ≥ 0, are defined in (20).Then operators T and G in the space ℓ 2 admit the factorizations T = Υ ′ Υ and G = Φ ′ Φ.
Proof.The proof of the lemma is the same as the proof of factorization of the operator P in lemma 1. 2 Lemmas 1, 2 and 3 can be applied to modify formulas ( 6) and (7).Put e = Θ ′ Ra.Canonical factorization (8 m+1) , where e(m) = (Θ ′ Ra) m , m ≥ 0, is the mth element of the vector e = Θ ′ Ra.Since the following equality holds true: Factorization (20) and relation ( 9) let us make the following transformations: Relations ( 21) and ( 22) allow us to write the following formula for calculating the spectral characteristic h(λ) of the optimal estimate Aξ using coefficients from canonical factorizations ( 8), (19): The element (Cψ) m , m ≥ 0, from the last relation is the mth element of the vector Cψ = Ψ ′ Ga, ψ = (ψ(0), ψ(1), ψ(2), . ..) ′ , C is a linear operator defined by the matrix with elements (C) k,j = c(k + j), k, j ≥ 0, c = Ga, G is a linear operator defined by the matrix with elements (G) k,j = f (k − j), k, j ≥ 0. Lemma 3 provides the representation G = Φ ′ Φ of the operator G, where Φ is a linear operator defined by the matrix with elements The value of the mean-square error ∆(f, g; Aξ) of the estimate Aξ is calculated by the formula h(e iλ )A(e iλ )f (λ)dλ = ⟨Ga, a⟩ − ⟨Cψ, Cψ⟩.(24) In such a way we justify an approach to estimate of the functional Aξ which uses the method of orthogonal projection in Hilbert space to prove the following result which was obtained in [23] by direct minimizing the value of the mean-square error of an arbitrary estimate of the functional Aξ.
Minimality condition (1) holds true if |y| < 1.To determine the spectral characteristic of the optimal estimate of the functional A 1 ξ we use formula (11).The operator P is defined by the coefficients , where Let the spectral characteristic of the estimate A 1 ξ be presented in the form Then the optimal estimate of the functional where

FILTERING PROBLEM FOR FUNCTIONALS OF STATIONARY SEQUENCES
The value of the mean-square error is If we take a = 0, b = 1 then we will have where Putting a = 1, b = 0 we obtain solution to the smoothing problem: where

Minimax-robust method of filtering
The value of the mean-square error ∆(h(f, g); f, g) := ∆(f, g; Aξ) and the spectral characteristic h(λ) of the optimal linear estimate Aξ of the functional Aξ which depends on the unknown values of the sequence ξ(k) based on observations of the sequence ξ(m) + η(m) are calculated by formulas (6) and (7) if the spectral densities f (λ) and g(λ) of the stochastic sequences ξ(m) and η(m) are exactly known.In the case where a set D = D f × D g of admissible spectral densities is given, it is reasonable to apply the minimax-robust method of filtering of the functionals which consists in minimizing the value of the mean-square error for all spectral densities from the given class D = D f × D g .For description of the proposed method we propose the following definitions [23].

Definition 1
For a given class of spectral densities D = D f × D g the spectral densities f 0 (λ) ∈ D f , g 0 (λ) ∈ D g are called least favorable in the class D for the optimal linear filtering of the functional Aξ if the following relation holds true

Definition 2
For a given class of spectral densities D = D f × D g the spectral characteristic h 0 (λ) of the optimal linear estimate of the functional Aξ is called minimax-robust if there are satisfied conditions From the introduced definitions and formulas derived in the previous section we can conclude that the following statement holds true.
The minimax-robust spectral characteristic is determined as The minimax-robust spectral characteristic h 0 and the pair (f 0 , g 0 ) of the least favorable spectral densities form a saddle point of the function ∆(h; f, g) on the set H D × D. The saddle point inequalities where (f 0 , g 0 ) is a solution to the constrain optimization problem This problem is equivalent to the following optimization problem: A solution of this problem is determined by the condition 0 ∈ ∂∆ D (f 0 , g 0 ), which is the necessary and sufficient condition that the pair (f 0 , g 0 ) belongs to the set of minimums of the convex functional ∆ D (f, g) [27], [33], [34].
Here ∂∆ D (f 0 , g 0 ) is a subdifferential of the functional ∆ D (f, g) at the point (f, g) = (f 0 , g 0 ), which is a set of all linear bounded functionals Λ on L 1 × L 1 satisfying the inequality The form of the functional ∆ D (f, g) allows us to find derivatives and differentials in the space L 1 × L 1 .Hence, the complexity of the optimization problem ( 26) is determined by the complexity of finding a subdifferential of the indicator function δ(f, g|D f × D g ) of the set D f × D g .

Least favorable densities in the class
The introduced classes of spectral densities represent limitation on the variance of the sequences ξ(m) and η(m).
Assume that the spectral densities f 0 ∈ D f , g 0 ∈ D g and functions are bounded.Then the functional ∆(h(f 0 , g 0 ); f, g) is continuous and bounded in the space L 1 × L 1 .To solve the constrain optimization problem (26) we use the method of Lagrange multipliers.As a result we obtain the following relations determining the least favorable spectral densities A where constants α 1 ≥ 0 and α 2 ≥ 0. Particularly, The described relations allow us to formulate the following theorems.

Theorem 4
Suppose that the spectral densities f 0 (λ) ∈ D f and g 0 (λ) ∈ D g satisfy minimality condition (1) and the functions h f (f 0 , g 0 ) and h g (f 0 , g 0 ) determined by formulas ( 27) and ( 28) are bounded.The functions f 0 (λ) and g 0 (λ) determined by equations (29), (30) are the least favorable in the class D = D f × D g for the optimal linear filtering of the functional Aξ if they determine a solution to optimization problem (25).The function h(f 0 , g 0 ) determined by formula (6) is the minimax-robust spectral characteristic of the optimal estimate of the functional Aξ.

Theorem 5
Suppose that the spectral density f (λ) is known, the spectral density g 0 (λ) ∈ D g and they satisfy minimality condition (1).Suppose also that the function h g (f, g 0 ) determined by formula (28) is bounded.The spectral density g 0 (λ) is the least favorable in the class D g for the optimal linear filtering of the functional Aξ if it is of the form } and the pair (f, g 0 ) is a solution of the optimization problem (25).The function h(f, g 0 ) determined by formula (6) is the minimax-robust spectral characteristic of the optimal estimate of the functional Aξ.

Theorem 7
Suppose that f 0 (λ) ∈ D u v , g 0 (λ) ∈ D ε and condition (1) holds true.Let the functions h f (f 0 , g 0 ) and h g (f 0 , g 0 ) determined by formulas ( 27) and ( 28) are bounded.The functions f 0 (λ) and g 0 (λ) determined by equations (35) and (36) are the least favorable in the class D = D u v × D ε for the optimal linear filtering of the functional Aξ if they determine a solution to optimization problem (25).The function h(f 0 , g 0 ) determined by formula (6) is the minimax-robust spectral characteristic of the optimal estimate of the functional Aξ.

Theorem 8
Let the spectral density f (λ) be known, the spectral density g 0 ∈ D ε and they satisfy minimality condition (1).Assume that the function h g (f, g 0 ) determined by formula ( 28) is bounded.The spectral density g 0 (λ) is the least favorable in the class D ε for the optimal linear filtering of the functional Aξ if it is of the form and the pair (f, g 0 ) determines a solution to optimization problem (25).The function h(f, g 0 ) determined by formula (6) is the minimax-robust spectral characteristic of the optimal estimate of the functional Aξ.

Conclusions
In this paper, we presented methods of the mean-square optimal linear filtering of the linear functionals Aξ = ∑ ∞ k=0 a(k)ξ(−k) and A N ξ = ∑ N k=0 a(k)ξ(−k) which depend on the unknown values of a stationary stochastic sequence ξ(k) with the spectral density f (λ) from observations of the sequence ξ(m) + η(m) at time m = 0, −1, −2, . .., where η(k) is an uncorrelated with the sequence ξ(k) stationary stochastic sequence with the spectral density g(λ).Formulas for calculating the spectral characteristics and the values of the mean-square errors of the functionals are proposed in the case of spectral certainty, where the spectral densities f (λ) and g(λ) of the stationary stochastic sequences ξ(m) and η(m) are exactly known.In the case of spectral uncertainty, where the spectral densities are not known while a set D = D f × D g of admissible spectral densities is given, the minimaxrobust method of filtering is applied.For some given classes of admissible spectral densities we found relations that determine the least favorable spectral densities and the minimax-robuct spectral characteristics of the estimate of the functionals.