Robust Liu-Type Estimator for SUR Model

The Liu-type estimator is one of the shrink estimators that is used to remedy for a problem of multicollinearity in SUR model, but it is sensitive to the outlier. In this paper, we introduce the S Liu-type (SLiu-type) and MM Liu-type estimator (MMLiu-type) for SUR model. These estimators merge Liu-type estimator with S-estimator and with MMestimator which makes it have high robustness at the different level of efficiency and at the same time prevents the bad effects of multicollinearity. Moreover, to get more robust features, we have modified the Liu-type estimator by making it depend on MM estimator instead of GLS estimator. The asymptotical properties for the suggested estimator were discussed and we used the fast and robust bootstrap (FRB) to obtain the suggested robust estimators. Furthermore, we run the simulation study to show the extent of excellence for the suggested robust estimators relative to the other estimators by many factors.


Introduction
The SUR model was introduced by Zellner in [27] which is a special case of the linear regression model. The SUR model stacks the several regression equations under the assumption that they have error terms are correlated across them. The SUR model can be denoted by Where Y = (y T 1 , y T 2 , · · · , y T m ) T is nm × 1 vector of dependent variables in m equations with n value of observation, X = bdiag(X 1 , X 2 , · · · , X m ) is a block diagonal explanatory matrix, β = (β T 1 , β T 2 , · · · , β T m ) T is pm × 1 vector of coefficients in m equation and µ = (µ T 1 , µ T 2 , · · · , µ T m ) is nm × 1 vector of error term in m equation with mean E(µ) = 0 nm×1 vector and variance E(µµ T ) = Σ ⊗ I n where Σ is m × m matrix contains Cov(µ i µ j ) = σ ij I n , i, j = 1, 2, · · · , m elements. We can form the SUR model in (1) as the multivariate liner regression model Y =βẌ +μ.
(2) WhereŸ = (y 1 , y 2 , . . . , y m ), y i = A T i Y,β = dig(β 1 , β 2 , . . . , β m ),Ẍ = (X 1 , X 1 , . . . , X m ), X i = A T i X and µ = (μ 1 ,μ 2 , ...,μ m ) = (e 1 , e 2 , e n ) T with e i = (μ i1 ,μ i2 , ...μ im ). The mean E(μ) = 0 nm×1 vector and the variance E(μμ T ) = I n ⊗ Σ. The Aitken's estimator(GLS) for the SUR model in (1) was form aŝ 608 ROBUST LIU-TYPE ESTIMATOR FOR SUR MODEL The (GLS) estimator is a suitable method for estimating the SUR Model since it is the best unbiased estimator, but it works poorly when the data have multicollinearity. So, [19] suggested that ridge estimator prevent this problem of multicollinearity in SUR model, Where λ > 0 is ridge parameter. This estimator adds more ingredients to the hat matrix to eliminate the ill condition. In the same way, [1] modified the ridge estimator for the SUR model in canonical form. The ridge estimator depends on ridge parameter, so, [19,6,9] suggested the different methods to choose the ridge parameter. [5] suggested robust cross validation (CV) to choose the ridge parameter. Since the ridge parameter has instability, the Liu-type estimator, that was suggested by [12], is a suitable alternative for the ridge estimator. The Liu-Type estimator is the result of merging the ridge estimator and stein estimator and it has two parameters that work in parallel to effect the multicollinearity. The first parameter limits the excessive increase in the other parameter, and therefore it limits the problem of instability associated with the ridge estimator. [8] introduced a Liu-Type estimator for two SUR model. The Liu-Type estimator for SUR model is calculated based on the following equation: The (Liu-Type) estimator for SUR model has another formula aŝ Finally, [21] , introduced a stochastic restricted Liu-type estimator for SUR model. In many cases, the presence of multicollinearity may coincide with the presence of outliers. Many authors used a robust estimator to neutralize outliers. The M, S, MM, Least Trimmed Squares (LTS) and τ are the robust estimators that were suggested by [7,15,16,24,25], respectively to overcome the contamination in the data resulting from outliers. [10] introduced M-estimator for SUR model by the following equation: Where, the value of ϑ is tuning constant which is chosen as E ϕ ρ(μ) such that F ∼ N (0, 1) and ρ is Turkey's biweight function as the [13] ρ Where c > 0 is tuning parameter such as: ρ : ) and ρ is continuous, differentiable and symmetric. The Turkey's biweight loss function is suitable choice for ρ against outliers. [4] introduced Sestimator for SUR model that is calculated based on the following equation: The S-estimator in (9) has another form as Where λ s is the Lagrange multipliers. [18,20] introduced another kind of robust estimator for SUR model that depends on the least absolute deviations.

609
The MM estimator has a greater breakdown point where it uses the S-estimator as an initial point and it is asymptotically normal where it satisfies the first condition of the M estimator. Also, the MM estimator is computable when p > n and at the same time, it is robust when p/n is large [25] . Despite that S-estimator has a high breakdown point that reaches to 50% and it has low efficiency. So, [11] used anther robust estimator for SUR model that depends on the MM-function. Given that S-scalar (σ s ) it can be gotten by solving the following equation: Then, The MM-estimator is based on the following equation: Where ρ 0 ≥ ρ 1 , sup ρ 0 = sup ρ 1 and c 0 < c 1 . When we use the mm estimator, the breakdown point can be controlled independently from the efficiency by using c 0 ,c 1 in ρ 0 , ρ 1 notwithstanding the efficiency. These properties do not exist at S estimator since the degree of breakdown affects the degree of efficiency, for example when the maximum value for breakdown point is (50%), the efficiency reaches about (29%) [4]. Some studies have made robust modifications for ridge estimator like [14], where he showed that the robust ridge estimator is superior to partial least squares and principal components regression estimator. [5] suggested robust ridge estimator for SUR model that merges ridge estimator with Sestimator that is calculated based on the following equation: (Tarek) Suggested Ridge MM estimator for SUR model. Given that S-scalar (σ s ) as in (11), then the estimator got by solving the following equation: In this paper, we introduce MM Liu-Type estimator for SUR model that merges Liu-Type estimator with MMestimator. This estimator has a high breakdown point and is able to reduce the bad effect of multicollinearity at the same time.

S Liu-Type and MM Liu-Type estimator for SUR model
In SUR model in (1), if we merge the Liu-Type estimator in (5) with the S-estimator in (9), we get the (SLiu-Type) estimator for SUR model by solving the following equation: We can rewrite (15) aŝ λs . If we merge the Liu-type estimator in (5) with the MM estimator in (12), we get the (MMLiu-type) estimator for SUR model. Given S-scalarσ s as in (11), the (MMLiu-type) estimator for SUR got by solving the following equation: We can rewrite (18) aŝ Where Similarly, If we useΣ s instead ofΣ GLS in (16) then Where The unweighting versions of F M M and F S are formed as In this case, the (22) and (23) are modified as: In fact, the (SLiu-Type) estimator for SUR model has the special cases as the following: The (MM Liu-Type) estimator for SUR model has the special cases as the following: For the special cases of the (SLiu-Type) estimator and (MMLiu-Type) estimator for SUR model, we note that, the

Choosing the tuning parameters (λ, d)
In order to arrive at the best estimate of robust Liu-Type regression, appropriate tuning parameters must be selected. The robust tuning parameters are a good choice in this case. So, [2,3] chose tuning parameters for the Liu-Type estimator that have robust features. In this section, we develop the [23] formula by using the robust function. The suggested single value of tuning parameter (d) iŝ ), i = 1, 2, · · · , m, j = 1, 2, · · · , p.
On the other hand, there are many methods to choose the tuning parameter (Λ) such as AIC, BIC and Cross validation (C.V). Some authors used K-fold cross validation to choose the tuning parameter(Λ). [26] used the Kfold cross validation to choose the penalty parameter for a high dimension. We use the K-fold cross validation with Turkey's biweight to choose the tuning parameter (Λ) Whereβ (−k),λ is penalized estimator, we chose it for Liu-Type estimator.

Theoretical properties
In this section, we discuss the properties for the S Liu-type and MM Liu-type estimator for SUR model. [4] discuss the properties of the S-estimator for SUR model. We will make this discussion a main point for studying Proof: Frist, the SLiu-Type estimator fulfill the following condition Following [4] , the equation (16),(17) they can be form as: Where w 2 i = e TΣ−1 e. If we assume that, e is a symmetric distribution and independent from Z i andβ M M are asymptotically independent [4] then Cov(Ψ T 1 , Ψ T 2 ) = 0. For the central limit theorem, we get to . Lemma2: Assuming n 1/2 d → d 0 and n 1/2 λ d → λ 0 , then the estimator in (20) satisfy the following:

Fast and robust bootstrap
When we estimate the robust SUR Model by using the classic bootstrap, the results suffer from a lack of speed and durability and it produces breakdown of the estimator [11]. To compute a robust estimator, we need robust bootstrap procedures. This requirement is not necessary for non-robust estimators. Fast and robust bootstrap (FRB) which were introduced by [17] is a suitable option to calculate the robust estimators. This method is asymptotically consistent. In this paper, we apply (FRB) to (SLiu-Type) and (MMLiu-Type) estimator. At first, we rewrite (16), (17), (20) and (21) as is a vector that join the (SLiu-Type) and (MMLiu-Type) estimator and let TheΘ is a solution of a fixed-point equationsΘ = g(Θ) where g is function depend on the set data (y ij , x T ij ) T , i = 1, 2, · · · , m, j = 1, 2, · · · , n . The resampled corresponding estimator is Θ i * that is solved byΘ i * = g i * (Θ i * ). These approximations bootstrap underestimate the true variability since the starting value is the same for all bootstrap. The fast and robust bootstrap (FRB) is given bŷ Where ▽g(Θ)) is a matrix of partial derivatives which calculated by [11]. TheΘ i * F RB has more robust and easier to compute thanΘ i * .
We chose p= 4 and p=10 and we generate the explanatory variables from multivariate normal distribution and ρ µ is a correlation coefficient between cross error between equations. We choose ρ µ = 0.10, 0.85. We choose γ = 20% and 50% as contamination rate and we chose the number of observation n=20 and n=200. The initial value for coefficients is chosen as β p=4 = [1, 1, 1, 1] T andβ p=10 = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1] T and we run the simulation 500 replications. We use the equation (26) to choose the tuning parameter (d) and we use the equation (27) with 5-fold cross validation to choose the tuning parameter (λ). Table (1) summarizing the factors of the simulation study. We use MSE that is formed as 20%,50%

The results of simulation:
The tables of the result of the simulation are designed to include all factors, and each table has been divided so that it contains two main parts, the first part is the correlation coefficient between the independent variables and the second part the pollution percentage, and these tables were designed in this way to deal with the evaluators of multicollinearity and outliers. The result of the simulation study are summarized in table (2) This results clearly show that The MMLiu-Type and WMMliu-Type estimators work well when the other estimators at all factors and the Liu-Type estimator is the worst estimator at all factors. Also, the result of the simulation illustrated that,for all estimator, when n and p increase, the MSE decreasing and when m, ρ x , ρ µ and γ increase, the MSE increase. It is clear that the Liu-Type estimator, which is non robust estimator, is affected negatively as a result of increased contamination rate. On the other hand, the MM estimator works better than S estimator in all cases. The weighting versions of SLiu-Type and MMLiu-Type have a positive effect on the estimators. In the case (γ = 50%, ρ x = 0.90), the work of WMMliu-Type estimator is weaker than other cases in table (2), (3) and (5) but at the same time it works better than the other estimators.

Conclusion:
In this study, we introduce some robust Liu-Type estimators for SUR model which we got by merging the Sestimator or MM estimator and Liu-Type estimator to have the features that resist the multicollinearity and outliers at the same time. For more robust features, we replaced the GLS estimator by MM estimator in the Liu-Type estimator. This modification had good effects on the suggested estimators. Also, we discussed the asymptotical properties for this estimator. To verify superioriority of the proposed estimators, we ran the simulation study which it illustrated the superiority for the MMLiu-Type and WMMliu-Type estimators over the other estimators for resisting the multicollinearity and outliers.   Factors  m=3  n=20  n=200  p=4 p=10 p=4 p=10 ρ µ = 0.10 ρ µ = 0.85 ρ µ = 0.10 ρ µ = 0.85 ρ µ = 0.10 ρ µ = 0.85 ρ µ = 0. 10