On the mixed Kibria–Lukman estimator for the linear regression model

This paper considers a linear regression model with stochastic restrictions,we propose a new mixed Kibria–Lukman estimator by combining the mixed estimator and the Kibria–Lukman estimator.This new estimator is a general estimation, including OLS estimator, mixed estimator and Kibria–Lukman estimator as special cases. In addition, we discuss the advantages of the new estimator based on MSEM criterion, and illustrate the theoretical results through examples and simulation analysis.

It is easy to know from formula (2), Eβ = β , and the OLS estimator has been widely used because of its unbiased nature and concise form. However, the ill condition of the design matrix X caused by the increasing number of dependent predictors often makes the OLS estimates unstable.
Massy 1 proposed principal component estimator. Hoerl and Kennard 2 obtained the ridge estimation by introducing a ridge parameter k into the design X ′ X matrix calculation. Swindel 3 proposed a modified ridge estimator with prior information while Lukman et al. 4 proposed the two-parameter form of the ridge estimator called the modified ridge estimator (MRT). Liu 5 obtained a linearized form of the ridge estimator called the Liu estimator. Akdeniz and Kaciranlara 6 proposed the generalized Liu estimator. Liu 7 obtained a two-parameter form of the Liu estimator.
Many scholars have found that a new estimator can be obtained by combining the two estimators, which generally have good statistical properties. Baye and Parker 8 proposed r-k estimator by combining ridge estimator and principal component estimator. Kaciranlar and Sakallioglu 9 proposed r-d estimator by combining Liu estimator and principal component estimator. Ozkale and Kaciranlar 10 proposed two parameter estimator by combining the James-Stein Shrinkage estimator and the modified ridge estimator proposed by Swindel. Batah et al. 11 proposed a modified r-k estimator combining unbiased ridge estimator and principal component estimator. Yang and Chang 12 proposed another two parameter estimator based on ridge estimator and Liu estimator. Lukman et al. 13 proposed a new estimator by combining modified ridge estimator (MRT) and principal component estimator. Kibria and Lukman 14 proposed Kibria-Lukman estimator by combining ridge estimator and Liu estimator.
In practice, in addition to the sample information given by model (1), additional information about parameters in the sample information, such as certain deterministic or stochastic restrictions on unknown parameters, can also be considered. This method can also overcome the complex collinearity problem. Theil and Goldberger 15 and Theil 16 proposed mixed estimator by comprehensively considering sample information and constraints. Schiffrin and Toutenburg 17 proposed weighted mixed estimator for the different importance of sample information and prior information.
In recent years, biased estimation and estimation methods with prior information are often combined to form a broader biased estimation. Hubert and Wijekoon 18 proposed a stochastic restricted Liu estimator by combining Liu estimator and mixed estimator. Yang and Xu 19 obtained another stochastic mixed Liu estimator. In the same year, Yang and Chang further studied the stochastic mixed Liu estimator and obtained the weighted mixed Liu

The proposed estimator
Hoerl and Kennard 2 proposed the ridge estimator (RE): where k > 0 is the parameter. In fact, ridge estimator is obtained by solving the following extreme value problem: where c is constant, k is the Lagrange constant.
Kibria and Lukman 14 proposed the Kibria Lukman (KL) estimator: where k > 0 is the parameter.KL estimator is obtained by solving the following extreme value problem: where c is constant, k is the Lagrange constant. Consider the following stochastic restrictions: where r is the known random vector of j × 1 , R is the row full rank sample data matrix of j × p , let e be the j × 1 random error vector and independent of each other, and ψ be the known positive definite matrix. Theil and Goldberger 15 and Theil 16 proposed the mixed estimator by integrating sample information and constraints. The derivation idea is to rewrite models (1) and (6) into a new linear model:

above model is transformed into
By applying the least square estimator to the new linear model (7), the mixed estimator (ME)of parameter β is obtained: Combined mixed estimator and ridge estimator and proposed stochastic mixed ridge estimation (RME): The estimator proposed in this paper is obtained by solving the following extreme value problem: where c is constant, k is Lagrange constant. Regular equations can be obtained: from Eqs. (11) and (12), we can get the mixed KL estimator: It can be seen from Eq. (13) that mixed estimator, KL estimator and OLS estimator can be regarded as special cases of mixed KL estimator.Namely When k = 0,β ME =β MKL = X ′ X + R ′ ψ −1 R −1 X ′ y + R ′ ψ −1 r is mixed estimator;

The performance of the new estimator
If β is the estimation of β , then the mean square error matrix (MSEM) of β is given as: where Cov(β) is the covariance matrix of β , and Bias(β) = E(β) − β is the deviation vector. Two estimates β 1 and β 2 , β 2 are better than β 1 under MSEM criterion if and only if: The mean square error matrix of mixed KL estimator β MKL is calculated as follows: Deviation vector: By substituting k = 0 into Eq. (16), the mean square error matrix of the mixed estimator can be obtained: By substituting R = 0 into Eq. (16), the mean square error matrix of the KL estimator can be obtained: By substituting k = 0, R = 0 into Eq. (16), the mean square error matrix of the OLS estimator can be obtained: Mean square error matrix of mixed ridge estimator: Therefore, Comparison between mixed KL estimator and mixed estimator. From Eqs. (16) and (17), we make Because

Theorem 3.2
The necessary and sufficient conditions for mixed KL estimator β MKL to be superior to mixed estimator β ME under MSEM criterion are as follows:  (16) and (18), we make As long as k < min < 1 , following conclusions can be obtained: Comparison between mixed KL estimator and OLS estimator. From Eqs. (16) and (19), we make B e c a u s e C = C ′ , a n d > 0 , w e c a n g e t C > 0 , s o 2kI

Theorem 3.4 The necessary and sufficient conditions for mixed KL estimator β MKL to be superior to β OLS under MSEM criterion are as follows:
Comparison between mixed KL estimator and mixed ridge estimator. From Eqs. (16) and (22), we make .

Numerical example and simulation study
In order to further explain the theoretical results, this section will verify and analyze the above theoretical results through examples.
The example analysis data adopts the percentage data of research and development expenses in GNP of several countries from 1972 to 1986 used by Gruber 21 , Akdeniz and Erol 22 , in which x 1 represents France, x 2 represents West Germany, x 3 represents Japan, x 4 represents the former Soviet Union and y represents the United States. See Table 1 for specific data.
The data in Table 1 are processed as follows  14 to choose the biasing parameter k, and we can also use the generalized cross validation (GCV) criterion and the cross validation (CV) to choose the biasing parameter, the reference can refer to Arashi et al. 23 , Roozbeh 24 , and Roozbeh et al. 25 . In this paper we use the method propose by Kibria and Lukman 14 to choose the biasing parameter k, which is given as follows: we take k =k min .
Consider the following stochastic restrictions, this can refer to Roozbeh et al. 26 and Roozbeh and Hamzah 27 : For the mixed estimator, KL estimator, OLS estimator, mixed ridge estimator and mixed KL estimator proposed in this paper. The MSE is presented in Table 2.
As can be seen from Table 2: When k takes k min = 0.018 , the MSE value of mixed KL estimator β MKL is better than that of mixed estimator, KL estimator,OLS estimator and mixed ridge estimator. Consistent with the theoretical results of this paper, it can be concluded that adding stochastic restrictions may have better estimation effect under certain conditions. So in practice we can use the stochastic restrictions to address the multicollinearity.
Next, we consider Monte Carlo simulation analysis. Firstly, the generation of relevant parameters and data in the process of simulation analysis is briefly described.    where z ij is the random number generated by the standard normal random variable, ρ is the given constant, and ρ 2 theoretically represents the correlation between two different variables, so ρ 2 reflects the degree of complex collinearity of the model to some extent. In this simulation analysis, we consider three cases ρ = 0.85, 0.9, 0.99 , set p = 3, r = 1, R = 1 −2 −2 , e ∼ (0, σ 2 ), n = 30, 50, 70, 100.
For a given design matrix X, we take the orthogonalized eigenvector corresponding to the maximum eigenvalue of X ′ X as the real value of parameter vector β.
The data corresponding to the response variable is generated by the following equation: where ε i is the mean of zero, and random vector with variance of σ 2 = 0.1, 1, 5, 10.

Conclusions
In this paper, a new mixed KL estimator considering the prior information about parameters in sample information in linear model is proposed, and the properties of the new estimator are discussed. The necessary and sufficient conditions for KL estimator to be better than mixed estimator, KL estimator, OLS estimator and mixed ridge estimator under the criterion of mean square error matrix are given, and the proofs are given respectively. Then the theoretical results are verified by examples and simulation analysis.