Estimation Procedure for Reduced Rank Regression, PLSSVD

  • Willín Álvarez Universidad de Carabobo
  • Victor John Griffin Universidad de Carabobo
Keywords: Reduced Rank Multivariate Regression PLSSVD, Partial Least Squares, Singular Value Decomposition, Multicollinearity, reduced rank

Abstract

This paper presents a procedure for coefficient estimation in a multivariate regression model of reduced rank in the presence of multicollinearity.  The procedure permits the prediction of the dependent variables taking advantage of both Partial Least Squares (PLS) and Singular Value Decomposition (SVD) methods, which is denoted by PLSSVD. Global variability indices and prediction error sums are used to compare the results obtained by classical regression with reduced rank (OLSSVD) and the PLSSVD procedure when applied to examples with different grades of multicollinearity (severe, moderate and low).  In addition, simulations to compare the methods were performed with different sample sizes under four scenarios. The new PLSSVD method is shown to be more effective when the multicollinearity is severe and especially for small sample sizes.

Author Biographies

Willín Álvarez, Universidad de Carabobo
Associate ProfesorMath DepartmentFaculty of Science and Technology
Victor John Griffin, Universidad de Carabobo
Agregate ProfesorMathematics DepartmentFaculty of Science and Technology

References

Álvarez W. and Cárdenas O., 2010. Un enfoque basado en la descomposición en valores singulares generalizada para el biplot de coeficientes de regresión. Ingenier Industrial.Actualidad y Nuevas Tendencias. A III. vol. II, N4. ISSN: 1856-8327, p. 65-74.

Anderson T., (2002). Specification and Misspecification in Reduced Rank Regression. The Indian Journal of Statistics, 64,193-205.

Anderson T., (1999). Asymptotic Distibution of the Reduced Rank Regression estimator under general conditions. The Annals of Statistcs, 27, 1141-1154.

Anderson T., (1951). Estimating Linear Restrictions on Regression Coefficients for Multivariate Normal Distributions. Ann. Math. Statist, 22, 327-351.

Davies, P.and Tso M., (1982). Procedures for reduced rank regression. Applied Statistic 31, 244-255.

Eckart, C.and Young, G, (1936). The Approximation de One Matrix for Another of Lower Rank. Psychometrika 1, 211-218.

Gabriel, K., (1971). The Biplot-graphic display of matrices with applications to principal component analysis. Biometrika, 58, 453-467.

Graybill, F.A. (1969). Introducction to Matrices with Statistical Applications. Belmont, California. Wadsworth.

Heinen A. and Rengifo E., (2004). Multivariate Reduced Rank Regression in non-Gaussian Contexts, Using Copulas. Center for Operations Research and Econometrics Catholic University of Louvain Belgium. Roma.

Hervé Abdi, (2003). Partial Least Squares (PLS) Regression. Eds. Encyclopedia of Social Sciencies Research Methods.

Höskuldsson, A., (1988). PLS regression methods. Journal of Chemometrics, 2, 211- 228.

Izenman, A. J., (1975). Reduced rank regression for the multivariate linear model. Journal of Multivaiate Analysis 5, 248-264.

Johnson, R.and Wichern, (1998). Applied Multivariate Statistical Analysis. Prentice Hall, Upper Saddle River, New Jersey.

Jollife I.T. (1986) Principal Component Analysis. Springer - Verlag.

Mardia K.V., Kent J.T. and Bibby J.M. (1979). Multivariate Analysis, London Academic Press.

Martens H. and NæT. (1989). Multivariante Calibration. John Wiley and Sons.

Montgomery D. and Peck E. (1992). Introduction to Linear Regression Analysis. Second Edition. John Wiley and Sons.

Pena, D., (2002). Análisis de Datos Multivariantes. Mc Graw Hill.

Rencher A. (2002). Methods of Multivariate Analysis. Wiley-Interscience Publication.

Tenenhaus, M. (1998). La régression pls, théorie et pratique.Éditions Technip. Paris.

Ter Braak, C. and Looman, C. (1994). Biplots in reduced rank regression. Biometrical Journal 8, 983-1003.

TerBraak, C. (1990).Interpreting canonical correlationanalysis through biplotsof structuralcorrelations and weights.Psychometrika, 55, 519-531.

Tso, M. K-S (1981). Reduced-rank regression and canonical analysis. J.R. Statist. Soc. B, 43, 183-189.

Valencia J., D´ıaz-Llanos F. and Sainz-Calleja (2004). Métodos de Predicción en Situaciones L´ımite. Editorial La Muralla, S.A. Madrid Espa.

Van Der Wollenberg, A.L. (1977). Redundancy Analysis. An Alternative for Canonical Correlation Analysis. Psychometrika, 42:207-219.

Vásquez, M. (1995). Aportaciones al Análisis Biplot: Un enfoque algebraico. Thesis Doctoral. Universidad de Salamanca.

Velu Raja P., Reinsel Gregory C. and Wichern Dean W. (1986). Reduced rank models for multiple time series. Biometrika Vol. 73.1, pp.105-18.

Wold H. (1966). Estimation of principal components and related models by iterative least squares. In Multivariate analysis (ed. P.R. Krishnaiah), Academic Press, New York.

Wold S., Martens H. and Wold H. (1983), in Lecture Notes in Mathematics: Matrix Pencils, ed. by A. Ruhe and B. Kagström, pp. 286-293. Springer, Heidelberg

Qing-Song Xu, Yi-Zeng Liang and Hai- Lin Shen (2001). Generalized PLS regression. Journal of Chemometric, Vol. 15, 135-148.

Published
2016-06-01
How to Cite
Álvarez, W., & Griffin, V. J. (2016). Estimation Procedure for Reduced Rank Regression, PLSSVD. Statistics, Optimization & Information Computing, 4(2), 107-117. https://doi.org/10.19139/soic.v4i2.146
Section
Research Articles