Journal of the Operations Research Society of China >

2024 , Vol. 12 >Issue 3: 573 - 599

DOI: https://doi.org/10.1007/s40305-022-00449-x

Trace Lasso Regularization for Adaptive Sparse Canonical Correlation Analysis via Manifold Optimization Approach

Expand
  • 1 Beijing International Center for Mathematical Research, Peking University, Beijing 100871, China;
    2 School of Mathematics and Computational Science, Xiangtan University, Xiangtan 411105, China

Received date: 2021-06-04

  Revised date: 2022-09-24

  Online published: 2024-08-15

Supported by

This research was supported by the National Science Foundation of China (No. 12071398), the Natural Science Foundation of Hunan Province (No. 2020JJ4567) and the Key Scientific Research Found of Hunan Education Department (Nos. 20A097 and 18A351).

Fold

Abstract

Canonical correlation analysis (CCA) describes the relationship between two sets of variables by finding a linear combination that maximizes the correlation coefficient. However, in high-dimensional settings where the number of variables exceeds sample size, or in the case that the variables are highly correlated, the traditional CCA is no longer appropriate. In this paper, a new matrix regularization is introduced, which is an extension of the trace Lasso in the vector case. Then we propose an adaptive sparse version of CCA (ASCCA) to overcome these disadvantages by utilizing the trace Lasso regularization. The adaptability of ASCCA is that the sparsity regularization of canonical vectors depends on the sample data, which is more realistic in practical applications. The ASCCA model is further reformulated to an optimization problem on the Riemannian manifold. Then we adopt a manifold inexact augmented Lagrangian method to solve the resulting optimization problem. The performance of the ASCCA model is compared with some existing sparse CCA techniques in different simulation settings and real datasets.

Key words： Canonical correlation analysis; Sparsity of canonical vectors; Trace Lasso regularization; Manifold optimization

Cite this article

Kang-Kang Deng, Zheng Peng . Trace Lasso Regularization for Adaptive Sparse Canonical Correlation Analysis via Manifold Optimization Approach[J]. Journal of the Operations Research Society of China, 2024 , 12(3) : 573 -599 . DOI: 10.1007/s40305-022-00449-x

References

[1] Hotelling, H.: Relations between two sets of variates. Biometrika 28(3/4), 321-377(1936)
[2] Witten, D.M., Tibshirani, R.J.: Extensions of sparse canonical correlation analysis with applications to genomic data. Stat. Appl. Genet. Mol. Biol. 8(1), 1-27(2009)
[3] Parkhomenko, E.: Sparse canonical correlation analysis. Stat. Appl. Genet. Mol. Biol. 8(1), 1(2008)
[4] Lin, D., Calhoun, V.D., Wang, Y.: Correspondence between fMRI and SNP data by group sparse canonical correlation analysis. Med. Image Anal. 18(6), 891-902(2014)
[5] Correa, N.M., Li, Y.O., Adali, T., Calhoun, V.D.: Canonical correlation analysis for feature-based fusion of biomedical imaging modalities and its application to detection of associative networks in schizophrenia. IEEE J. Select. Top. Signal Process. 2(6), 998-1007(2008)
[6] Fu, Y., Huang, T.S.: Image classification using correlation tensor analysis. IEEE Trans. Image Process. 17(2), 226-234(2008)
[7] Loog, M., Van Ginneken, B., Duin, R.P.: Dimensionality reduction by canonical contextual correlation projections. In: European Conference on Computer Vision, pp. 562-573. Springer, Berlin (2004)
[8] Vinod, H.D.: Canonical ridge and econometrics of joint production. J. Econom. 4(2), 147-166(1976)
[9] Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 42(1), 80-86(2000)
[10] Parkhomenko, E., Tritchler, D., Beyene, J.: Sparse canonical correlation analysis with application to genomic data integration. Stat. Appl. Genet. Mol. Biol. 8(1), 1-34(2009)
[11] Witten, D.M., Tibshirani, R., Hastie, T.: A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. Biostatistics 10(3), 515-534(2009)
[12] Waaijenborg, S., de Witt Hamer, P.C.V., Zwinderman, A.H.: Quantifying the association between gene expressions and DNA-markers by penalized canonical correlation analysis. Stat. Appl. Genet. Mol. Biol. 7(1) (2008). PMID: 18241193. https://doi.org/10.2202/1544-6115.1329
[13] Lin, D., Zhang, J., Li, J., Calhoun, V.D., Deng, H., Wang, Y.: Group sparse canonical correlation analysis for genomic data integration. BMC Bioinf. 14(1), 245(2013)
[14] Chen, X., Liu, H., Carbonell, J.G.: Structured sparse canonical correlation analysis. Appl. Intell. 40(2), 291-304(2012)
[15] Wilms, I., Croux, C.: Sparse canonical correlation analysis from a predictive point of view. Biom. J. 57(5), 834-851(2015)
[16] Suo, X., Minden, V., Nelson, B., Tibshirani, R., Saunders, M.: Sparse canonical correlation analysis. arXiv:1705.10865(2017)
[17] Gao, C., Ma, Z., Zhou, H.H.: Sparse CCA: adaptive estimation and computational barriers. Ann. Stat. 45(5), 2074-2101(2017)
[18] Mai, Q., Zhang, X.: An iterative penalized least squares approach to sparse canonical correlation analysis. Biometrics 75(3), 734-744(2019)
[19] Grave, E., Obozinski, G., Bach, F.: Trace Lasso: a trace norm regularization for correlated designs. Adv. Neural Inf. Process. Syst. pp. 2187-2195(2012). https://doi.org/10.48550/arXiv.1109.1990
[20] Wang, Y., Lin, X.,Wu, L., Zhang,W., Zhang, Q., Huang, X.: Robust subspace clustering formulti-view data by exploiting correlation consensus. IEEE Trans. Image Process. 24(11), 3939-3949(2015)
[21] Wang, J., Lu, C., Wang, M., Li, P., Yan, S., Hu, X.: Robust face recognition via adaptive sparse representation. IEEE Trans. Cybern. 44(12), 2368-2378(2014)
[22] Lu, C., Feng, J., Lin, Z., Yan, S.: Correlation adaptive subspace segmentation by trace Lasso. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1345-1352(2013)
[23] Hu, J., Liu, X., Wen, Z., Yuan, Y.: A brief introduction to manifold optimization. J. Oper. Res. Soc. China 8(2), 199-248(2020)
[24] Deng, K., Peng, Z.: A manifold inexact augmented Lagrangian method for nonsmooth optimization on Riemannian submanifolds in Euclidean space. IMA J. Numer. Anal. (2022). https://doi.org/10.1093/imanum/drac018
[25] Zhang, J., Ma, S., Zhang, S.: Primal-dual optimization algorithms over Riemannian manifolds: an iteration complexity analysis. Math. Program. 184(1), 445-490(2020)
[26] Yang, W., Zhang, L., Song, R.: Optimality conditions for the nonlinear programming problems on Riemannian manifolds. Pac. J. Optim. 10(2), 415-434(2014)
[27] Iannazzo, B., Porcelli, M.: The Riemannian Barzilai-Borwein method with nonmonotone line search and the matrix geometric mean computation. IMA J. Numer. Anal. 38(1), 495-517(2015)
[28] Chin, K., DeVries, S., Fridlyand, J., Spellman, P.T., Roydasgupta, R., Kuo, W., Lapuk, A., Neve, R.M., Qian, Z., Ryder, T., et al.: Genomic and transcriptional aberrations linked to breast cancer pathophysiologies. Cancer Cell 10(6), 529-541(2006)
[29] Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2009)
[30] Beck, A.: First-order Methods in Optimization vol. 25. SIAM (2017)
[31] Boumal, N., Absil, P.A., Cartis, C.: Global rates of convergence for nonconvex optimization on manifolds. IMA J. Numer. Anal. 39(1), 1-33(2018)
Options
Outlines

/

〈 〉

Copyright © Operations Research Society of China, The Periodicals Agency of Shanghai University , and Springer-Verlag Berlin Heidelberg 2016