Journal of the Operations Research Society of China >

2025 , Vol. 13 >Issue 4: 1108 - 1156

DOI: https://doi.org/10.1007/s40305-023-00522-z

Computation over t-Product Based Tensor Stiefel Manifold: A Preliminary Study

Expand
  • 1 School of Physical Science and Technology, Guangxi University, Nanning 530004, Guangxi, China;
    2 College of Mathematics and Information Science, Guangxi University, Nanning 530004, Guangxi, China;
    3 Center for Applied Mathematics of Guangxi, Guangxi University, Nanning 530004, Guangxi, China

Received date: 2022-12-14

  Revised date: 2023-09-18

  Online published: 2025-12-19

Supported by

This work was supported by the National Natural Science Foundation of China (No. 12171105), Fok Ying Tong Education Foundation (No. 171094), and the special foundation for Guangxi Bagui Scholars.

Fold

Abstract

Let $*$ denote the t -product between two third-order tensors proposed by Kilmer and Martin (Linear Algebra Appl 435(3): 641-658, 2011). The purpose of this work is to study fundamental computation over the set $\operatorname{St}(n, p, l):=\left\{\mathcal{X} \in \mathbb{R}^{n \times p \times l} \mid \mathcal{X}^{\top} * \mathcal{X}=\right. \mathcal{I}\}$, where $\mathcal{X}$ is a third-order tensor of size $n \times p \times l(n \geqslant p)$ and $\mathcal{I}$ is the identity tensor. It is first verified that St ( $n, p, l$) endowed with the Euclidean metric forms a Riemannian manifold, which is termed as the (third-order) tensor Stiefel manifold in this work. We then derive the tangent space, Riemannian gradient, and Riemannian Hessian on St ($n, p, l$). In addition, formulas of various retractions based on t-QR, t-polar decomposition, t-Cayley transform, and t-exponential, as well as vector transports, are presented. It is expected that analogous to their matrix counterparts, the derived formulas may serve as building blocks for analyzing optimization problems over the tensor Stiefel manifold and designing Riemannian algorithms.

Key words： Tensor; t-Product; Stiefel manifold; Retraction; Vector transport; Manifold optimization

Cite this article

Xian-Peng Mao, Ying Wang, Yu-Ning Yang . Computation over t-Product Based Tensor Stiefel Manifold: A Preliminary Study[J]. Journal of the Operations Research Society of China, 2025 , 13(4) : 1108 -1156 . DOI: 10.1007/s40305-023-00522-z

References

[1] Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009)
[2] Comon, P.: Tensors: a brief introduction. IEEE Signal Process. Mag. 31(3), 44–53 (2014)
[3] Cichocki, A., Mandic, D., De Lathauwer, L., Zhou, G., Zhao, Q., Caiafa, C., Phan, H.A.: Tensor decompositions for signal processing applications: from two-way to multiway component analysis. IEEE Signal Process. Mag. 32(2), 145–163 (2015)
[4] Sidiropoulos, N., De Lathauwer, L., Fu, X., Huang, K., Papalexakis, E., Faloutsos, C.: Tensor decomposition for signal processing and machine learning. IEEE Trans. Signal Process. 65(13), 3551–3582 (2017)
[5] Braman, K.: Third-order tensors as linear operators on a space of matrices. Linear Algebra Appl. 433(7), 1241–1253 (2010)
[6] Kilmer, M.E., Martin, C.D.: Factorization strategies for third-order tensors. Linear Algebra Appl. 435(3), 641–658 (2011)
[7] Kilmer, M.E., Braman, K., Hao, N., Hoover, R.C.: Third-order tensors as operators on matrices: a theoretical and computational framework with applications in imaging. SIAM J. Matrix Anal. Appl. 34(1), 148–172 (2013)
[8] Lu, C., Feng, J., Chen, Y., Liu, W., Lin, Z., Yan, S.: Tensor robust principal component analysis with a new tensor nuclear norm. IEEE Trans. Pattern Anal. Mach. Intell. 42(4), 925–938 (2019)
[9] Miao, Y., Qi, L., Wei, Y.: T-Jordan canonical form and t-Drazin inverse based on the t-product. Commun. Appl. Math. Comput. Sci. 3(2), 201–220 (2021)
[10] Lund, K.: The tensor t-function: a definition for functions of third-order tensors. Numer. Linear Algebra Appl. 27(3), e2288 (2020)
[11] Miao, Y., Qi, L., Wei, Y.: Generalized tensor function via the tensor singular value decomposition based on the T-product. Linear Algebra Appl. 590, 258–303 (2020)
[12] Liu, W.H., Jin, X.Q.: A study on T-eigenvalues of third-order tensors. Linear Algebra Appl. 612, 357–374 (2020)
[13] Zheng, M.M., Huang, Z.H., Wang, Y.: T-positive semidefiniteness of third-order symmetric tensors and T-semidefinite programming. Comput. Optim. Appl. 78(1), 239–272 (2021)
[14] Qi, L., Luo, Z.: Tubal matrices (2021). arXiv:2105.00793
[15] Huang, W., Absil, P.A., Gallivan, K.A.: A Riemannian BFGS method without differentiated retraction for nonconvex optimization problems. SIAM J. Optim. 28(1), 470–495 (2018)
[16] Hu, J., Jiang, B., Lin, L., Wen, Z., Yuan, Y.X.: Structured quasi-Newton methods for optimization with orthogonality constraints. SIAM J. Sci. Comput. 41(4), A2239–A2269 (2019)
[17] Chen, S., Ma, S., So, A.M.C., Zhang, T.: Proximal gradient method for nonsmooth optimization over the Stiefel manifold. SIAM J. Optim. 30(1), 210–239 (2020)
[18] Huang, W., Wei, K.: Riemannian proximal gradient methods. Math. Program. 194, 371–413 (2022)
[19] Gao, B., Liu, X., Chen, X., Yuan, Y.X.: A new first-order algorithmic framework for optimization problems with orthogonality constraints. SIAM J. Optim. 28(1), 302–332 (2018)
[20] Hu, J., Liu, X., Wen, Z.W., Yuan, Y.X.: A brief introduction to manifold optimization. J. Oper. Res. Soc. China 8(2), 199–248 (2020)
[21] Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2009)
[22] Tu, L.W.: An Introduction to Manifolds, 2nd edn. Springer, Universitext, New York (2011)
[23] Boumal, N.: An Introduction to Optimization on Smooth Manifolds. Cambridge University Press, Cambridge (2022)
[24] Uschmajew, A., Vandereycken, B.: The geometry of algorithms using hierarchical tensors. Linear Algebra Appl. 439(1), 133–166 (2013)
[25] Holtz, S., Rohwedder, T., Schneider, R.: On manifolds of tensors of fixed TT-rank. Numer. Math. 120(4), 701–731 (2012)
[26] Kressner, D., Steinlechner, M., Vandereycken, B.: Low-rank tensor completion by Riemannian optimization. BIT Numer. Math. 54(2), 447–468 (2014)
[27] Heidel, G., Schulz, V.: A Riemannian trust-region method for low-rank tensor completion. Numer. Linear Algebra Appl. 25(6), e2175 (2018)
[28] Steinlechner, M.: Riemannian optimization for high-dimensional tensor completion. SIAM J. Sci. Comput. 38(5), S461–S484 (2016)
[29] Breiding, P., Vannieuwenhoven, N.: A Riemannian trust region method for the canonical tensor rank approximation problem. SIAM J. Optim. 28(3), 2435–2465 (2018)
[30] Gilman, K., Tarzanagh, D.A., Balzano, L.: Grassmannian optimization for online tensor completion and tracking with the t-SVD. IEEE Trans. Signal Process. 70, 2152–2167 (2022)
[31] Song, G.J., Wang, X.Z., Ng, M.K.: Riemannian conjugate gradient descent method for fixed multi rank third-order tensor completion. J. Comput. Appl. Math. 421, 114866 (2023)
[32] Zhang, X., Yang, Z.P., Cao, C.G.: Inequalities involving Khatri–Rao products of positive semidefinite matrices. Appl. Math. E-Notes 2, 117–124 (2002)
[33] Huang, W.: Optimization algorithms on Riemannian manifolds with applications. Ph.D. thesis, The Florida State University (2013)
[34] Zhu, X.: A Riemannian conjugate gradient method for optimization on the Stiefel manifold. Comput. Optim. Appl. 67(1), 73–110 (2017)
[35] Edelman, A., Arias, T., Smith, S.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1998)
[36] Bunse-Gerstner, A., Byers, R., Mehrmann, V.: Numerical methods for simultaneous diagonalization. SIAM J. Matrix Anal. Appl. 14(4), 927–949 (1993)
[37] Pesquet-Popescu, B., Pesquet, J.C., Petropulu, A.P.: Joint singular value decomposition-a new tool for separable representation of images. In: International Conference on Image Processing. vol. 2, pp. 569–572. IEEE, Thessaloniki, Greece (2001)
[38] Shashua, A., Levin, A.: Linear image coding for regression and classification using the tensor-rank principle. In: International Conference on Artificial Intelligence and Statistics. vol. 1, pp. I–42–I–49. IEEE Computer Society, Kauai, HI, USA (2001)
[39] Allen, G.I.: Sparse higher-order principal components analysis. In: International Conference on Artificial Intelligence and Statistics. vol. 22, pp. 27–36. PMLR, La Palma, Canary Islands (2012)
[40] Wang, Y., Dong, M., Xu, Y.: A sparse rank-1 approximation algorithm for high-order tensors. Appl. Math. Lett. 102, 106140 (2020)
[41] Mao, X., Yang, Y.: Several approximation algorithms for sparse best rank-1 approximation to higherorder tensors. J. Glob. Optim. (2022). https://doi.org/10.1007/s10898-022-01140-4
[42] Kwak, N.: Principal component analysis based on 1-norm maximization. IEEE Trans. Pattern Anal. Mach. Intell. 30(9), 1672–1680 (2008)
[43] Hao, N., Kilmer, M.E., Braman, K., Hoover, R.C.: Facial recognition using tensor–tensor decompositions. SIAM J. Imaging Sci. 6(1), 437–463 (2013)
[44] Schönemann, P.H.:Ageneralizedsolutionoftheorthogonalprocrustesproblem.Psychometrika 31(1), 1–10 (1966)
[45] Lin, J., Huang, T.Z., Zhao, X.L., Jiang, T.X., Zhuang, L.: A tensor subspace representation-based method for hyperspectral image denoising. IEEE Tran. Geosci. Remote Sens. 59(9), 7739–7757 (2020)
[46] Xu, S.S., Huang, T.Z., Lin, J., Chen, Y.: T-hy-demosaicing: hyperspectral reconstruction via tensor subspace representation under orthogonal transformation. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 14, 4842–4853 (2021)
[47] Xu, T., Huang, T.Z., Deng, L.J., Yokoya, N.: An iterative regularization method based on tensor subspace representation for hyperspectral image super-resolution. IEEE Trans. Geosci. Remote Sens. 60, 1–16 (2022)
[48] Hoover, R.C., Caudle, K., Braman, K.: Multilinear discriminant analysis through tensor-tensor eigendecomposition. In: ICMLA. pp. 578–584. IEEE, Orlando, FL (2018)
[49] Ozdemir, C., Hoover, R.C., Caudle, K., Braman, K.: High-order multilinear discriminant analysis via order-n tensor eigendecomposition. Technical report, SSRN (2022). https://dx.doi.org/10.2139/ssrn. 4203431
[50] Vervliet, N., Debals, O., Sorber, L., Van Barel, M., De Lathauwer, L.: Tensorlab 3.0 (2016). http://www.tensorlab.net
[51] Lu, C.: Tensor-Tensor Product Toolbox. Carnegie Mellon University, Pittsburgh (2018)
[52] Iannazzo, B., Porcelli, M.: The Riemannian Barzilai–Borwein method with nonmonotone line search and the matrix geometric mean computation. IMA J. Numer. Anal. 38(1), 495–517 (2018)
[53] Kilmer, M.E., Horesh, L., Avron, H., Newman, E.: Tensor–tensor algebra for optimal representation and compression of multiway data. Proc. Natl. Acad. Sci. U.S.A. 118(28), e2015851118 (2021)
[54] Kernfeld, E., Kilmer, M., Aeron, S.: Tensor–tensor products with invertible linear transforms. Linear Algebra Appl. 485, 545–570 (2015)
[55] Hall, B.C.: Lie Groups, Lie Algebras, and representations. Springer, Cham (2015)
[56] Van Loan, C.: Computing integrals involving the matrix exponential. IEEE Trans. Autom. Control 23(3), 395–404 (1978)
[57] Van Loan, C.F.: The ubiquitous kronecker product. J. Comput. Appl. Math. 123(1–2), 85–100 (2000)
[58] Kolda, T.G.: Multilinear operators for higher-order decompositions. Tech. Rep. SAND2006-2081, 923081, Citeseer (2006)
Options
Outlines

/

〈 〉

Copyright © Operations Research Society of China, The Periodicals Agency of Shanghai University , and Springer-Verlag Berlin Heidelberg 2016