Table of Content

30 June 2020, Volume 8 Issue 2

Previous Issue    Next Issue

Special Issue: Mathematical Optimization: Past, Present and Future

Preface–Special Issue on Mathematical Optimization: Past, Present and Future

Zai-Wen Wen, Ya-Xiang Yuan

2020, 8(2):  197-198.  doi:10.1007/s40305-020-00310-z

Asbtract ( 618 )   PDF  

Related Articles | Metrics

A Brief Introduction to Manifold Optimization

Jiang Hu, Xin Liu, Zai-Wen Wen, Ya-Xiang Yuan

2020, 8(2):  199-248.  doi:10.1007/s40305-020-00295-9

Asbtract ( 891 )   PDF  

References | Related Articles | Metrics

Manifold optimization is ubiquitous in computational and applied mathematics, statistics,engineering,machinelearning,physics,chemistry,etc.Oneofthemainchallenges usually is the non-convexity of the manifold constraints. By utilizing the geometry of manifold, a large class of constrained optimization problems can be viewed as unconstrained optimization problems on manifold. From this perspective, intrinsic structures, optimality conditions and numerical algorithms for manifold optimization are investigated. Some recent progress on the theoretical results of manifold optimization is also presented.

Optimization for Deep Learning: An Overview

Ruo-Yu Sun

2020, 8(2):  249-294.  doi:10.1007/s40305-020-00309-6

Asbtract ( 796 )   PDF  

References | Related Articles | Metrics

Optimization is a critical component in deep learning. We think optimization for neural networks is an interesting topic for theoretical research due to various reasons. First, its tractability despite non-convexity is an intriguing question and may greatly expand our understanding of tractable problems. Second, classical optimization theory is far from enough to explain many phenomena. Therefore, we would like to understand the challenges and opportunities from a theoretical perspective and review the existing research in this field. First, we discuss the issue of gradient explosion/vanishing and the more general issue of undesirable spectrum and then discuss practical solutions including careful initialization, normalization methods and skip connections. Second, we review generic optimization methods used in training neural networks, such as stochastic gradient descent and adaptive gradient methods, and existing theoretical results. Third, we review existing research on the global issues of neural network training, including results on global landscape, mode connectivity, lottery ticket hypothesis and neural tangent kernel.

Review of Mathematical Methodology for Electric Power Optimization Problems

Dong Han, Xiao-Jiao Tong

2020, 8(2):  295-309.  doi:10.1007/s40305-020-00304-x

Asbtract ( 564 )   PDF  

References | Related Articles | Metrics

Electric power system is a physical energy system consisting of power generation, substations, transmission, distribution, and consumption. The objective of power system optimization is to improve power system security, economy, and reliability. This paper summarizes the classical mathematical optimization methods and modeling techniques of power system optimization associated with system planning, operation, and control. Along with the development of electric power industry, the concept of Energy Internet is addressed, which consists of power network, gas network, and transportation network. Under such new environments, electric power optimization faces some challenging with respect to the cooperation of multi-energy networks. According to the design structure and operational characteristics of the Energy Internet, some research areas of electric power optimization are presented from the view of mathematical optimization modeling and calculation. The aim is to provide some optimization methodology to solve the optimal issues of power system under the background of Energy Internet.

A Review on Deep Learning in Medical Image Reconstruction

Hai-Miao Zhang, Bin Dong

2020, 8(2):  311-340.  doi:10.1007/s40305-019-00287-4

Asbtract ( 795 )   PDF  

References | Related Articles | Metrics

Medical imaging is crucial in modern clinics to provide guidance to the diagnosis and treatment of diseases. Medical image reconstruction is one of the most fundamental and important components of medical imaging, whose major objective is to acquire high-quality medical images for clinical usage at the minimal cost and risk to the patients. Mathematical models in medical image reconstruction or, more generally, image restoration in computer vision have been playing a prominent role. Earlier mathematical models are mostly designed by human knowledge or hypothesis on the image to be reconstructed, and we shall call these models handcrafted models. Later, handcrafted plus data-driven modeling started to emerge which still mostly relies on human designs, while part of the model is learned from the observed data. More recently, as more data and computation resources are made available, deep learning based models (or deep models) pushed the data-driven modeling to the extreme where the models are mostly based on learning with minimal human designs. Both handcrafted and data-driven modeling have their own advantages and disadvantages. Typical handcrafted models are well interpretable with solid theoretical supports on the robustness, recoverability, complexity, etc., whereas they may not be flexible and sophisticated enough to fully leverage large data sets. Data-driven models, especially deep models, on the other hand, are generally much more flexible and effective in extracting useful information from large data sets, while they are currently still in lack of theoretical foundations. Therefore, one of the major research trends in medical imaging is to combine handcrafted modeling with deep modeling so that we can enjoy benefits from both approaches. The major part of this article is to provide a conceptual review of some recent works on deep modeling from the unrolling dynamics viewpoint. This viewpoint stimulates new designs of neural network architectures with inspirations from optimization algorithms and numerical differential equations. Given the popularity of deep modeling, there are still vast remaining challenges in the field, as well as opportunities which we shall discuss at the end of this article.

How Can Machine Learning and Optimization Help Each Other Better?

Zhou-Chen Lin

2020, 8(2):  341-351.  doi:10.1007/s40305-019-00285-6

Asbtract ( 580 )   PDF  

References | Related Articles | Metrics

Optimization is an indispensable part of machine learning as machine learning needs to solve mathematical models efficiently. On the other hand, machine learning can also provide new momenta and new ideas for optimization. This paper aims at investigating how to make the interactions between optimization and machine learning more effective.