Zero-Sum Continuous-Time Markov Games with One-Side Stopping

doi:10.1007/s40305-023-00502-3

Journal of the Operations Research Society of China ›› 2024, Vol. 12 ›› Issue (1): 169-187.doi: 10.1007/s40305-023-00502-3

Previous Articles Next Articles

Zero-Sum Continuous-Time Markov Games with One-Side Stopping

Yurii Averboukh^1,2

1. Laboratory of Stochastic Analysis and Its Applications, National Research University Higher School of Economics (HSE), Moscow, 109028, Russia;
2. Department of Differential Equations, Krasovskii Institute of Mathematics and Mechanics, Yekaterinburg, 620108, Russia

Received:2023-02-22 Revised:2023-05-30 Online:2024-03-30 Published:2024-03-13
Contact: Yurii Averboukh E-mail:averboukh@gmail.com
Supported by:
The article was prepared within the framework of the HSE University Basic Research Program in 2023.

Abstract

Abstract: The paper is concerned with a variant of the continuous-time finite state Markov game of control and stopping where both players can affect transition rates, while only one player can choose a stopping time. The dynamic programming principle reduces this problem to a system of ODEs with unilateral constraints. This system plays the role of the Bellman equation. We show that its solution provides the optimal strategies of the players. Additionally, the existence and uniqueness theorem for the deduced system of ODEs with unilateral constraints is derived.

Key words: Continuous-time Markov games, Dynamic programming, Verification theorem, Stopping time

CLC Number:

Yurii Averboukh. Zero-Sum Continuous-Time Markov Games with One-Side Stopping[J]. Journal of the Operations Research Society of China, 2024, 12(1): 169-187.

TrendMD

References

[1] Guo, X.P., Hernández-Lerma, O.: Continuous-Time Markov Decision Processes: Theory and Applications. Springer, New York (2009)
[2] Guo, X., Hernández-Lerma, O., Prieto-Rumeau, T.: A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder). TOP 14(2), 177-261 (2006). https://doi.org/10.1007/BF02837562
[3] Mordecki, E.: Optimal stopping and perpetual options for Lévy processes. Finance Stoch. 6, 473-493 (2002)
[4] Shiryaev, A.N.: Optimal Stopping Rules. Springer, New York (2008)
[5] Horiguchi, M.: Markov decision processes with a stopping time constraint. Math. Methods Oper. Res. 53(2), 279-295 (2001). https://doi.org/10.1007/PL00003996
[6] Zachrisson, L.: Markov games. Ann. Math. Stud. 52, 211-253 (1964)
[7] Dynkin, E.B.: Game variant of a problem on optimal stopping. Soviet Math. Dokl. 10, 270-274 (1969)
[8] Frid, E.B.: The optimal stopping rule for a two-person Markov chain with opposing interests. Theory Probab. Appl. 14(4), 713-716 (1969). https://doi.org/10.1137/1114091
[9] Ghosh, M.K., Rao, K.S.M.: Zero-sum stochastic games with stopping and control. Oper. Res. Lett. 35(6), 799-804 (2007). https://doi.org/10.1016/j.orl.2007.02.002
[10] Pal, C., Saha, S.: Continuous-time zero-sum stochastic game with stopping and control. Oper. Res. Lett. 48(6), 715-719 (2020). https://doi.org/10.1016/j.orl.2020.08.012
[11] Averboukh, Y.V.: Approximation of value function of differential game with minimal cost. Vestn. Udmurt. Univ. Mat. Mekhanika Komp’yuternye Nauki 31(4), 536-561 (2021). https://doi.org/10.35634/vm210402
[12] Averboukh, Y.: Approximate solutions of continuous-time stochastic games. SIAM J. Control. Optim. 54, 2629-2649 (2016). https://doi.org/10.1137/16M1062247
[13] Barron, E.N.: Differential games with maximum cost. Nonlinear Anal. Theory Methods Appl. 14(11), 971-989 (1990)
[14] Subbotin, A.I.: Generalized Solutions of First-Order PDEs. The Dynamical Perspective. Birkhauser, Boston (1995)
[15] Krasovskii, N.N., Subbotin, A.I.: Game-Theoretical Control Problems. Springer, New York (1988)
[16] Kallenberg, O.: Foundations of Modern Probability. Springer, New York (2001)

Zero-Sum Continuous-Time Markov Games with One-Side Stopping

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 7

Recommended Articles

Metrics

Comments

[1]	Yan-Ying Zhao, Pei-Wen Yu, Jian-Qiang Hu. Strategic Admission Behavior and Its Implications: Evidence from a Cardiac Surgery Department [J]. Journal of the Operations Research Society of China, 2023, 11(1): 29-50.
[2]	Xian-Ping Wu, Seakweng Vong, Wen-Xin Zhou. Optimal Stopping Time of a Portfolio Selection Problem with Multi-assets [J]. Journal of the Operations Research Society of China, 2021, 9(1): 163-179.
[3]	Peng-Fei Wan, Xin-Zhuang Chen. Computing the Number of k-Component Spanning Forests of a Graph with Bounded Treewidth [J]. Journal of the Operations Research Society of China, 2019, 7(2): 385-394.
[4]	Juan Zou, Jin-Jiang Yuan. Equivalence of Some Different Maintenance Activities in Single-Machine Scheduling [J]. Journal of the Operations Research Society of China, 2018, 6(4): 545-556.
[5]	Wei-Ping Wu, Jian-Jun Gao. Explicit Solution for Constrained Optimal Execution Problem with General Correlated Market Depth [J]. Journal of the Operations Research Society of China, 2018, 6(1): 159-174.
[6]	Meng-Fei He · Duan Li · Yuan-Yuan Chen. Stochastic Control for Optimal Execution: Fast Approximation Solution Scheme Under Nested Mean-semi Deviation and Conditional Value at Risk [J]. Journal of the Operations Research Society of China, 2017, 5(2): 161-.
[7]	Cu-Xia Miao · Yun-Jie Xia · Yu-Zhong Zhang · Juan Zou. Batch Scheduling with Deteriorating Jobs to Minimize the Total Completion Time [J]. Journal of the Operations Research Society of China, 2013, 1(3): 377-384.