The paper is concerned with a variant of the continuous-time finite state Markov game of control and stopping where both players can affect transition rates, while only one player can choose a stopping time. The dynamic programming principle reduces this problem to a system of ODEs with unilateral constraints. This system plays the role of the Bellman equation. We show that its solution provides the optimal strategies of the players. Additionally, the existence and uniqueness theorem for the deduced system of ODEs with unilateral constraints is derived.
Yurii Averboukh
. Zero-Sum Continuous-Time Markov Games with One-Side Stopping[J]. Journal of the Operations Research Society of China, 2024
, 12(1)
: 169
-187
.
DOI: 10.1007/s40305-023-00502-3
[1] Guo, X.P., Hernández-Lerma, O.: Continuous-Time Markov Decision Processes: Theory and Applications. Springer, New York (2009)
[2] Guo, X., Hernández-Lerma, O., Prieto-Rumeau, T.: A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder). TOP 14(2), 177-261 (2006). https://doi.org/10.1007/BF02837562
[3] Mordecki, E.: Optimal stopping and perpetual options for Lévy processes. Finance Stoch. 6, 473-493 (2002)
[4] Shiryaev, A.N.: Optimal Stopping Rules. Springer, New York (2008)
[5] Horiguchi, M.: Markov decision processes with a stopping time constraint. Math. Methods Oper. Res. 53(2), 279-295 (2001). https://doi.org/10.1007/PL00003996
[6] Zachrisson, L.: Markov games. Ann. Math. Stud. 52, 211-253 (1964)
[7] Dynkin, E.B.: Game variant of a problem on optimal stopping. Soviet Math. Dokl. 10, 270-274 (1969)
[8] Frid, E.B.: The optimal stopping rule for a two-person Markov chain with opposing interests. Theory Probab. Appl. 14(4), 713-716 (1969). https://doi.org/10.1137/1114091
[9] Ghosh, M.K., Rao, K.S.M.: Zero-sum stochastic games with stopping and control. Oper. Res. Lett. 35(6), 799-804 (2007). https://doi.org/10.1016/j.orl.2007.02.002
[10] Pal, C., Saha, S.: Continuous-time zero-sum stochastic game with stopping and control. Oper. Res. Lett. 48(6), 715-719 (2020). https://doi.org/10.1016/j.orl.2020.08.012
[11] Averboukh, Y.V.: Approximation of value function of differential game with minimal cost. Vestn. Udmurt. Univ. Mat. Mekhanika Komp’yuternye Nauki 31(4), 536-561 (2021). https://doi.org/10.35634/vm210402
[12] Averboukh, Y.: Approximate solutions of continuous-time stochastic games. SIAM J. Control. Optim. 54, 2629-2649 (2016). https://doi.org/10.1137/16M1062247
[13] Barron, E.N.: Differential games with maximum cost. Nonlinear Anal. Theory Methods Appl. 14(11), 971-989 (1990)
[14] Subbotin, A.I.: Generalized Solutions of First-Order PDEs. The Dynamical Perspective. Birkhauser, Boston (1995)
[15] Krasovskii, N.N., Subbotin, A.I.: Game-Theoretical Control Problems. Springer, New York (1988)
[16] Kallenberg, O.: Foundations of Modern Probability. Springer, New York (2001)
