Blackbox Simulation Optimization

doi:10.1007/s40305-024-00549-w

Abstract

Abstract: Simulation optimization is a widely used tool in the analysis and optimization of complex stochastic systems. The majority of the previous works on simulation optimization rely heavily on detailed system information, such as the distributions of input variables and the system dynamics. However, it is usually costly and even unrealistic to obtain such detailed information in practice. To overcome this difficulty, new methods are needed to solve stochastic optimization problems based on simulation outputs or real-world data and fairly limited knowledge of the underlying system, i.e., in a blackbox setting. In this paper, we provide an up-to-date overview of the subject of blackbox simulation optimization, which has only been attracting attention from the simulation community in recent years. We discuss some of the new stochastic approximation algorithms developed recently for blackbox stochastic optimization. We also discuss challenges and potential future research opportunities.

Key words: Simulation, Blackbox, Stochastic optimization, Stochastic approximation

CLC Number:

Hao Cao, Jian-Qiang Hu, Teng Lian. Blackbox Simulation Optimization[J]. Journal of the Operations Research Society of China, 2025, 13(3): 723-749.

TrendMD

References

[1] Adrian, T., Brunnermeier, M.K.: CoVarR. Federal Reserve Bank of New York Staff Report (2008). https://doi.org/10.2139/ssrn.1269446
[2] Ahn, D., Chen, N., Kim, K.-K.: Robust risk quantification via shock propagation in financial networks. Oper. Res. 72(1), 1-18(2023)
[3] Alexander, S., Coleman, T.F., Li, Y.: Minimizing CVaR and VaR for a portfolio of derivatives. J. Bank. Finance 30(2), 583-605(2006)
[4] Andradóttir, S.: A review of random search methods. In: Handbook of Simulation Optimization, pp. 277-292. Springer (2015)
[5] Astudillo, R., Frazier, P.I.: Thinking inside the box: A tutorial on grey-box Bayesian optimization. In: 2021 Winter Simulation Conference, pp. 1-15. IEEE (2021)
[6] Audet, C.: A Survey on Direct Search Methods for Blackbox Optimization and Their Applications. Springer, Berlin (2014)
[7] Audet, C., Hare, W.: Derivative-Free and Blackbox Optimization. Springer (2017). https://doi.org/10.1007/978-3-319-68913-5
[8] Audet, C., Dzahini, K.J., Kokkolaras, M., Le Digabel, S.: Stochastic mesh adaptive direct search for blackbox optimization using probabilistic estimates. Comput. Optim. Appl. 79(1), 1-34(2021)
[9] Bartle, R.G.: The Elements of Integration and Lebesgue Measure. Wiley, London (2014)
[10] Barton, R.R., Nelson, B.L., Xie, W.: Quantifying input uncertainty via simulation confidence intervals. INFORMS J. Comput. 26(1), 74-87(2014)
[11] Barton, R.R., Lam, H., Song, E.: Input uncertainty in stochastic simulation. In: The Palgrave Handbook of Operations Research, pp. 573-620. Springer (2022)
[12] Bernardi, M., Maruotti, A., Petrella, L.: Multivariate Markov-switching models and tail risk interdependence (2013). arXiv:1312.6407
[13] Bhatnagar, S.: Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization. ACM Trans. Model. Comput. Simul. 15(1), 74-107(2005)
[14] Bhatnagar, S., Borkar, V.S.: Multiscale stochastic approximation for parametric optimization of hidden Markov models. Probab. Eng. Inf. Sci. 11(4), 509-522(1997)
[15] Bhatnagar, S., Fu, M.C., Marcus, S.I., Bhatnagar, S.: Randomized difference two-timescale simultaneous perturbation stochastic approximation algorithms for simulation optimization of hidden Markov models (2000). https://apps.dtic.mil/sti/citations/ADA637176
[16] Bhatnagar, S., Sutton, R.S., Ghavamzadeh, M., Lee, M.: Natural actor-critic algorithms. Automatica 45, 2471-2482(2009)
[17] Bianchi, M.L., De Luca, G., Rivieccio, G.: Non-Gaussian models for CoVarR estimation. Int. J. Forecast. 39(1), 391-404(2023)
[18] Bohlin, T.P.: Practical Grey-box Process Identification: Theory and Applications. Springer, Berlin (2006)
[19] Borkar, V.S.: Stochastic approximation with two time scales. Syst. Control Lett. 29(5), 291-294(1997)
[20] Borkar, V.S.: Stochastic Approximation: A Dynamical Systems Viewpoint, vol. 48. Springer, Berlin (2009)
[21] Cao, H., Hu, J.-Q., Hu, J.: Black-box CoVarR and its gradient estimation (2023). https://ssrn.com/abstract=4583631
[22] Cao, H., Hu, J.-Q., Lian, T., Yang, X.: Infinitesimal perturbation analysis (ipa) derivative estimation with unknown parameters (2023). https://ssrn.com/abstract=4880736
[23] Cassandras, C., Gong, W., Lee, J.: Robustness of perturbation analysis estimators for queueing systems with unknown distributions. J. Optim. Theory Appl. 70(3), 491-519(1991)
[24] Cassandras, C., Gong, W.-B., Ho, Y., Hu, J., Vakili, P.: Analysis, control, and optimization of discrete event dynamic systems. In: Proceedings of 1995 American Control Conference, vol. 2, pp. 1065-1068. IEEE (1995)
[25] Chang, H.S., Fu, M.C., Hu, J., Marcus, S.I.: An adaptive sampling algorithm for solving Markov decision processes. Oper. Res. 53(1), 126-139(2005)
[26] Chang, H.S., Fu, M.C., Hu, J., Marcus, S.I.: A survey of some simulation-based algorithms for Markov decision processes. Commun. Inf. Syst. 7(1), 59-92(2007)
[27] Chang, H.S., Hu, J., Fu, M.C., Marcus, S.I.: Simulation-based Algorithms for Markov Decision Processes, vol. 68. Springer, Berlin (2013)
[28] Chang, K.-H.: Stochastic Nelder-Mead simplex method-a new globally convergent direct search method for simulation optimization. Eur. J. Oper. Res. 220(3), 684-694(2012)
[29] Chang, K.H., Hong, L.J., Wan, H.: Stochastic trust-region response-surface method (STRONG)-a new response-surface framework for simulation optimization. Inf. J. Comput. 25(2), 230-243(2013)
[30] Chau, M., Fu, M.C.: An overview of stochastic approximation. In: Handbook of simulation optimization, pp. 149-178(2015)
[31] Chen, C.-H., Chick, S.E., Lee, L.H., Pujowidianto, N.A.: Ranking and selection: efficient simulation budget allocation. Handbook of simulation optimization, pp. 45-80(2015)
[32] Chen, N., Glasserman, P.: Malliavin Greeks without Malliavin calculus. Stochastic Process. Appl. 117(11), 1689-1723(2007)
[33] Chen, R., Menickelly, M., Scheinberg, K.: Stochastic optimization using a trust-region method and random models. Math. Program. 169, 447-487(2018)
[34] Chen, R.T., Rubanova, Y., Bettencourt, J., Duvenaud, D.K.: Neural ordinary differential equations. Adv. Neural Inf. Process. Syst. 31, 25(2018)
[35] Chen, X., Ankenman, B.E., Nelson, B.L.: Enhancing stochastic kriging metamodels with gradient estimators. Oper. Res. 61(2), 512-528(2013)
[36] Cheng, H.-F., Liu, X., Zhang, K.: Constructing confidence intervals for nested simulation. Naval Res. Logist. 69(8), 1138-1149(2022)
[37] Cheng, Z., Luo, J., Wu, R.: On the finite-sample statistical validity of adaptive fully sequential procedures. Eur. J. Oper. Res. 307(1), 266-278(2023)
[38] Chia, Y.L., Glynn, P.W.: Limit theorems for simulation-based optimization via random search. ACM Trans. Model. Comput. Simul. 23(3), 1-18(2013)
[39] Chong, E.K.P., Ramadge, P.J.: Optimization of queues using an infinitesimal perturbation analysis-based stochastic algorithm with general update times. SIAM J. Control Optim. 31(3), 698-732(1993)
[40] Conn,A.R.,Scheinberg,K.,Vicente,L.N.:Introduction to derivative-free optimization.SIAM(2009)
[41] Corlu, C.G., Akcay, A., Xie, W.: Stochastic simulation under input uncertainty: a review. Oper. Res. Persp. 7, 100162(2020)
[42] Custódio, A.L., Scheinberg, K., Nunes Vicente, L.: Methodologies and software for derivative-free optimization. In: Advances and Trends in Optimization with Engineering Applications, pp. 495-506(2017)
[43] De Boer, P.-T., Kroese, D.P., Mannor, S., Rubinstein, R.Y.: A tutorial on the cross-entropy method. Ann. Oper. Res. 134, 19-67(2005)
[44] Dereventsov, A., Webster, C.G., Daws, J.: An adaptive stochastic gradient-free approach for high-dimensional blackbox optimization. In: Tiwari, R., Apoorva, M., Neha, Y., Pavone, M. (eds), Proceedings of International Conference on Computational Intelligence, pp. 333-348. Springer Singapore (2022)
[45] Ensor, K.B., Glynn, P.W.: Stochastic optimization via grid search. Lectures in Applied MathematicsAmerican Mathematical Society 33, 89-100(1997)
[46] Fan, Q., Hu, J.: Surrogate-based promising area search for Lipschitz continuous simulation optimization. INFORMS J. Comput. 30(4), 677-693(2018)
[47] Fan, W., Hong, L.J., Zhang, X.: Robust selection of the best. In: 2013 Winter Simulations Conference, pp. 868-876. IEEE (2013)
[48] Fan, W., Hong, L.J., Zhang, X.: Distributionally robust selection of the best. Manage. Sci. 66(1), 190-208(2020)
[49] Fu, H., Hong, L.J., Jiang, G.: Sensitivity analysis of CoVaR. In: 2023 IEEE 19th International Conference on Automation Science and Engineering, pp. 1-6. IEEE (2023)
[50] Fu, M., Hu, J.-Q.: Conditional Monte Carlo: Gradient Estimation and Optimization Applications. Springer, New York (1997). https://doi.org/10.1007/978-1-4615-6293-1
[51] Fu, M.C.: Convergence of a stochastic approximation algorithm for the GI/G/1 queue using infinitesimal perturbation analysis. J. Optim. Theory Appl. 65, 149-160(1990)
[52] Fu, M.C.: What you should know about simulation and derivatives. Naval Res. Logist. 55(8), 723- 736(2008)
[53] Fu, M.C.: Stochastic gradient estimation. Handbook of simulation optimization, pp. 105-148(2015)
[54] Fu, M.C.: Handbook of Simulation Optimization. Springer, Berlin (2015)
[55] Fu, M.C., Hu, J.-Q.: Extensions and generalizations of smoothed perturbation analysis in a generalized semi-Markov process framework. IEEE Trans. Autom. Control 37(10), 1483-1500(1992)
[56] Fu, M.C., Hu, J.-Q.: Smoothed perturbation analysis derivative estimation for Markov chains. Oper. Res. Lett. 15(5), 241-251(1994)
[57] Fu, M.C., Hu, J.-Q.: (s, S) inventory systems with random lead times: Harris recurrence and its implications in sensitivity analysis. Probab. Eng. Inf. Sci. 8(3), 355-376(1994)
[58] Fu, M.C., Hu, J.-Q.: Sensitivity analysis for Monte Carlo simulation of option pricing. Probab. Eng. Inf. Sci. 9(3), 417-446(1995)
[59] Fu, M.C., Hong, L.J., Hu, J.-Q.: Conditional Monte Carlo estimation of quantile sensitivities. Manage. Sci. 55(12), 2019-2027(2009)
[60] Ghadimi, S., Lan, G.: An overview of stochastic approximation. Handbook of simulation optimization, pp. 179-206(2015)
[61] Glasserman, P.: Glasserman, P.: Gradient Estimation Via Perturbation Analysis. Kluwer Academic Publishers, Boston (1991)
[62] Glasserman, P.: Monte Carlo Methods in Financial Engineering, vol. 53. Springer, Berlin (2004)
[63] Glasserman, P., Tayur, S.: Sensitivity analysis for base-stock levels in multiechelon production-inventory systems. Manage. Sci. 41(2), 263-281(1995)
[64] Glasserman, P., Hu, J.-Q., Strickland, S.G.: Strongly consistent steady-state derivative estimates. Probab. Eng. Inf. Sci. 5(4), 391-413(1991)
[65] Glasserman, P., Heidelberger, P., Shahabuddin, P.: Variance reduction techniques for estimating value-at-risk. Manage. Sci. 46(10), 1349-1364(2000)
[66] Glynn, P.W.: Likelilood ratio gradient estimation: an overview. In: Proceedings of the 19th conference on Winter simulation, pp. 366-375(1987)
[67] Gong, W.-B., Hu, J.-Q.: The maclaurin series for the gi/g/1 queue. J. Appl. Probab. 29(1), 176-184(1992)
[68] Goodwin, T., Xu, J., Chen, C.-H., Celik, N.: Efficient simulation optimization with simulation learning. In: 2021 IEEE 17th International Conference on Automation Science and Engineering, pp. 2268-2273. IEEE (2021)
[69] Gu, S., Lillicrap, T., Sutskever, I., Levine, S.: Continuous deep Q-learning with model-based acceleration. In: International conference on machine learning, pp. 2829-2838. PMLR (2016)
[70] Hare, W., Nutini, J., Tesfamariam, S.: A survey of non-gradient optimization methods in structural engineering. Adv. Eng. Softw. 59, 19-28(2013)
[71] He, S., Jiang, G., Lam, H., Fu, M.C.: Adaptive importance sampling for efficient stochastic root finding and quantile estimation. Oper. Res. 2, 56(2023)
[72] Heston, S.L.: A closed-form solution for options with stochastic volatility with applications to bond and currency options. Rev. Financial Stud. 6(2), 327-343(1993)
[73] Ho, Y.-C., Cao, X.-R.: Perturbation Analysis of Discrete Event Dynamic Systems, vol. 145. Springer, Berlin (1991)
[74] Ho, Y.-C., Hu, J.-Q.: An infinitesimal perturbation analysis algorithm for a multiclass G/G/1 queue. Oper. Res. Lett. 9(1), 35-44(1990)
[75] Hong, L.J.: Estimating quantile sensitivities. Oper. Res. 57(1), 118-130(2009)
[76] Hong, L.J., Liu, G.: Simulating sensitivities of conditional value at risk. Manage. Sci. 55(2), 281-293(2009)
[77] Hong, L.J., Liu, G.: Monte Carlo estimation of value-at-risk, conditional value-at-risk and their sensitivities. In: Proceedings of the 2011 Winter Simulation Conference, pp. 95-107. IEEE (2011)
[78] Hong, L.J., Nelson, B.L.: Discrete optimization via simulation using compass. Oper. Res. 54(1), 115-129(2006)
[79] Hong, L.J., Nelson, B.L.: A brief introduction to optimization via simulation. In: Proceedings of the 2009 Winter simulation conference, pp. 75-85. IEEE (2009)
[80] Hong, L.J., Zhang, X.: Surrogate-based simulation optimization. In: Tutorials in Operations Research: Emerging Optimization Methods and Modeling Techniques with Applications, pp. 287- 311. INFORMS (2021)
[81] Hong, L.J., Hu, Z., Liu, G.: Monte Carlo methods for value-at-risk and conditional value-at-risk: a review. ACM Trans. Model. Comput. Simul. 24(4), 52(2014)
[82] Hong, L.J., Juneja, S., Luo, J.: Estimating sensitivities of portfolio credit risk using Monte Carlo. INFORMS J. Comput. 26(4), 848-865(2014)
[83] Hong, L.J., Nelson, B.L., Xu, J.: Discrete optimization via simulation. Handbook of simulation optimization, pp. 9-44(2015)
[84] Hong, L.J., Luo, J., Zhong, Y.: Speeding up pairwise comparisons for large scale ranking and selection. In: 2016 Winter Simulation Conference, pp. 749-757. IEEE (2016)
[85] Hong, L.J., Fan, W., Luo, J.: Review on ranking and selection: a new perspective. Front. Eng. Manag. 8(3), 321-343(2021)
[86] Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw. 2(5), 359-366(1989)
[87] Hu, J., Hu, P.: On the performance of the cross-entropy method. In: Proceedings of the 2009 Winter Simulation Conference, pp. 459-468. IEEE (2009)
[88] Hu, J., Fu, M.C., Marcus, S.I.: A model reference adaptive search method for global optimization. Oper. Res. 55(3), 549-568(2007)
[89] Hu, J., Wang, Y., Zhou, E., Fu, M.C., Marcus, S.I.: A survey of some model-based methods for global optimization. Optimization, Control, and Applications of Stochastic Systems: In Honor of Onésimo Hernández-Lerma, pp. 157-179(2012)
[90] Hu, J., Peng, Y., Zhang, G., Zhang, Q.: A stochastic approximation method for simulation-based quantile optimization. INFORMS J. Comput. 34(6), 2889-2907(2022)
[91] Hu, J., Song, M., Fu, M.C.: Quantile optimization via multiple timescale local search for black-box functions. Oper. Res. Articles in Advance, 1-23(2024)
[92] Hu, J.-Q.: Convexity of sample path performance and strong consistency of infinitesimal perturbation analysis estimates. IEEE Trans. Autom. Control. 37(2), 258-262(1992)
[93] Hu, J.-Q.: The departure process of the GI/G/1 queue and its Maclaurin series. Oper. Res. 44(5), 810-815(1996)
[94] Hu, J.-Q., Lian, T.: On comparison of steady-state infinitesimal perturbation analysis and likelihood ratio derivative estimates. In: Discrete Event Dynamic Systems, pp. 1-10(2023)
[95] Hu, J.-Q., Strickland, S.G.: Strong consistency of sample path derivative estimates. Appl. Math. Lett. 3(4), 55-58(1990)
[96] Hu, J.-Q., Nananukul, S., Gong, W.-B.: A new approach to (s, S) inventory systems. J. Appl. Probab. 30(4), 898-912(1993)
[97] Hu, M., Xu, J., Hu, J.-Q., Chen, C.-H.: Optimal computation budget allocation with Gaussian process regression (2024). https://ssrn.com/abstract=4880333
[98] Hu, X.: Dynamic data driven simulation. SCS M&S Magazine 5, 16-22(2011)
[99] Huang, W., Lin, N., Hong, L.J.: Monte-Carlo estimation of CoVarR (2022). http://arxiv.org/pdf/2210.06148v1
[100] Jiang, G., Fu, M.C.: On estimating quantile sensitivities via infinitesimal perturbation analysis. Oper. Res. 63(2), 435-441(2015)
[101] Jiang, G., Fu, M.C.: Quantile sensitivity estimation for dependent sequences. J. Appl. Probab. 53(3), 715-732(2016)
[102] Jiang, G., Yun, X.: Importance sampling for CoVarR estimation. In: 2022 Winter Simulation Conference, pp. 879-890. IEEE (2022)
[103] Jiang, G., Fu, M.C., Xu, C.: Bias reduction in estimating quantile sensitivities. IFAC Proc. Vol. 47(3), 10463-10468(2014)
[104] Jiang, G., Fu, M.C., Xu, C.: Optimal importance sampling for simulation of Lévy processes. In: 2015 Winter Simulation Conference, pp. 3813-3824. IEEE (2015)
[105] Jiang, G., Hong, L.J., Nelson, B.L.: A simulation analytics approach to dynamic risk monitoring. In: 2016 Winter Simulation Conference, pp. 437-447. IEEE (2016)
[106] Jiang, G., Xu, C., Fu, M.C.: Jiang, G., Xu, C., Fu, M.C.: On sample average approximation algorithms for determining the optimal importance sampling parameters in pricing financial derivatives on Lévy processes. Oper. Res. Lett. 44(1), 44-49(2016)
[107] Jiang, G., Hong, L.J., Nelson, B.L.: Online risk monitoring using offline simulation. INFORMS J. Comput. 32(2), 356-375(2020)
[108] Jiang, J., Peng, Y., Hu, J.: Quantile-based policy optimization for reinforcement learning. In: 2022 Winter Simulation Conference (WSC), pp. 2712-2723. IEEE (2022)
[109] Jiang, J., Hu, J., Peng, Y.: Quantile-based deep reinforcement learning using two-timescale policy gradient algorithms (2023). arXiv:2305.07248
[110] Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237-285(1996)
[111] Karimalis, E.N., Nomikos, N.K.: Measuring systemic risk in the european banking sector: a copula CoVarR approach. Eur. J. Finance. 24(11), 944-975(2018)
[112] Kibzun, A., Matveev, E.: Optimization of the quantile criterion for the convex loss function by a stochastic quasigradient algorithm. Ann. Oper. Res. 200(1), 183-198(2012)
[113] Kiefer, J., Wolfowitz, J.: Stochastic estimation of the maximum of a regression function. Ann. Math. Stat. 23(3), 462-466(1952)
[114] Kim, S., Pasupathy, R., Henderson, S.G.: A guide to sample average approximation. In: Handbook of simulation optimization, pp. 207-243(2015)
[115] Kim, S.-H., Nelson, B.L.: On the asymptotic validity of fully sequential selection procedures for steady-state simulation. Oper. Res. 54(3), 475-488(2006)
[116] Konda, V.R., Tsitsiklis, J.N.: Actor-critic-type learning algorithms for Markov decision processes. SIAM J. Control. Optim. 38, 94-123(1999)
[117] Konda, V.R., Tsitsiklis, J.N.: On actor-critic algorithms. SIAM J. Control. Optim. 42(4), 1143-1166(2003)
[118] Kushner, H.J., Yin, G.G.: Stochastic Approximation and Recursive Algorithms and Applications. Springer, Berlin (2003)
[119] Lan, H., Nelson, B.L., Staum, J.: A confidence interval procedure for expected shortfall risk measurement via two-level simulation. Oper. Res. 58(5), 1481-1490(2010)
[120] Larson, J., Billups, S.C.: Stochastic derivative-free optimization using a trust region framework. Comput. Optim. Appl. 64, 619-645(2016)
[121] Law, A.M., Kelton, W.D., Kelton, W.D.: Simulation Modeling and Analysis, vol. 3. Mcgraw-hill, New York (2007)
[122] Lei, L., Peng, Y., Fu, M.C., Hu, J.-Q.: Applications of generalized likelihood ratio method to distribution sensitivities and steady-state simulation. Dis. Event Dyn. Syst. 28, 109-125(2018)
[123] Li, Z., Fan, W., Hong, L.J.: The (surprising) sample optimality of greedy procedures for large-scale ranking and selection. Manage. Sci. Articles in Advance, 1-22(2023)
[124] Liu, G., Hong, L.J.: Kernel estimation of quantile sensitivities. Naval Res. Log. 56(6), 511-525(2009)
[125] Liu, G., Hong, L.J.: Kernel estimation of the Greeks for options with discontinuous payoffs. Oper. Res. 59(1), 96-108(2011)
[126] Liu, G., Jeff Hong, L.: Kernel estimation for quantile sensitivities. In: 2007 Winter Simulation Conference, pp. 941-948(2007)
[127] Liu, M., Staum, J.: Stochastic kriging for efficient nested simulation of expected shortfall. J. Risk. 12(3), 3(2010)
[128] Liu, M., Nelson, B.L., Staum, J.: An efficient simulation procedure for point estimation of expected shortfall. In: Proceedings of the 2010 Winter Simulation Conference, pp. 2821-2831. IEEE (2010)
[129] Liu, T., Lin, Y., Zhou, E.: Bayesian stochastic gradient descent for stochastic optimization with streaming input data. SIAM J. Optim. 34(1), 389-418(2024)
[130] Luo, J., Hong, L.J.: Large-scale ranking and selection using cloud computing. In: Proceedings of the 2011 Winter Simulation Conference, pp. 4046-4056. IEEE (2011)
[131] Luo, J., Hong, L.J., Nelson, B.L., Wu, Y.: Fully sequential procedures for large-scale ranking-andselection problems in parallel computing environments. Oper. Res. 63(5), 1177-1194(2015)
[132] Ma, J., Zhu, S., Wu, Y.: Joint effects of the liability network and portfolio overlapping on systemic financial risk: contagion and rescue. Quant. Finance. 21(5), 753-770(2021)
[133] Ma, J.-L., Zhu, S.-S., Pang, X.-C.: How is systemic risk amplified by three typical financial networks. J. Oper. Res. Soc. China. 10(3), 579-598(2022)
[134] Mainik, G., Schaanning, E.: On dependence consistency of CoVaR and some other systemic risk measures. Stat. Risk Model. 31(1), 49-77(2014)
[135] Menborn, F., Augustin, F., Bungartz, H.-J., Marzouk, Y.M.: A trust-region method for derivative-free nonlinear constrained stochastic optimization (2017). arXiv: 1703.04156
[136] Muqattash, I., Hu, J.: An ε-greedy multiarmed bandit approach to markov decision processes. Stats. 6(1), 99-112(2023)
[137] Ni, E.C., Ciocan, D.F., Henderson, S.G., Hunter, S.R.: Efficient ranking and selection in parallel computing environments. Oper. Res. 65(3), 821-836(2017)
[138] Oh, D.H., Patton, A.J.: Time-varying systemic risk: evidence from a dynamic copula model of CDS spreads. J. Bus. Econ. Stat. 36(2), 181-195(2018)
[139] Pang, X., Zhu, S., Cui, X., Ma, J.: Systemic risk of optioned portfolio: controllability and optimization. J. Econ. Dyn. Control. 153, 104701(2023)
[140] Paquette, C., Scheinberg, K.: A stochastic line search method with expected complexity analysis. SIAM J. Optim. 30(1), 349-376(2020)
[141] Peng, Y., Fu, M.C., Hu, J.-Q.: Estimating distribution sensitivity using generalized likelihood ratio method. In: 201613th International Workshop on Discrete Event Systems, pp. 123-128. IEEE (2016)
[142] Peng, Y., Fu, M.C., Hu, J.-Q.: Gradient-based simulated maximum likelihood estimation for stochastic volatility models using characteristic functions. Quant. Finance 16(9), 1393-1411(2016)
[143] Peng, Y., Fu, M.C., Glynn, P.W., Hu, J.: On the asymptotic analysis of quantile sensitivity estimation by Monte Carlo simulation. In: 2017 Winter Simulation Conference, pp. 2336-2347. IEEE (2017)
[144] Peng, Y., Fu, M.C., Hu, J.-Q., Heidergott, B.: A new unbiased stochastic derivative estimator for discontinuous sample performances with structural parameters. Oper. Res. 66(2), 487-499(2018)
[145] Pflug, G.C.: Optimization of Stochastic Models: The Interface Between Simulation and Optimization, vol. 373. Springer, Berlin (2012)
[146] Qiu, J., Xia, J., Luo, J., Liu, Y., Liu, Y.: Integrated inventory placement and transportation vehicle selection using neural network. In: 2022 IEEE 18th International Conference on Automation Science and Engineering, pp. 1601-1608. IEEE (2022)
[147] Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686-707(2019)
[148] Reiman, M.I., Weiss, A.: Sensitivity analysis for simulations via likelihood ratios. Oper. Res. 37(5), 830-844(1989)
[149] Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22(3), 400-407(1951)
[150] Rubinstein, R., Shapiro, A.: Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method. Wiley, London (1993)
[151] Rubinstein, R.Y., Kroese, D.P.: The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning, vol. 133. Springer, Berlin (2004)
[152] Scaillet, O.: Nonparametric estimation and sensitivity analysis of expected shortfall. Mathematical Finance: An International Journal of Mathematics, Statistics and Financial Economics 14(1), 115- 129(2004)
[153] Shashaani, S., Hashemi, F.S., Pasupathy, R.: ASTRO-DF: a class of adaptive sampling trust-region algorithms for derivative-free stochastic optimization. SIAM J. Optim. 28(4), 3145-3176(2018)
[154] Shen, H., Hong, L.J., Zhang, X.: Ranking and selection with covariates for personalized decision making. INFORMS J. Comput. 33(4), 1500-1519(2021)
[155] Song, E., Shanbhag, U.V.: Stochastic approximation for simulation optimization under input uncertainty with streaming data. In: 2019 Winter Simulation Conference, pp. 3597-3608(2019)
[156] Spall, J.C.: Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Autom. Control 37(3), 332-341(1992)
[157] Sun, L., Hong, L.J., Hu, Z.: Optimization via simulation using Gaussian process-based search. In: Proceedings of the 2011 Winter Simulation Conference, pp. 4134-4145. IEEE (2011)
[158] Sun, L., Hong, L.J., Hu, Z.: Balancing exploitation and exploration in discrete optimization via simulation through a Gaussian process-based search. Oper. Res. 62(6), 1416-1438(2014)
[159] Sun, W., Hu, Z., Hong, L.J.: Gaussian mixture model-based random search for continuous optimization via simulation. In: 2018 Winter Simulation Conference, pp. 2003-2014. IEEE (2018)
[160] Sun, Y., Apley, D.W., Staum, J.: Efficient nested simulation for estimating the variance of a conditional expectation. Oper. Res. 59(4), 998-1007(2011)
[161] Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. The MIT Press, London (2020)
[162] Tong, J., Hu, J., Hu, J.: Computing equilibrium prices for a capital asset pricing model with heterogeneous beliefs and margin-requirement constraints. Eur. J. Oper. Res. 256(1), 24-34(2017)
[163] Tsai, S.C., Luo, J., Sung, C.C.: Combined variance reduction techniques in fully sequential selection procedures. Naval Res. Log. 64(6), 502-527(2017)
[164] Tsai, S.C., Luo, J., Jiang, G., Yeh, W.C.: Adaptive fully sequential selection procedures with linear and nonlinear control variates. IISE Trans. 55(6), 561-573(2023)
[165] Tulleken, H.J.: Grey-box modelling and identification using physical knowledge and Bayesian techniques. Automatica 29(2), 285-308(1993)
[166] Wang, S., Hong, L.J.: Option pricing by neural stochastic differential equations: a simulationoptimization approach. In: 2021 Winter Simulation Conference, pp. 1-11. IEEE (2021)
[167] Wang, T., Hong, L.J.: Large-scale inventory optimization: a recurrent neural networks-inspired simulation approach. INFORMS J. Comput. 35(1), 196-215(2023)
[168] Wang, T., Xu, J., Hu, J.-Q., Chen, C.-H.: Optimal computing budget allocation for regression with gradient information. Automatica 134, 109927(2021)
[169] Wang, X., Hong, L.J., Jiang, Z., Shen, H.: Gaussian process-based random search for continuous optimization via simulation. Articles in Advance, 1-23(2023)
[170] Wolfowitz, J.: On the stochastic approximation method of Robbins and Monro. Ann. Math. Stat. 2, 457-461(1952)
[171] Wu, D., Zhou, E.: Fixed confidence ranking and selection under input uncertainty. In: 2019 Winter Simulation Conference, pp. 3717-3727, 12(2019)
[172] Wu, D., Wang, Y., Zhou, E.: Data-driven ranking and selection under input uncertainty. Oper. Res. 72(2), 781-795(2022)
[173] Wu, J., Frazier, P.: The parallel knowledge gradient method for batch Bayesian optimization. Adv. Neural Inf. Process. Syst. 29, 28(2016)
[174] Wu, R., Liu, S., Luo, J.: Adaptive sampling rule for ranking-and-selection problem. In: 201713th IEEE Conference on Automation Science and Engineering, pp. 1499-1505. IEEE (2017)
[175] Wu, Y., Hu, J., Zhang, X.: Moment estimators for the parameters of Ornstein-Uhlenbeck processes driven by compound Poisson processes. Dis. Event Dyn. Syst. 29, 57-77(2019)
[176] Wu, Y., Hu, J., Yang, X.: Moment estimators for parameters of Levy-driven Ornstein-Uhlenbeck processes. J. Time Ser. Anal. 43(4), 610-639(2022)
[177] Wu, Y., Zheng, Z., Zhang, G., Zhang, Z., Wang, C.: Adaptive A/B tests and simultaneous treatment parameter optimization (2022). https://arxiv.org/abs/2210.06737
[178] Xu, J., Nelson, B.L., Hong, J.L.: Industrial strength compass: a comprehensive algorithm and software for optimization via simulation. ACM Trans. Model. Comput. Simul. 20(1), 1-29(2010)
[179] Yakowitz, S., L’ecuyer, P., Vazquez-Abad, F.: Global stochastic optimization with low-dispersion point sets. Oper. Res. 48(6), 939-950(2000)
[180] Yang, X., Hu, J.-Q., Hu, J., Peng, Y.: Asynchronous value iteration for Markov decision processes with continuous state spaces. In: 2020 Winter Simulation Conference, pp. 2856-2866. IEEE (2020)
[181] Yang, X., Hu, J., Hu, J.: Relative Q-learning for average-reward Markov decision processes with continuous states. Available at SSRN 3993508(2021)
[182] Yang, X., Wu, Y., Zheng, Z., Hu, J.-Q.: Method of moments estimation for Levy-driven OrnsteinUhlenbeck stochastic volatility models. Probab. Eng. Inf. Sci. 35(4), 975-1004(2021)
[183] Zhang, K., Liu, G., Wang, S.: Bootstrap-based budget allocation for nested simulation. Oper. Res. 70(2), 1128-1142(2021)
[184] Zhang, Q., Hu, J.: Actor-critic-like stochastic adaptive search for continuous simulation optimization. Oper. Res. 70(6), 3519-3537(2022)
[185] Zhong, Y., Hong, L.J.: Knockout-tournament procedures for large-scale ranking and selection in parallel computing environments. Oper. Res. 70(1), 432-453(2022)
[186] Zhong, Y., Liu, S., Luo, J., Hong, L.J.: Speeding up Paulsons procedure for large-scale problems using parallel computing. INFORMS J. Comput. 34(1), 586-606(2022)
[187] Zhou, E., Liu, T.: Online quantification of input uncertainty for parametric models. In: 2018 Winter Simulation Conference, pp. 1587-1598, 12(2018)