CONVERGENCE OF ADAPTIVE TIME-VARYING MARKOV CHAINS UNDER DYNAMIC ENVIRONMENTS
DOI:
https://doi.org/10.60787/tnamp.v25.743Keywords:
Non-homogeneous Markov chain, Weak ergodicity, Dynamic environment, Strong ergodicity, Dobrushin coefficientAbstract
This study develops an adaptive time-varying Markov chain model for stochastic systems operating in dynamic environments. The model combines adaptive parameter updating with non-homogeneous transition probabilities to capture learning and environmental changes. Using weak ergodicity, strong ergodicity, and the Dobrushin coefficient, convergence properties are established, demonstrating the existence of a unique stationary distribution and asymptotic stability under suitable conditions. An illustrative numerical example validates the theoretical results under a periodic environment. The proposed framework extends adaptive and non-homogeneous Markov chain theory and provides a basis for modeling evolving stochastic systems.
Downloads
References
Andrieu, C., & Moulines, É. (2006). On the ergodicity properties of some adaptive MCMC algorithms. The Annals of Applied Probability, 16(3), 1462-1505.
Atchade, Y. F., Fort, G., Moulines, É., & Priouret, P. (2011). Adaptive Markov chain Monte Carlo: Theory and methods. In Bayesian Time Series Models (pp. 32–51). Cambridge University Press. https://doi.org/10.1017/CBO9780511984679.003
Brown, A., & Jeffrey S. Rosenthal (2024). Weak convergence of adaptive Markov chain Monte Carlo. arXiv. arXiv:2406.00820.
Dobrushin, R. L. (1956). Central limit theorem for non-stationary Markov chains. Theory of Probability and Its Applications, 1(1), 65–80.
Erb, R. (2023). Bounds on mixing time for time-inhomogeneous Markov chains. arXiv.
Fort, G., Moulines, É., & Priouret, P. (2011). Convergence of adaptive and interacting Markov chain Monte Carlo algorithms. The Annals of Statistics, 39(6), 3262–3289.
Golomoziy, V., & Moskanova, O. (2023). Polynomial recurrence of time-inhomogeneous Markov chains. Austrian Journal of Statistics, 52(Special Issue), 40–53.
Haario, H., Saksman, E., & Tamminen, J. (2001). An adaptive Metropolis algorithm. Bernoulli, 7(2), 223–242. https://doi.org/10.2307/3318737
Hajnal, J., & Maurice, S. B,(1958). Weak ergodicity in non-homogeneous Markov chains. Proceedings of the Cambridge Philosophical Society, 54(2), 233–246.
Jin, S. S., Ju, H., & Jung, H. J. (2019). Adaptive Markov chain Monte Carlo algorithms for Bayesian inference: Recent advances and comparative study. Structure and Infrastructure Engineering, 15(11), 1548–1565. https://doi.org/10.1080/15732479.2019.1628077
Kushner, H. J., & Yin, G. G. (2003). Stochastic approximation and recursive algorithms and applications (2nd ed.). Springer.
Maire, F., Friel, N., Mira, A., & Adrian E. Raftery (2019). Adaptive incremental mixture Markov chain Monte Carlo. Journal of Comp. and Graphical Statistics, 28(4), 790–805.
Meyn, S. P., & Tweedie, R. L. (2009). Markov chains and stochastic stability (2nd ed.). Cambridge University Press.
Moumeni, N. (2024). Quantitative merging for time-inhomogeneous Markov chains in nondecreasing environments via functional inequalities. arXivhttps://arxiv.org/abs/2404.11432
Norris, J. R. (1998). Markov chains. Cambridge University Press.
Oçafrain, W. (2023). An ergodic theorem for asymptotically periodic time-inhomogeneous Markov processes, with application to quasi-stationarity with moving boundaries. Advances in Applied Probability. Advance online publication. https://doi.org/10.1017/apr.2023.53
Robbins, H., & Monro, S. (1951). A stochastic approximation method. The Annals of Mathematical Statistics, 22(3), 400–407. https://doi.org/10.1214/aoms/1177729586
Roberts, G. O., & Jeffrey S. Rosenthal (2007). Coupling and ergodicity of adaptive Markov chain Monte Carlo algorithms. Journal of Applied Probability, 44(2), 458–475.
Seneta, E. (2006). Non-negative matrices and Markov chains. Springer.
Vassiliou, P.C. G. (2024). Estimation–calibration of continuous-time non-homogeneous Markov chains with finite state space. Mathematics, 12(5), Article 668.
Vassiliou, P.C. G. (2024b). Strong ergodicity in nonhomogeneous Markov systems with chronological order. Mathematics, 12(5), Article 660.
Damasio, B. & Nicolau, J. (2024). Time-inhomogeneous multivariate Markov chains: Detecting and testing multiple structural breaks occurring at unknown dates. Chaos, Solitons & Fractals, 181, Article 114478. https://doi.org/10.1016/j.chaos.2024.114478
Wolfowitz, J. (1963). Products of indecomposable, aperiodic, stochastic matrices.
Proceedings of the American Mathematical Society, 14, 733–737.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 The Transactions of the Nigerian Association of Mathematical Physics
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
