Discrete-Time Markov Chains

Abstract

In this chapter, we look at how Monte Carlo simulations and the Markov chain theory can be used to analyze urban transportation problems. Vectors describing the starting and final states of a public transportation network are introduced as fundamental notions. The transition probability matrix and the stochastic matrix are investigated as potential tools for modeling the dynamic evolution of urban mobility using discrete-time Markov Chains. The features of Markov Chains, as revealed by their eigenvalues and eigenvectors, are examined. The Markov Chain Monte Carlo technique for statistical sampling and analysis of urban mobility issues is also discussed. The methodologies’ potential utility in urban transportation planning and decision-making is emphasized. Understanding and addressing the difficulties of urban mobility are greatly aided by the theoretical and conceptual groundwork laid out in this chapter.

Keywords: Discrete-time Markov chain, eigenvalues, eigenvectors, Markov chain Monte Carlo, regular matrix, states, stochastic matrix, transition probabilitiesr, . transition probability matrix, urban mobility issues, vector in the initial state, vector in a steady state.