Main Content

Markov Chain Models

Discrete state-space processes characterized by transition matrices

A discrete state-space Markov process, or Markov chain, is represented by a directed graph and described by a right-stochastic transition matrix P. The distribution of states at time t + 1 is the distribution of states at time t multiplied by P. The structure of P determines the evolutionary trajectory of the chain, including asymptotics.

For an overview of the Markov chain analysis tools, see Markov Chain Modeling.


expand all

dtmcCreate discrete-time Markov chain
mcmixCreate random Markov chain with specified mixing structure
asymptoticsDetermine Markov chain asymptotics
isergodicCheck Markov chain for ergodicity
isreducibleCheck Markov chain for reducibility
classifyClassify Markov chain states
lazyAdjust Markov chain state inertia
subchainExtract Markov subchain
hitprobCompute Markov chain hitting probabilities
hittimeCompute Markov chain hitting times
redistributeCompute Markov chain redistributions
simulateSimulate Markov chain state walks
distplotPlot Markov chain redistributions
eigplotPlot Markov chain eigenvalues
graphplotPlot Markov chain directed graph
simplotPlot Markov chain simulations