transitionmatrix

A transition matrix, in probability theory, is a matrix used to describe the probabilities of transitioning from one state to another in one step of a stochastic process. In a finite state space with n states, the transition matrix P is an n-by-n matrix with entries p_ij = P(X_{t+1} = j | X_t = i). Under the common convention of row-stochastic matrices, each row sums to 1, since from state i the process must move to some state j. Some texts use column-stochastic convention.

Properties: all entries are nonnegative; P^k gives the k-step transition probabilities, with (P^k)_{ij} = P(X_{t+k} = j | X_t = i). If the chain is time-homogeneous, P does not depend on t; if not, P varies with time. A stationary distribution pi satisfies pi P = pi and sum pi_i = 1; under irreducibility and aperiodicity, the chain converges to a unique stationary distribution as k grows.

Types and structures: absorbing states correspond to rows with a single 1 and zeros elsewhere. Absorbing Markov chains have a fundamental matrix N = (I - Q)^{-1} for the transient-submatrix Q. Other common classifications include ergodicity, irreducibility, and periodicity.

Applications and computation: transition matrices are central to modeling Markov chains, stochastic processes, and algorithms such as PageRank. They are typically sparse in large problems, and methods such as power iteration or eigen-decomposition are used to analyze long-run behavior.

Example: a 2x2 matrix P = [[0.7, 0.3], [0.4, 0.6]] describes a process with two states where from state 1 you remain in state 1 with probability 0.7 and move to state 2 with probability 0.3, and similarly for transitions from state 2.