Overgangsprobabilities

Overgangsprobabilities, or transition probabilities, describe the likelihood that a stochastic process moves from one state to another over a given time step. In discrete-time Markov chains, these probabilities are organized in a transition matrix P, where P_ij = Pr(X_{t+1} = j | X_t = i). Each row of P sums to 1 and all entries are non-negative, reflecting the total probability of moving from a given state to all possible next states.

In continuous-time Markov chains, the concept is tied to instantaneous transition rates q_ij, which form a generator matrix Q. The transition probabilities over a time interval t are given by P(t) = exp(Qt). Here the diagonal entries satisfy q_ii = -sum_{j ≠ i} q_ij, ensuring that rows of P(t) again sum to 1 for each t.

Key properties include the Markov (memoryless) property and, in many models, time-homogeneity, where transition probabilities depend only on the elapsed time, not on calendar time. The notions of ergodicity, recurrent and absorbing states, and a stationary distribution pi satisfying pi = pi P (for discrete-time) describe long-run behavior.

Applications and computation: transition probabilities allow prediction of the state distribution after n steps via pi^{(n)} = pi^{(0)} P^n. They are estimated from data as relative frequencies of observed transitions, with smoothing used when data are sparse. They underpin models in economics, genetics, queueing theory, reliability, and many areas of decision making.

Limitations include model assumptions such as the Markov property and stationarity; real systems may require time-varying or higher-order models to capture dynamics accurately.