CTMC

CTMC stands for continuous-time Markov chain. It is a stochastic process {X_t, t ≥ 0} with a state space S (finite or countably infinite) that satisfies the Markov property in continuous time: future evolution depends only on the present state, not on the past, and transitions can occur at any time.

The process is described by a generator or rate matrix Q = [q_ij], where q_ij ≥ 0 for i ≠ j and q_ii = −∑_{j ≠ i} q_ij. The rows of Q sum to zero. From state i, the holding time before a jump is exponential with rate −q_ii, and the next state j ≠ i is chosen with probability q_ij/(−q_ii). The transition probability matrix P(t) = [P_ij(t)] has P_ij(t) = P(X_t = j | X_0 = i) and satisfies P(t+s) = P(t)P(s) and P(0) = I. If conditions permit, P(t) = exp(Qt).

CTMCs are typically time-homogeneous, meaning Q is constant in time. They can be finite or infinite in state space and are often analyzed via the Kolmogorov forward and backward equations: d/dt P_ij(t) = ∑_k P_ik(t) q_kj (forward) or d/dt P_ij(t) = ∑_k q_ik P_kj(t) (backward).

Common examples include birth-death processes, where transitions occur only between neighboring states with rates λ_i (i → i+1) and μ_i (i → i−1). CTMCs model queueing networks, reliability systems, population dynamics, finance, and genetics.

A stationary distribution π exists (and is unique under irreducibility and positive recurrence) as a solution to πQ = 0 with ∑_i π_i = 1, describing long-run state probabilities when it exists.