semiMarkov

A semi-Markov process is a stochastic model that generalizes a Markov process by allowing the time spent in each state before transitioning to the next state to have a general, possibly state-dependent distribution rather than an exponential one. This makes sojourn times non-memoryless and can capture age- or history-dependent behavior.

Formally, let S be a countable state space. The process is described by an embedded Markov chain Xn with transition probabilities pij = P(Xn+1 = j | Xn = i). When the process is in state i, it waits for a random time before jumping to the next state, with a distribution Fi j(t) that may depend on both the current state i and the next state j. The joint law of the next state and the sojourn time is given by a semi-Markov kernel Q(i, j, t) = P(Xn+1 = j, Tn ≤ t | Xn = i) = pij Fi j(t). The continuous-time process X(t) remains in state i for the sojourn time and then jumps to j, so X(t) = i during the ith sojourn and changes at the jump times.

Relation to Markov processes: if all holding times are exponential with rates that depend only on the current state (and not on the chosen next state), the semi-Markov process reduces to a continuous-time Markov chain. Conversely, choosing general Fi j(t) yields non-exponential waiting times and greater modeling flexibility.

Applications and theory: semi-Markov models are used in reliability engineering, queueing theory, finance, economics, and survival analysis to model systems with non-exponential or age-dependent sojourns. They are analyzed using Markov renewal process theory and renewal theory to study long-run behavior, stationary distributions, and ergodicity under conditions on the embedded chain and holding-time distributions. See also Markov renewal process and phase-type distributions.