NonMarkov

Non-Markov, in the context of stochastic processes, refers to systems whose future evolution cannot be predicted solely from their present state. In a Markov process, the future is independent of the past given the present; non-Markov processes exhibit memory effects in which historical information influences transitions and outcomes.

Causes of non-Markov behavior include correlations in noise, delayed feedback, or interactions with an environment that retains memory of the system’s past states, leading to history-dependent evolution. Classic examples are fractional Brownian motion, autoregressive processes with long-range dependence, and renewal processes with non-exponential waiting times. In physics and chemistry, non-Markovian dynamics describe systems where the environment or bath retains memory of the system's past states, leading to history-dependent evolution.

Mathematical treatment often involves non-local in time equations, such as generalized Langevin equations with memory kernels, or integro-differential equations. A common modeling strategy is to augment the state to create a higher-dimensional Markovian representation, a method known as Markovian embedding. Semi-Markov processes generalize Markov chains by allowing non-exponential waiting times but preserve the Markov property for the embedded chain; more general non-Markovian models are not captured by semi-Markov form.

Applications span physics, chemistry, finance, biology, and queueing theory. In machine learning and reinforcement learning, non-Markovian environments require memory-aware models and recurrent architectures; but many methods assume Markov decision processes, so acknowledging non-Markovian dynamics is important for model design and evaluation.