Epsilongreedy

Epsilon-greedy, sometimes written as epsilongreedy in some texts, is a simple action-selection strategy used in reinforcement learning to balance exploration and exploitation. In a given state s, the agent computes the greedy action a* = argmax_a Q(s,a). With probability 1 minus epsilon, it selects a*, exploiting current knowledge. With probability epsilon, it selects an action at random from the available actions, exploring alternatives. The parameter epsilon (between 0 and 1) controls the exploration-exploitation trade-off.

Variants of the approach include fixed epsilon, where epsilon remains constant, and decaying or annealed epsilon, where epsilon decreases over time to shift toward exploitation as the agent learns. Common schedules reduce epsilon linearly or exponentially, often with a floor to preserve occasional exploration.

Epsilon-greedy is widely used in tabular Q-learning and SARSA, as well as in solving stochastic multi-armed bandit problems. Its appeal lies in simplicity and low computational cost: evaluating the greedy action and performing a random selection are inexpensive even in large state spaces when Q-values are already maintained.

Limitations of the method include inefficiency in large or continuous action spaces, where random exploration can waste trials on poor actions. It can also adapt slowly in non-stationary environments. To address these issues, practitioners sometimes employ alternatives or enhancements such as soft or tempered action selection, upper confidence bound, or Thompson sampling, which aim to produce more informative exploration strategies.