Qfunction

Q-function is a term used in several disciplines, referring to different concepts that share a common label but serve distinct purposes. In statistics and communications, the Q-function denotes the tail probability of the standard normal distribution. In reinforcement learning, the Q-function is the action-value function used to estimate expected returns.

In statistics, the Q-function is defined as Q(x) = ∫_x^∞ φ(t) dt = 1 − Φ(x), where φ is the standard normal density and Φ is the standard normal cumulative distribution function. It can also be written as Q(x) = (1/2) erfc(x/√2). The function is nonincreasing, with Q(-x) = Φ(x) and Q(x) → 0 as x → ∞. It is commonly used to express error probabilities in binary communication over additive white Gaussian noise (AWGN), for example P_e = Q(√(2 E_b/N_0)) for binary phase-shift keying (BPSK).

In reinforcement learning, the Q-function Q(s, a) represents the expected discounted return when taking action a in state s and thereafter following a given policy. It satisfies the Bellman equation: Q(s, a) = E[ r_{t+1} + γ max_{a'} Q(s', a') | s_t = s, a_t = a ]. Q-learning and deep Q-networks approximate the Q-function from interaction data without a complete model of the environment, enabling policy improvement by selecting actions that maximize Q.

The two uses of Q-function are distinct and context-dependent, and the symbol is not universally synonymous across fields.