DvalueF

DvalueF is a term used in decision theory and machine learning to describe a family of value functions that incorporate a time- or state-dependent devaluation of future rewards. It generalizes the standard discounted value function by allowing an additional weighting, D(t) or D(s, a, t), to modulate the contribution of rewards over time or across states. This flexible framing enables modeling scenarios where distant outcomes are valued differently due to risk, uncertainty, or horizon considerations.

Formally, for a policy π in a Markov decision process, the DvalueF of a starting state s can be defined as V^π_D(s) = E_π [ sum_{t=0}^∞ γ^t · D(t) · R_{t} | s_0 = s ], where γ is the base discount factor, R_t is the reward received at time t, and D(t) is a nonnegative devaluation function. If D(t) ≡ 1 for all t, DvalueF reduces to the standard value function. Several common choices include D(t) = exp(-λt) for continuous decay or D(t) = (1 - t/T) for finite-horizon contexts.

Computation and learning with DvalueF adapt standard temporal-difference or Monte Carlo methods by replacing the plain discounting with the D-weighted rewards. The contraction properties of the Bellman operator are preserved under suitable boundedness assumptions on D(t). This makes DvalueF amenable to existing RL algorithms with minor modifications.

Applications of DvalueF appear in long-horizon planning under uncertainty, risk-sensitive control, robotics, and finance, where near-term outcomes are prioritized or where discounting should reflect dynamic risk or horizon constraints. It serves as a bridge between conventional value functions and models that require flexible, context-dependent valuation.