Subpolicy

Subpolicy is a term used in artificial intelligence and policy design to refer to a subordinate policy that governs behavior for a subtask within a larger task. In hierarchical or modular policy architectures, a high-level controller selects among a set of subpolicies and may activate a chosen subpolicy for a bounded period or until a termination condition is met. Subpolicies enable specialization and can simplify learning and planning by decomposing complex tasks into more manageable components.

In reinforcement learning, subpolicies appear in frameworks such as options or feudal reinforcement learning. Each subpolicy can be viewed as an option with its own action policy, termination condition, and initiation criteria. The master policy chooses which subpolicy to execute in a given state. While the subpolicy runs, it interacts with the environment and may learn its own value function or policy, potentially sharing representations with other components. Training is often pursued end-to-end or via separate phases, and using subpolicies can improve sample efficiency and transfer to related tasks.

In policy management outside AI, subpolicy may refer to a subordinate policy document within a broader policy framework, addressing domain-specific rules while aligning with overall policy objectives. In practice, subpolicies provide clarity, governance, and auditability by detailing procedures under the umbrella of a main policy.

See also: hierarchical reinforcement learning, options framework, feudal reinforcement learning, policy decomposition.