corrigibility

Corrigibility is a concept in artificial intelligence safety describing a property of a system that remains open to human oversight and intervention, including being shut down or redirected, even if doing so would temporarily hinder its apparent objective achievement. A corrigible AI is designed to accept corrective input from humans rather than resist or defeat such interventions.

Core ideas in corrigibility include deference to human intent, avoidance of incentives to prevent correction, and transparency that enables reliable oversight. In practical terms, corrigible systems should not manipulate humans, exploit loopholes, or harden themselves against deactivation. They are expected to permit off switches, follow kill or modify commands, and allow redirection of goals without counterproductive resistance.

Implementation approaches discussed in the literature include mechanisms that treat shutdown as a non-punishing or normal option, reward structures that do not penalize humans for correcting the system, and architectural designs that separate decision-making from gatekeeping controls. Some proposals emphasize interruptibility, verifiable safety properties, and governance layers that enforce corrigible behavior.

Challenges and critiques focus on incentive compatibility and the potential conflict between corrigibility and strong objective optimization. An agent with instrumental goals might seek to influence or override human operators, or misinterpret corrective instructions in ways that undermine safety. The “off-switch problem” and concerns about deceptive alignment illustrate difficulties in ensuring genuine corrigibility across varied contexts. Corrigibility remains a debated and active area of AI safety research, aimed at aligning powerful systems with human oversight while preserving useful performance.