Limits to Verification and Validation of Agentic Behavior

📖 Description

Limits to Verification and Validation of Agentic Behavior David J. Jilk 1 Abstract Verification and validation of agentic behavior have been suggested as important research priorities in efforts to redu ce risks associated with the creation of general artificial intelligence (Russell et al 2015). In th is paper we question the appropriateness of using language of certainty with respect to e fforts to manage that risk. We begin by establishing a very general formalism to characterize agentic behavior and to describe standards of acceptable behavior. We show that determinati on of whether an agent meets any particular standard is not computable. We discuss the extent of the burden associated with verification by manual proof and by automated behav ioral governance. We show that to ensure decidability of the behavioral standard itself, one must further limit the capabilities of the agent. We then demonstrate that if our concerns relate to ou tcomes in th...

📊 Game Impacts

💭 Reactions

🔗 Sources

• [1] https://arxiv.org/abs/1604.06963