🔬

Limits to Verification and Validation of Agentic Behavior

📅 2016

technical research breakthrough

🔵 Rare

📖 Description

Limits to Verification and Validation of Agentic Behavior David J. Jilk 1 Abstract Verification and validation of agentic behavior have been suggested as important research priorities in efforts to redu ce risks associated with the creation of general artificial intelligence (Russell et al 2015). In th is paper we question the appropriateness of using language of certainty with respect to e fforts to manage that risk. We begin by establishing a very general formalism to characterize agentic behavior and to describe standards of acceptable behavior. We show that determinati on of whether an agent meets any particular standard is not computable. We discuss the extent of the burden associated with verification by manual proof and by automated behav ioral governance. We show that to ensure decidability of the behavioral standard itself, one must further limit the capabilities of the agent. We then demonstrate that if our concerns relate to ou tcomes in th...

📊 Game Impacts

Not verified in game

Which variables this event was proposed to move, and in which direction. The magnitudes are held in the corpus but are not shown here, because they have not been verified against the shipped game. They come from pdoom-data. They describe what an event was proposed to do, not what the shipped game does with it. Most events in the corpus are flavour: they are shown for colour and do not move any game variable. Only a small minority reach the systems below, and several of the variables listed here are not read by the game at all yet. Treat this table as a design proposal under review, not as a measurement of play. Corrections and arguments are welcome — the suggestion links at the foot of this page go straight to the data repo.

Variable	Direction	Condition
Research	proposed: up	Always
Papers	proposed: up	Always
Vibey Doom	proposed: up	Always

💭 Reactions

🔬 Safety Researcher Reaction: Not recorded

📰 Media Reaction: Not recorded

💡 Found a Real Quote? Suggest it here

🔗 Sources

[1] https://arxiv.org/abs/1604.06963

🏷️ Event Metadata

Think this event's metadata could be improved? Category and tags describe the real-world event and are maintained in pdoom-data. Rarity, game impacts and p(doom) effects are game-mechanical values owned by pdoom1. Each link below goes to the repository that decides that field.

📁 Category Technical Research Breakthrough → Suggest different category

⭐ Rarity 🔵 Rare → Suggest different rarity

🏷️ Tags (0) → Suggest tag changes

📊 Game Impacts (3) 3 variable changes → Suggest impact changes

☢️ p(Doom) Impact Not recorded → Suggest p(doom) change

📝 General Metadata Year, description, reactions → Comprehensive review

🤝 Found an Issue?

This event data is sourced from the pdoom-data repository. If you notice errors or want to suggest improvements:

GitHub Issue (Preferred) 📧 Email (No GitHub)

← Back to All Events