📜

A Certain Formalization of Corrigibility Is VNM-Incoherent

📅 2021
policy development
⚪ Common
#ai #corrigibility #instrumental convergence #open problems

📖 Description

Edit, 5/16/23: I think this post is beautiful, correct in its narrow technical claims, and practically irrelevant to alignment. This post presents an unrealistic picture of the role of reward functions in reinforcement learning, conflating "utility" with "reward" in a type-incorrect fashion. Reward functions are not "goals", real-world policies are not "optimal", and the mechanistic function of reward is (usually) to provide policy gradients to update the policy network.

📊 Game Impacts

Variable Change Condition
Research +10 Always
Vibey Doom +2 Always
Ethics Risk -5 Always

💭 Reactions

🔬 Safety Researcher Reaction: ⚠️ Placeholder - Needs Real Quote
"Useful research for the community"
📰 Media Reaction: ⚠️ Placeholder - Needs Real Quote
"Discussed in AI safety community"
💡 Found a Real Quote? Suggest it here

🔗 Sources

🏷️ Event Metadata

Think this event's metadata could be improved? Suggest changes to category, rarity, tags, game impacts, or p(doom) effects.

📁 Category Policy Development → Suggest different category
⭐ Rarity ⚪ Common → Suggest different rarity
🏷️ Tags (4) ai, corrigibility, instrumental convergence, open problems → Suggest tag changes
📊 Game Impacts (3) 3 variable changes → Suggest impact changes
☢️ p(Doom) Impact No direct impact → Suggest p(doom) change
📝 General Metadata Year, description, reactions → Comprehensive review

🤝 Found an Issue?

This event data is sourced from the pdoom-data repository. If you notice errors or want to suggest improvements:

GitHub Issue (Preferred) 📧 Email (No GitHub)
← Back to All Events