📜

My AGI Threat Model: Misaligned Model-Based RL Agent

📅 2021

policy development

🔵 Rare

#ai #inner alignment #outer alignment #reinforcement learning #threat models

📖 Description

[Rohin Shah advocates](https://www.youtube.com/watch?v=VC_J_skJNMs) a vigorous discussion of "Threat Models", i.e. stories for how AGI is developed, what the AGI then looks like, and then what might go catastrophically wrong.

📊 Game Impacts

Variable	Change	Condition
Research	+10	Always
Vibey Doom	+5	Always
Ethics Risk	-5	Always

💭 Reactions

🔬 Safety Researcher Reaction: ⚠️ Placeholder - Needs Real Quote
"This is a significant contribution to alignment research"

📰 Media Reaction: ⚠️ Placeholder - Needs Real Quote
"Discussed in AI safety community"

💡 Found a Real Quote? Suggest it here

🔗 Sources

[1] https://www.alignmentforum.org/posts/zzXawbXDwCZobwF9D/my-agi-threat-model-misaligned-model-based-rl-agent

🏷️ Event Metadata

Think this event's metadata could be improved? Suggest changes to category, rarity, tags, game impacts, or p(doom) effects.

📁 Category Policy Development → Suggest different category

⭐ Rarity 🔵 Rare → Suggest different rarity

🏷️ Tags (5) ai, inner alignment, outer alignment, reinforcement learning, threat models → Suggest tag changes

📊 Game Impacts (3) 3 variable changes → Suggest impact changes

☢️ p(Doom) Impact No direct impact → Suggest p(doom) change

📝 General Metadata Year, description, reactions → Comprehensive review

🤝 Found an Issue?

This event data is sourced from the pdoom-data repository. If you notice errors or want to suggest improvements:

GitHub Issue (Preferred) 📧 Email (No GitHub)

← Back to All Events