📢

[Proposal] Method of locating useful subnets in large models

📅 2021
public awareness
🔵 Rare
#ai #interpretability (ml & ai) #machine learning (ml) #mesa-optimization #outer alignment #reinforcement learning

📖 Description

I've seen it suggested (e.g, [here](https://www.alignmentforum.org/posts/SzrmsbkqydpZyPuEh/my-take-on-vanessa-kosoy-s-take-on-agi-safety)) that we could tackle the outer alignment problem by using interpretability tools to locate the learned "human values" subnet of powerful, unaligned models. Here I outline a general method of extracting such subnets from a large model.

📊 Game Impacts

Variable Change Condition
Research +5 Always
Vibey Doom +5 Always
Ethics Risk -5 Always

💭 Reactions

🔬 Safety Researcher Reaction: ⚠️ Placeholder - Needs Real Quote
"Important work advancing our understanding of AI safety"
📰 Media Reaction: ⚠️ Placeholder - Needs Real Quote
"Discussed in AI safety community"
💡 Found a Real Quote? Suggest it here

🔗 Sources

🏷️ Event Metadata

Think this event's metadata could be improved? Suggest changes to category, rarity, tags, game impacts, or p(doom) effects.

📁 Category Public Awareness → Suggest different category
⭐ Rarity 🔵 Rare → Suggest different rarity
🏷️ Tags (6) ai, interpretability (ml & ai), machine learning (ml), mesa-optimization, outer alignment, reinforcement learning → Suggest tag changes
📊 Game Impacts (3) 3 variable changes → Suggest impact changes
☢️ p(Doom) Impact No direct impact → Suggest p(doom) change
📝 General Metadata Year, description, reactions → Comprehensive review

🤝 Found an Issue?

This event data is sourced from the pdoom-data repository. If you notice errors or want to suggest improvements:

GitHub Issue (Preferred) 📧 Email (No GitHub)
← Back to All Events