📢

Thoughts on gradient hacking

📅 2021
public awareness
⚪ Common
#ai #gradient hacking #mesa-optimization

📖 Description

Gradient hacking is the hypothesised phenomenon of a machine learning model, during training, deliberate thinking in ways which guide gradient descent to update its parameters in the directions it desires. The key intuition here is that because the loss landscape of a model is based on the cognition it does, models can make decisions for the purpose of affecting their loss landscapes, thereby affecting the directions in which they are updated. [Evan writes](https://www.alignmentforum.org/posts/uXH4r6MmKPedk8rMA/gradient-hacking):

📊 Game Impacts

Variable Change Condition
Research +5 Always
Vibey Doom +2 Always
Ethics Risk -5 Always

💭 Reactions

🔬 Safety Researcher Reaction: ⚠️ Placeholder - Needs Real Quote
"Interesting perspective on safety challenges"
📰 Media Reaction: ⚠️ Placeholder - Needs Real Quote
"Discussed in AI safety community"
💡 Found a Real Quote? Suggest it here

🔗 Sources

🏷️ Event Metadata

Think this event's metadata could be improved? Suggest changes to category, rarity, tags, game impacts, or p(doom) effects.

📁 Category Public Awareness → Suggest different category
⭐ Rarity ⚪ Common → Suggest different rarity
🏷️ Tags (3) ai, gradient hacking, mesa-optimization → Suggest tag changes
📊 Game Impacts (3) 3 variable changes → Suggest impact changes
☢️ p(Doom) Impact No direct impact → Suggest p(doom) change
📝 General Metadata Year, description, reactions → Comprehensive review

🤝 Found an Issue?

This event data is sourced from the pdoom-data repository. If you notice errors or want to suggest improvements:

GitHub Issue (Preferred) 📧 Email (No GitHub)
← Back to All Events