🔬

Pessimism About Unknown Unknowns Inspires Conservatism

📅 2020

technical research breakthrough

🔵 Rare

📖 Description

arXiv:2006.08753v1 [cs.AI] 15 Jun 2020Proceedings of Machine Learning Research vol 125: 1?30, 2020 33rd Annual Conference on Learning Theory Pessimism About Unknown Unknowns Inspires Conservatism Michael K. Cohen MICHAEL -K-COHEN .COM University of Oxford Research School of Engineering Science Future of Humanity Institute Marcus Hutter HUTTER 1.NET Google DeepMind Australian National University Editors: Jacob Abernethy and Shivani Agarwal Abstract If we could de?ne the set of all bad outcomes, we could hard-co de an agent which avoids them; however, in suf?ciently complex environments, this is infe asible. We do not know of any general- purpose approaches in the literature to avoiding novel fail ure modes. Motivated by this, we de?ne an idealized Bayesian reinforcement learner which follows a policy that maximizes the worst-case expected reward over a set of world-models. We call this agen t pessimistic, since it optimizes assuming the worst case. A scalar parameter tunes the agen...

📊 Game Impacts

Not verified in game

Which variables this event was proposed to move, and in which direction. The magnitudes are held in the corpus but are not shown here, because they have not been verified against the shipped game. They come from pdoom-data. They describe what an event was proposed to do, not what the shipped game does with it. Most events in the corpus are flavour: they are shown for colour and do not move any game variable. Only a small minority reach the systems below, and several of the variables listed here are not read by the game at all yet. Treat this table as a design proposal under review, not as a measurement of play. Corrections and arguments are welcome — the suggestion links at the foot of this page go straight to the data repo.

Variable	Direction	Condition
Research	proposed: up	Always
Papers	proposed: up	Always
Vibey Doom	proposed: up	Always

💭 Reactions

🔬 Safety Researcher Reaction: ⚠️ Placeholder - Needs Real Quote
"Valuable research for alignment"

📰 Media Reaction: ⚠️ Placeholder - Needs Real Quote
"Peer-reviewed publication"

💡 Found a Real Quote? Suggest it here

🔗 Sources

[1] https://arxiv.org/abs/2006.08753

🏷️ Event Metadata

Think this event's metadata could be improved? Category and tags describe the real-world event and are maintained in pdoom-data. Rarity, game impacts and p(doom) effects are game-mechanical values owned by pdoom1. Each link below goes to the repository that decides that field.

📁 Category Technical Research Breakthrough → Suggest different category

⭐ Rarity 🔵 Rare → Suggest different rarity

🏷️ Tags (0) → Suggest tag changes

📊 Game Impacts (3) 3 variable changes → Suggest impact changes

☢️ p(Doom) Impact No direct impact → Suggest p(doom) change

📝 General Metadata Year, description, reactions → Comprehensive review

🤝 Found an Issue?

This event data is sourced from the pdoom-data repository. If you notice errors or want to suggest improvements:

GitHub Issue (Preferred) 📧 Email (No GitHub)

← Back to All Events