Motivations, Natural Selection, and Curriculum Engineering

📖 Description

*Epistemic status: I am professionally familiar with contemporary machine learning theory and practice, and have a long-standing amateur interest in natural history. But (despite being a long-term lurker on LW and AF) I feel very uncertain about the nature and status of goals and motivations! This post is an attempt, heavily inspired by Richard Ngo's* [Shaping Safer Goals sequence](https://www.lesswrong.com/s/boLPsyNwd6teK5key), *to lay out a few ways of thinking about goals and motivations in natural and artificial systems, to point to a class of possible prosaic development paths to AGI where general capabilities emerge from extensive open-ended training, and finally observe some desiderata and gaps in our understanding regarding alignment conditional on this situation.*

📊 Game Impacts

💭 Reactions

🔗 Sources

• [1] https://www.alignmentforum.org/posts/RwYh4grJs4pbJdTh3/motivations-natural-selection-and-curriculum-engineering