Training goals for large language models

📖 Description

*This post was written under Evan Hubinger's mentorship, as part of the*[*Stanford Existential Risks Initiative ML Alignment Theory Scholars (SERI MATS) program*](https://www.lesswrong.com/posts/8vLvpxzpc6ntfBWNo/seri-ml-alignment-theory-scholars-program-2022)*. Many of the ideas in this post, including the main idea behind the training goal, are due to Kyle McDonell and Laria Reynolds. In addition, I am grateful for comments and feedback from Arun Jose (who wrote a related post on*[*conditioning generative models for alignment*](https://www.alignmentforum.org/posts/JqnkeqaPseTgxLgEL/conditioning-generative-models-for-alignment)*) and Caspar Oesterheld, and for a helpful discussion with James Lucassen.*

📊 Game Impacts

💭 Reactions

🔗 Sources

• [1] https://www.alignmentforum.org/posts/dWJNFHnC4bkdbovug/training-goals-for-large-language-models