The two-layer model of human values, and problems with synthesizing preferences

📖 Description

I have been thinking about Stuart Armstrong's [preference synthesis research agenda](https://www.lesswrong.com/posts/CSEdLLEkap2pubjof/research-agenda-v0-9-synthesising-a-human-s-preferences-into), and have long had the feeling that there's something off about the way that it is currently framed. In the post I try to describe why. I start by describing my current model of human values, how I interpret Stuart's implicit assumptions to conflict with it, and then talk about my confusion with regard to reconciling the two views.

📊 Game Impacts

💭 Reactions

🔗 Sources

• [1] https://www.alignmentforum.org/posts/2yLn8iTrvHoEgqXcJ/the-two-layer-model-of-human-values-and-problems-with