Announcing the Inverse Scaling Prize ($250k Prize Pool)

📖 Description

*TL;DR*: We're launching the [Inverse Scaling Prize](https://twitter.com/EthanJPerez/status/1541454949397041154): a contest with $250k in prizes for finding zero/few-shot text tasks where larger language models show increasingly undesirable behavior ("inverse scaling"). We hypothesize that inverse scaling is often a sign of an alignment failure and that more examples of alignment failures would benefit empirical alignment research. We believe that this contest is an unusually concrete, tractable, and safety-relevant problem for engaging alignment newcomers and the broader ML community. This post will focus on the relevance of the contest and the inverse scaling framework to longer-term AGI alignment concerns. See our [GitHub repo](https://github.com/inverse-scaling/prize) for contest details, prizes we'll award, and task evaluation criteria.

📊 Game Impacts

💭 Reactions

🔗 Sources

• [1] https://www.alignmentforum.org/posts/eqxqgFxymP8hXDTt5/announcing-the-inverse-scaling-prize-usd250k-prize-pool