Safety considerations for online generative modeling

📖 Description

**Summary:** the [online decision transformer](https://arxiv.org/pdf/2202.05607.pdf) is a recent approach to creating agents in which a decision transformer is pre-trained offline (as usual) before producing its own trajectories which are fed back into the model in an online finetuning phase. I argue that agents made with generative modeling have safety advantages - but capabilities *dis*advantages - over agents made with other RL approaches, and agents made with *online* generative modeling (like the online decision transformer) may maintain these safety advantages while being closer to parity in capabilities. I propose experiments to test all this. (There is also an appendix discussing the connections between some of these ideas and KL-regularized RL.)

📊 Game Impacts

💭 Reactions

🔗 Sources

• [1] https://www.alignmentforum.org/posts/BMfNu82iunjqKyQA9/safety-considerations-for-online-generative-modeling