Position: RL Researchers Need to Distinguish Between Solving Simulators and Using Simulators as a Proxy

Short summary

An arXiv position paper argues that RL researchers conflate two distinct simulator use cases: solving the simulator itself versus using simulators as proxies for real-world deployment. The authors distinguish the different constraints, algorithms, and metrics appropriate for each, highlighting how confusion between them leads to misleading conclusions. They call on the research community to clearly delineate which use case they're investigating.

• •RL research often confuses solving simulators with using them as deployment proxies—fundamentally different problems
• •Each requires different constraints, algorithms, and evaluation metrics
• •Lack of clarity leads to research conclusions that don't transfer to real-world agent deployment

Generated with AI, which can make mistakes.