Is Learning World Model Always Beneficial For Reinforcement Learning?

We propose a hypothesis in model-based reinforcement learning (MBRL): the RL agent can learn to solve tasks faster by learning to interact with a learned world model and exploit the imperfect information about the environment. We develop two different architectures to evaluate this hypothesis. We show that the policy with access to such information outperforms the standalone policy on toy benchmarks. The results suggest that this is a promising revenue of research towards efficient MBRL algorithms that do not rely on rollouts.