2

Model based RL attempts to learn a function $f(s_{t+1}|s_t, a_t)$ representing the environment transitions, otherwise known as a model of the system. I see linear functions are still being used in model-based RL such as in robotic manipulation to learn system dynamics, and can work effectively well. (Here, I mean in learning the model, not as an optimization method for the controller selecting the best actions).

In model-based RL, are there situations where a learning a linear model such as using a Lyapunov function would be better suited than using a neural network, or are the examples of problems framed to use linear models when addressing them using model-based RL?

mugoh
  • 549
  • 4
  • 21

1 Answers1

1

This is just a case of supervised learning. You are trying to predict $s_{t+1}$ given $s_t$ and $a_t$, so the answer to your question depends on how complex your state dynamics are.

For example, if the state space is really complex, e.g. if your state space is an image and you want to predict the next image given the current image and an action, then linear methods are unlikely to work well.

David
  • 5,100
  • 1
  • 11
  • 33