How can imitation learning data be collected?

Question

How can imitation learning data be collected? Can I use a neural network for that? It might be noisy. Should I use manual gathering?

Answer 1

Imitation learning data usually means data gathered from an expert, that is data from an agent proficient in the task.

The agent may be:

• A human operator: have the operator complete the task and record inputs and action taken.
• A pre-trained reinforcement learning agent. Same as above.

The collected data is exactly the same for both cases.

The hard part is usually building an interface to collect this data, which is very task specific. For instance imitation learning for robotics may require expensive sensors. For cartpole and similar RL environments have a look at https://github.com/HumanCompatibleAI/imitation, they have scripts to learn from RL agents.