4

Youtube was recently suggesting to me videos of people training NEAT neural networks for video games. I've noticed that often the training process was quite slow (for example in this Trackmania example).

Is there a way (algorithmic approach or an idea) to easily simulate video games, without actually rendering the pixels on screen and making the training much quicker? In addition to that, if you also know of a tool that does that, please, share it with me.

Update:

Following the Trackmania example, I found out the youtuber uses a standard tool called "TMInterface", and the webpage of the tool states it is a "TAS tool" - Tool-assisted speedrun.

I'll investigate how such tools work and whats the idea behind them (and if indeed they do what I think they do). Will update!

nbro
  • 42,615
  • 12
  • 119
  • 217
snatchysquid
  • 89
  • 6

2 Answers2

7

It depends on the game environment and on the model being trained.

  • If you are training an agent that uses vision to decide action, then typically you need a copy of the rendered screen:
    • If that comes from a modern game using GPU acceleration to generate output, then you will likely need to leave that as is, and also transfer the rendered output back to the main memory, so it will likely be slower than running the game normally. There may be game settings that help improve speed.
    • If the output is from a game emulator for older games (such as Atari), then it probably won't use a GPU, and you will not need to spend time rendering to the screen. The emulator may be able to run in a mode that doesn't render, whilst still giving in-memory access to the screen buffer. It may also be able to run faster than real time. It will depend on emulator configuration settings that are available to you.
  • If the game is written "natively" in a simulator or semi-native environment like Pygame or Box2D, you should be able to disable rendering and save the time. This is how a lot of OpenAI Gym's environments are set up, and these do run a lot faster when not rendering - a simple flag you can set when starting the environment and/or starting an episode. You may even notice with these environments that putting the view window for the environment behind another window will speed things up considerably (but for automation it is more reliable to disable rendering).

When the environments have been set up via a shared library, like Open AI's Gym, then the internal logic for deciding rendering (or not) will be different for each environment, but there may be a standardised config/method argument to determine whether the game engine is rendered to screen. That could apply to nearly all the environments available in the library. You should check the documentation if for example you are trying out NEAT on some Atari games that have been ready packaged for use with computer agents.

nbro
  • 42,615
  • 12
  • 119
  • 217
Neil Slater
  • 33,739
  • 3
  • 47
  • 66
3

Bypassing graphics

As mentioned in Neil Slater's answer, manipulating the engine to bypass graphics rendering to speed up AI simulation can be a valid approach. I have done that myself.

But there are some caveats to consider/work around:

  • The graphics subsystem can be used for core gameplay programming as well. For example, the picking problem (Which object in the scene is the user selecting/clicking on?) can be solved by rendering and evaluating framebuffers.
  • If the game has an FPS limiter, then you need to raise that FPS limit. Otherwise it quickly becomes the new bottleneck.
  • If the game also has real-time timers (i.e. wait five seconds to trigger some event), then your ultra-fast gameplay simulation can cause unintended effects/bugs. This is especially true if you break the aforementioned FPS limit.
  • Sometimes games can experience bugs and crashes when running at high framerates. (Take this PCGamingWiki entry on Peter Jackson's King Kong Gamer's Edition as an example.) With high framerates, gameplay simulation may operate with very small timesteps and glitches in physics and collision detection become more likely. Such effects will only intensify with ultra-high framerates from the accelerated AI simulation.

DLL injection

It is not uncommon for game mods to hijack or replace DLLs, including those of the graphics API. If you cannot manipulate the engine itself, then perhaps you can inject an alternative graphics API implementation that consists of stubs and mockups.

Bypass whatever is irrelevant to gameplay

Depending on the game and the goals of your project, you might take even more drastic steps to accelerate the simulation:

  • Bypass the audio layer as well. Besides graphics, audio is another major output channel that is often irrelevant to AI simulation.
  • Skip through cutscenes and dialogue, like a speedrunner would do.
  • If walking is not essential to gameplay, let the avatar teleport instead of simulating all the walking.
  • Skip animations in general. Always try to fast-forward to the "gameplay outcome" of an action.
meekee7
  • 31
  • 1