Skip to main content

Environment overview

What we are going to show you in this tutorial is how, given a game developed in python, wrap it in a gym environment and create an AI model able to play the game using the EzSpark distributed training ecosystem.

What is OpenAI Gym?​

OpenAI Gym is an open source Python library for developing and comparing reinforcement learning algorithms. Is supported by Pyezspark and is considered a standard to develop and test RL algorithms.

Project structure​

You can implement a Gym environment following the OpenAI Gym's standard tutorial

You can download an already-implemented gym environment from our Github or clone it

git clone https://github.com/ez-spark/gym-mytetris.git

The project has a structure like this:

- game-folder
      |
      |
      |
      ---- setup.py
      |
      ---- game_sub_folder
      |           |
      |           |
      |           |
      |           ---- __init__.py
      |           |
      |           ---- envs
      |           |     |
      |           |     |
      |           |     |
      |           |     ---- __init__.py
      |           |     |
      |           |     |
      |           |     ---- game.py

Here a video tutorial about the structure:

The structure you must follow is written in the structure.txt file of the cloned repo. We have a game folder that is the parent folder of everything related to the environment. The game-folder in this case is the gym-mytetris folder. Inside this folder we have another file: the setup.py file. The setup.py file is needed for you to install the environment locally using pip. Then we have our game sub folder, in the case it is named gym_mytetris. Inside of this subfolder, we have a file named init.py file. The init.py file is needed because you're gonna register the name of your game: import register from gym.envs.registration and then you're gonna register the name of your game and set the entry point. The entry point in this case is given by the folder you are in. Finally, in the envs directory you expose the main class of your environment that will be written inside game.py

EzSpark-Gym compliance​

EzSpark actually supports only environments with a discrete action space

EzSpark needed functions​

# @action:= int value
# must return: obs, reward, done, info
def step(self, action)

The step function must take a single integer value (action) which can range from 0 to n-1 where n is the number of all possible actions that can be taken. It must return 4 values:

  • obs (list of values or numpy array)
  • reward (value)
  • done (boolean)
  • info (whatever)
# must return: obs
def reset(self)

It must return 1 value:

  • obs (list of values or numpy array)

Here a tutorial Video

After you've created your gym environment you must install it locally

pip install -e game-folder/