Getting Started


The Autonomous Learning Library requires a recent version of Pytorch (>= 10.3). Additionally, Tensorboard is required in order to enable logging. We recommond installing these through Conda. We also strongly recommend using a machine with a fast GPU (a GTX 970 or better).


The autonomous-learning-library can be installed from PyPi using pip:

pip install autonomous-learning-library

If you don’t have PyTorch or Tensorboard previously installed, you can install them using:

pip install autonomous-learning-library[pytorch]

An alternate approach, that may be useful when following this tutorial, is to instead install by cloning the Github repository:

git clone
cd autonomous-learning-library
pip install -e .["dev"]

If you chose to clone the repository, you can test your installation by running the unit test suite:

make test

This should also tell you if CUDA (the GPU driver) is available.

Running a Preset Agent

The goal of the Autonomous Learning Library is to provide components for building new agents. However, the library also includes a number of “preset” agent configurations for easy benchmarking and comparison, as well as some useful scripts. For example, an A2C agent can be run on Cart-Pole as follows:

all-classic CartPole-v0 ppo

The results will be written to runs/_a2c <id>, where <id> is some some string generated by the library. You can view these results and other information through tensorboard:

tensorboard --logdir runs

By opening your browser to <http://localhost:6006>, you should see a dashboard that looks something like the following (you may need to adjust the “smoothing” parameter):


If you want to compare agents in a nicer, format, you can use the plot script:

all-plot --logdir runs

This should give you a plot similar to the following:


In this plot, each point represents the average of the episodic returns over the last 100 episodes. The shaded region represents the standard deviation over that interval.

Finally, to watch the trained model in action, we provide a watch scripts for each preset module:

all-watch-classic CartPole-v0 "runs/_a2c <id>"

You need to find the <id> by checking the runs directory.

Each of these scripts can be found the scripts directory of the main repository. Be sure to check out the atari and continuous scripts for more fun!