all.experiments

class all.experiments.CometWriter(experiment, agent_name, env_name, loss=True, logdir='runs')

Bases: all.logging.Writer

A Writer object to be used by all.experiments.Experiment. Writes logs using comet.ml Requires an API key to be stored in .comet.config or as an environment variable. Look at https://www.comet.ml/docs/python-sdk/advanced/#python-configuration for more info. :param experiment: The Experiment associated with the Writer object. :type experiment: all.experiments.Experiment :param agent_name: The name of the Agent the Experiment is being performed on :type agent_name: str :param env_name: The name of the environment the Experiment is being performed in :type env_name: str :param loss: Whether or not to log loss/scheduling metrics, or only evaluation and summary metrics. :type loss: bool, optional :param logdir: The directory where run information is stored. :type logdir: str

add_evaluation(name, value, step='frame')

Log the evaluation metric.

Parameters
  • name (str) – The tag to associate with the loss

  • value (number) – The evaluation metric at the current step

  • step (str, optional) – Which step to use (e.g., “frame” or “episode”)

add_loss(name, value, step='frame')

Log the given loss metric at the current step.

Parameters
  • name (str) – The tag to associate with the loss

  • value (number) – The value of the loss at the current step

  • step (str, optional) – Which step to use (e.g., “frame” or “episode”)

add_scalar(name, value, step='frame')

Log an arbitrary scalar.

Parameters
  • name (str) – The tag to associate with the scalar

  • value (number) – The value of the scalar at the current step

  • step (str, optional) – Which step to use (e.g., “frame” or “episode”)

add_schedule(name, value, step='frame')

Log the current value of a hyperparameter according to some schedule.

Parameters
  • name (str) – The tag to associate with the hyperparameter schedule

  • value (number) – The value of the hyperparameter at the current step

  • step (str, optional) – Which step to use (e.g., “frame” or “episode”)

add_summary(name, mean, std, step='frame')

Log a summary statistic.

Parameters
  • name (str) – The tag to associate with the summary statistic

  • mean (float) – The mean of the statistic at the current step

  • std (float) – The standard deviation of the statistic at the current step

  • step (str, optional) – Which step to use (e.g., “frame” or “episode”)

close()

Close the writer and perform any necessary cleanup.

class all.experiments.Experiment(writer, quiet)

Bases: abc.ABC

An Experiment manages the basic train/test loop and logs results.

Parameters
  • ( (writer) – torch.logging.writer:): A Writer object used for logging.

  • quiet (bool) – If False, the Experiment will print information about episode returns to standard out.

close()
abstract property episode

The index of the current training episode

abstract property frame

The index of the current training frame.

save()
abstract test(episodes=100)

Test the agent in eval mode for a certain number of episodes.

Parameters

episodes (int) – The number of test episodes.

Returns

A list of all returns received during testing.

Return type

list(float)

abstract train(frames=inf, episodes=inf)

Train the agent for a certain number of frames or episodes. If both frames and episodes are specified, then the training loop will exit when either condition is satisfied.

Parameters
  • frames (int) – The maximum number of training frames.

  • episodes (bool) – The maximum number of training episodes.

class all.experiments.ExperimentWriter(experiment, agent_name, env_name, loss=True, logdir='runs')

Bases: tensorboardX.writer.SummaryWriter, all.logging.Writer

The default Writer object used by all.experiments.Experiment. Writes logs using tensorboard into the current logdir directory (‘runs’ by default), tagging the run with a combination of the agent name, the commit hash of the current git repo of the working directory (if any), and the current time. Also writes summary statistics into CSV files. :param experiment: The Experiment associated with the Writer object. :type experiment: all.experiments.Experiment :param agent_name: The name of the Agent the Experiment is being performed on :type agent_name: str :param env_name: The name of the environment the Experiment is being performed in :type env_name: str :param loss: Whether or not to log loss/scheduling metrics, or only evaluation and summary metrics. :type loss: bool, optional

add_evaluation(name, value, step='frame')

Log the evaluation metric.

Parameters
  • name (str) – The tag to associate with the loss

  • value (number) – The evaluation metric at the current step

  • step (str, optional) – Which step to use (e.g., “frame” or “episode”)

add_loss(name, value, step='frame')

Log the given loss metric at the current step.

Parameters
  • name (str) – The tag to associate with the loss

  • value (number) – The value of the loss at the current step

  • step (str, optional) – Which step to use (e.g., “frame” or “episode”)

add_scalar(name, value, step='frame')

Log an arbitrary scalar. :param name: The tag to associate with the scalar :type name: str :param value: The value of the scalar at the current step :type value: number :param step: Which step to use (e.g., “frame” or “episode”) :type step: str, optional

add_schedule(name, value, step='frame')

Log the current value of a hyperparameter according to some schedule.

Parameters
  • name (str) – The tag to associate with the hyperparameter schedule

  • value (number) – The value of the hyperparameter at the current step

  • step (str, optional) – Which step to use (e.g., “frame” or “episode”)

add_summary(name, mean, std, step='frame')

Log a summary statistic.

Parameters
  • name (str) – The tag to associate with the summary statistic

  • mean (float) – The mean of the statistic at the current step

  • std (float) – The standard deviation of the statistic at the current step

  • step (str, optional) – Which step to use (e.g., “frame” or “episode”)

close()

Close the current SummaryWriter. This call flushes the unfinished write operation. Use context manager (with statement) whenever it’s possible.

class all.experiments.MultiagentEnvExperiment(preset, env, logdir='runs', name=None, quiet=False, render=False, save_freq=100, train_steps=inf, write_loss=True, writer='tensorboard')

Bases: object

An Experiment object for training and testing Multiagents.

Parameters
  • preset (all.presets.Preset) – A Multiagent preset.

  • env (all.environments.MultiagentEnvironment) – A multiagent environment.

  • log_dir (str, optional) – The directory in which to save the logs and model.

  • name (str, optional) – The name of the experiment.

  • quiet (bool, optional) – Whether or not to print training information.

  • render (bool, optional) – Whether or not to render during training.

  • save_freq (int, optional) – How often to save the model to disk.

  • train_steps (int, optional) – The number of steps for which to train.

  • write_loss (bool, optional) – Whether or not to log advanced loss information.

property episode
property frame
test(episodes=100)
train(frames=inf, episodes=inf)
class all.experiments.ParallelEnvExperiment(preset, env, name=None, train_steps=inf, logdir='runs', quiet=False, render=False, write_loss=True, writer='tensorboard')

Bases: all.experiments.experiment.Experiment

An Experiment object for training and testing agents that use parallel training environments.

property episode

The index of the current training episode

property frame

The index of the current training frame.

test(episodes=100)

Test the agent in eval mode for a certain number of episodes.

Parameters

episodes (int) – The number of test episodes.

Returns

A list of all returns received during testing.

Return type

list(float)

train(frames=inf, episodes=inf)

Train the agent for a certain number of frames or episodes. If both frames and episodes are specified, then the training loop will exit when either condition is satisfied.

Parameters
  • frames (int) – The maximum number of training frames.

  • episodes (bool) – The maximum number of training episodes.

class all.experiments.SingleEnvExperiment(preset, env, name=None, train_steps=inf, logdir='runs', quiet=False, render=False, write_loss=True, writer='tensorboard')

Bases: all.experiments.experiment.Experiment

An Experiment object for training and testing agents that interact with one environment at a time.

property episode

The index of the current training episode

property frame

The index of the current training frame.

test(episodes=100)

Test the agent in eval mode for a certain number of episodes.

Parameters

episodes (int) – The number of test episodes.

Returns

A list of all returns received during testing.

Return type

list(float)

train(frames=inf, episodes=inf)

Train the agent for a certain number of frames or episodes. If both frames and episodes are specified, then the training loop will exit when either condition is satisfied.

Parameters
  • frames (int) – The maximum number of training frames.

  • episodes (bool) – The maximum number of training episodes.

class all.experiments.SlurmExperiment(agents, envs, frames, test_episodes=100, write_loss=False, job_name='autonomous-learning-library', script_name='experiment.sh', outdir='out', logdir='runs', sbatch_args=None)

Bases: object

create_sbatch_script()
make_output_directory()
parse_args()
queue_jobs()
run_experiment()
run_sbatch_script()
all.experiments.load_and_watch(filename, env, fps=60)
all.experiments.run_experiment(agents, envs, frames, logdir='runs', quiet=False, render=False, test_episodes=100, write_loss=True, writer='tensorboard')
all.experiments.watch(agent, env, fps=60)