61 lines
1.7 KiB
Markdown
61 lines
1.7 KiB
Markdown
|
# Dynamic Bottleneck
|
||
|
|
|
||
|
|
## Introduction
|
||
|
|
|
||
|
|
This is a TensorFlow based implementation for our paper on
|
||
|
|
|
||
|
|
**"Dynamic Bottleneck for Robust Self-Supervised Exploration". NeurIPS 2021**
|
||
|
|
|
||
|
|
## Prerequisites
|
||
|
|
|
||
|
|
python3.6 or 3.7,
|
||
|
|
tensorflow-gpu 1.x, tensorflow-probability,
|
||
|
|
openAI [baselines](https://github.com/openai/baselines),
|
||
|
|
openAI [Gym](http://gym.openai.com/)
|
||
|
|
|
||
|
|
## Installation and Usage
|
||
|
|
|
||
|
|
### Atari games
|
||
|
|
|
||
|
|
The following command should train a pure exploration
|
||
|
|
agent on "Breakout" with default experiment parameters.
|
||
|
|
|
||
|
|
```
|
||
|
|
python run.py --env BreakoutNoFrameskip-v4
|
||
|
|
```
|
||
|
|
|
||
|
|
|
||
|
|
### Atari games with Random-Box noise
|
||
|
|
|
||
|
|
The following command should train a pure exploration
|
||
|
|
agent on "Breakout" with randomBox noise.
|
||
|
|
|
||
|
|
```
|
||
|
|
python run.py --env BreakoutNoFrameskip-v4 --randomBoxNoise
|
||
|
|
```
|
||
|
|
|
||
|
|
### Atari games with Gaussian noise
|
||
|
|
|
||
|
|
The following command should train a pure exploration
|
||
|
|
agent on "Breakout" with Gaussian noise.
|
||
|
|
|
||
|
|
```
|
||
|
|
python run.py --env BreakoutNoFrameskip-v4 --pixelNoise
|
||
|
|
```
|
||
|
|
|
||
|
|
|
||
|
|
### Atari games with sticky actions
|
||
|
|
|
||
|
|
The following command should train a pure exploration
|
||
|
|
agent on "sticky Breakout" with a probability of 0.25
|
||
|
|
|
||
|
|
```
|
||
|
|
python run.py --env BreakoutNoFrameskip-v4 --stickyAtari
|
||
|
|
```
|
||
|
|
|
||
|
|
### Baselines
|
||
|
|
|
||
|
|
- **ICM**: We use the official [code](https://github.com/openai/large-scale-curiosity) of "Curiosity-driven Exploration by Self-supervised Prediction, ICML 2017" and "Large-Scale Study of Curiosity-Driven Learning, ICLR 2019".
|
||
|
|
- **Disagreement**: We use the official [code](https://github.com/pathak22/exploration-by-disagreement) of "Self-Supervised Exploration via Disagreement, ICML 2019".
|
||
|
|
- **CB**: We use the official [code](https://github.com/whyjay/curiosity-bottleneck) of "Curiosity-Bottleneck: Exploration by Distilling Task-Specific Novelty, ICML 2019".
|