synthetic-gymnax 0.0.1

Synthetic gymnax environments

Synthetic Gymnax

💡 Make a one-line change ...

Simply replace	by
import gymnax env, params = gymnax.make("CartPole-v1") ... # your training code	import gymnax, synthetic_gymnax env, params = gymnax.make("Synthetic-CartPole-v1") # add 'synthetic' to env: ^^^^^^^^^^ ... # your training code

💨 ... and enjoy fast training.

The synthetic environments are meta-learned to train agents within 10k time steps. This can be much faster than training in the real environment, even when using tuned hyperparameters!

• 🟩 Real environment training, using tuned hyperparameters (IQM of 5 training runs)
• 🟦 Synthetic environment training, using any reasonable hyperparameters (IQM performance of 20 training runs with random HP configurations)

🏅 Performance of agents after training for 10k synthetic steps

Classic control: 10k synthetic 🦶
Environment	PPO	SAC	DQN	DDPG	TD3
Synthetic-Acrobot-v1	-84.1	-85.3	-82.6	-	-
Synthetic-CartPole-v1	500.0	500.0	500.0	-	-
Synthetic-Mountaincar-v0	-181.8	-170.1	-118.4	-	-
Synthetic-CountinuousMountainCar-v0	66.9	91.1	-	97.6	97.5
Synthetic-Pendulum-v1	-205.4	-188.3	-	-164.3	-168.5

Brax: 10k synthetic, 5m real 🦶
Environment	PPO	SAC	DDPG	TD3
Synthetic	Real	Synthetic	Real	Synthetic	Real	Synthetic	Real
halfcheetah	1657.4	3487.1	5810.4	7735.5	6162.4	3263.3	6555.8	13213.5
hopper	853.5	2521.9	2738.8	3119.4	3012.4	1536.0	2985.3	3325.8
humanoidstandup	13356.1	17243.5	21105.2	23808.1	21039.0	24944.8	20372.0	28376.2
swimmer	348.5	83.6	361.6	124.8	365.1	348.5	365.4	232.2
walker2d	858.3	2039.6	1323.1	4140.1	1304.3	698.3	1321.8	4605.8

💫Replicating our results

We provide the configurations used in meta-training the checkpoints for synthetic environments in synthetic_gymnax/checkpoints/*environment*/config.yaml. They can be used with the meta-learning script by calling e.g.

python examples/metalearn_synthenv.py --config synthetic_gymnax/checkpoints/hopper/config.yaml

Please note that when installing via pip, the configs are not bundled with the package. Please clone the repository to get them.

✍ Citing and more information

If you use the provided synthetic environments in your work, please cite us as

@article{
  ...
}