Async Gym Agents
Drop-in asynchronous data collection for Stable Baselines 3 agents.
Known framework callbacks are processed from complete episode batches when their timing permits. Off-policy checkpointing and unrecognized Stable Baselines callbacks retain their normal per-step behavior.
Usage
from functools import partial
import gymnasium as gym
from stable_baselines3 import TD3
from async_gym_agents.agents.async_agent import get_injected_agent
from async_gym_agents.envs.multi_env import IndexableMultiEnv
env = IndexableMultiEnv(
[partial(gym.make, "Pendulum-v1") for _ in range(8)]
)
model = get_injected_agent(TD3)(
"MlpPolicy",
env,
use_mp=False,
max_episodes_in_buffer=8,
)
model.learn(total_timesteps=10)
model.shutdown()
Workers send complete episodes, so on-policy rollouts may exceed n_steps by
the final episode. max_episodes_in_buffer limits buffered episodes across all
workers. queue_put_timeout defaults to None; set a finite timeout to allow
episode drops instead of waiting for buffer capacity.
Episodes are decoded in a background assembler. On-policy agents prepare the next rollout buffer, while off-policy agents prepare one episode ahead and keep replay-buffer insertion on the trainer thread.
On-policy episode packets retain both the environment reward and the time-limit-bootstrapped training reward. Stable Baselines callbacks, batched logging, and pruning receive the environment reward; rollout-buffer returns and advantages use the training reward. Off-policy episodes retain their existing single reward view.
Use get_profiler_report() to inspect trainer, worker, buffer, transport, and
policy statistics. transport.utilization is the current fraction of bounded
episode slots in use. Episode-send backpressure is reported once as
buffer.avg_push_wait_seconds; the buffer section does not contain historical
queue utilization or emptiness estimates.
2 maintainers
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file async_gym_agents-0.4.0.tar.gz.
File metadata
- Download URL: async_gym_agents-0.4.0.tar.gz
- Upload date:
- Size: 32.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
poetry/2.1.3 CPython/3.10.8 Windows/10
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
7f4af3137ad47704c36a64d0d1bee08f9f899b89e6b61581536bed5b929f9106
|
|
| MD5 |
a08a75033827574e21b719450c198999
|
|
| BLAKE2b-256 |
e40510c152a0dac10fe6b90ae74acc53a1728a04fc0230d2013f6c0fe34bc620
|
File details
Details for the file async_gym_agents-0.4.0-py3-none-any.whl.
File metadata
- Download URL: async_gym_agents-0.4.0-py3-none-any.whl
- Upload date:
- Size: 44.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
poetry/2.1.3 CPython/3.10.8 Windows/10
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
59398290aa01a6fe84579663cc95ad5bd51a29548d456ae3859ff7eb935236d0
|
|
| MD5 |
e537cff7b7268cbab4d3d9b2ae66f328
|
|
| BLAKE2b-256 |
55ea46495664617f262baa4905d6046e541af2270bb81258521d954a0a36df11
|
