TRL-ENV
ENV
TRL is a convenient library to train large language model (LLM) using reinforcement learning (RL). However, it is still too new, the interface is not well-developed yet. rollout_func is a low-level interface to write your own rollout for RL and environment_factory is a high-level interface to train your model with external environemnt, however, how it parse the model output for tool use is hard coded and not available for every model (check add_response_schema in trl/trainer/grpo_trainer.py and trl/chat_template_utils.py)
TRL-ENV addresses the middle-level with a very simple environment interface
type Action = str
type Delta = str
type Seed = str
class Env(Protocol):
reward: float
alive: bool
def reset(self, seed: Seed) -> tuple[Env, Delta]: ...
def step(self, action: Action) -> tuple[Env, Delta]: ...
It is similar to tool call if not the same. Note that, rollout_func is an experimental feature of TRL, this library is subject to break at anytime
It is important to note that, batch_rollout assumes the additivity of tokenizer, that is
tok(a ++ b) = tok(a) ++ tok(b)
where a and b are texts and ++ is concatenation. This is because Env interacts with LLM via text, not sequence of tokens, and as far as my knowledge, this is unavoidable.
PROCESSOR
transformers despite after 8 years of development (as of 2026) is still not stable. For example, not all models has Tokenizer.parse_response which should be a basic function that must be implemented from the beginning. TRL-ENV requires Tokenizer.parse_response to be existed by Processor interface
Language = str
class Processor(Protocol):
def init_system_input(self, prompt: Language) -> str: ...
def append_user_input(self, prompt: Language) -> str: ...
def parse_agent_output(self, completion: Language) -> tuple[str, str]: ...
EXAMPLES
TRL-ENV provides a very simple example for training agentic LLM. See experiment/examples
RESOURCES
INSTALL PLATFORM DEPENDENT PACKAGES
uv pip install vllm==0.12.0 flash-attn==2.8.3 --torch-backend=cu126 --no-build-isolation
INSTALL VLLM FOR MACOS
git clone https://github.com/vllm-project/vllm.git
cd vllm
uv pip install -r requirements/cpu.txt --index-strategy unsafe-best-match
uv pip install -e .
ROLLOUT_FUNC EXAMPLES FROM TRL
1 maintainer
Metadata
1 maintainer
Release files for trl-env 0.1.20
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| trl_env-0.1.20.tar.gz | 33.4 MB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| trl_env-0.1.20-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 33.4 MB
Release files / trl_env-0.1.20.tar.gz
| Download URL | trl_env-0.1.20.tar.gz |
|---|---|
| Size | 33.4 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
8f777a70e4afa1b613a2f6839c028e609b2cbcf75031fc2372d00d5cb04095e8
|
|
BLAKE2b-256 checksum How to use checksums |
fe214484652942b3a7170bed43b28991272c22cb0e54babb95d9fe8022219a90
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.7 {"installer":{"name":"uv","version":"0.11.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / trl_env-0.1.20-py3-none-any.whl
| Download URL | trl_env-0.1.20-py3-none-any.whl |
|---|---|
| Size | 12.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
37aa7f6b30ffaa79313a52c3c6ddcff5bfc47f79f6c15c1fcd2b207e9ad0e1d2
|
|
BLAKE2b-256 checksum How to use checksums |
d5efde1faf17e36f678aeac0724db21b523bfea4b146f7c76123678b2c4e6530
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.7 {"installer":{"name":"uv","version":"0.11.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for trl-env 0.1.20
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
