Skip to main content

No project description provided

Project description

Unit-tests Documentation Benchmarks codecov Twitter Follow Python version GitHub license pypi version pypi nightly version Downloads Downloads Discord Shield

TorchRL

Documentation | TensorDict | Features | Examples, tutorials and demos | Citation | Installation | Asking a question | Contributing

TorchRL is an open-source Reinforcement Learning (RL) library for PyTorch.

🚀 What's New

LLM API - Complete Framework for Language Model Fine-tuning

TorchRL now includes a comprehensive LLM API for post-training and fine-tuning of language models! This new framework provides everything you need for RLHF, supervised fine-tuning, and tool-augmented training:

  • 🤖 Unified LLM Wrappers: Seamless integration with Hugging Face models and vLLM inference engines - more to come!
  • 💬 Conversation Management: Advanced History class for multi-turn dialogue with automatic chat template detection
  • 🛠️ Tool Integration: Built-in support for Python code execution, function calling, and custom tool transforms
  • 🎯 Specialized Objectives: GRPO (Group Relative Policy Optimization) and SFT loss functions optimized for language models
  • High-Performance Collectors: Async data collection with distributed training support
  • 🔄 Flexible Environments: Transform-based architecture for reward computation, data loading, and conversation augmentation

The LLM API follows TorchRL's modular design principles, allowing you to mix and match components for your specific use case. Check out the complete documentation and GRPO implementation example to get started!

Quick LLM API Example
from torchrl.envs.llm import ChatEnv
from torchrl.modules.llm import TransformersWrapper
from torchrl.objectives.llm import GRPOLoss
from torchrl.collectors.llm import LLMCollector

# Create environment with Python tool execution
env = ChatEnv(
    tokenizer=tokenizer,
    system_prompt="You are an assistant that can execute Python code.",
    batch_size=[1]
).append_transform(PythonInterpreter())

# Wrap your language model
llm = TransformersWrapper(
    model=model,
    tokenizer=tokenizer,
    input_mode="history"
)

# Set up GRPO training
loss_fn = GRPOLoss(llm, critic, gamma=0.99)
collector = LLMCollector(env, llm, frames_per_batch=100)

# Training loop
for data in collector:
    loss = loss_fn(data)
    loss.backward()
    optimizer.step()

Key features

  • 🐍 Python-first: Designed with Python as the primary language for ease of use and flexibility
  • ⏱️ Efficient: Optimized for performance to support demanding RL research applications
  • 🧮 Modular, customizable, extensible: Highly modular architecture allows for easy swapping, transformation, or creation of new components
  • 📚 Documented: Thorough documentation ensures that users can quickly understand and utilize the library
  • Tested: Rigorously tested to ensure reliability and stability
  • ⚙️ Reusable functionals: Provides a set of highly reusable functions for cost functions, returns, and data processing

Design Principles

  • 🔥 Aligns with PyTorch ecosystem: Follows the structure and conventions of popular PyTorch libraries (e.g., dataset pillar, transforms, models, data utilities)
  • ➖ Minimal dependencies: Only requires Python standard library, NumPy, and PyTorch; optional dependencies for common environment libraries (e.g., OpenAI Gym) and datasets (D4RL, OpenX...)

Read the full paper for a more curated description of the library.

Getting started

Check our Getting Started tutorials for quickly ramp up with the basic features of the library!

Documentation and knowledge base

The TorchRL documentation can be found here. It contains tutorials and the API reference.

TorchRL also provides a RL knowledge base to help you debug your code, or simply learn the basics of RL. Check it out here.

We have some introductory videos for you to get to know the library better, check them out:

Spotlight publications

TorchRL being domain-agnostic, you can use it across many different fields. Here are a few examples:

  • ACEGEN: Reinforcement Learning of Generative Chemical Agents for Drug Discovery
  • BenchMARL: Benchmarking Multi-Agent Reinforcement Learning
  • BricksRL: A Platform for Democratizing Robotics and Reinforcement Learning Research and Education with LEGO
  • OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control
  • RL4CO: an Extensive Reinforcement Learning for Combinatorial Optimization Benchmark
  • Robohive: A unified framework for robot learning

Writing simplified and portable RL codebase with TensorDict

RL algorithms are very heterogeneous, and it can be hard to recycle a codebase across settings (e.g. from online to offline, from state-based to pixel-based learning). TorchRL solves this problem through TensorDict, a convenient data structure(1) that can be used to streamline one's RL codebase. With this tool, one can write a complete PPO training script in less than 100 lines of code!

Code
import torch
from tensordict.nn import TensorDictModule
from tensordict.nn.distributions import NormalParamExtractor
from torch import nn

from torchrl.collectors import SyncDataCollector
from torchrl.data.replay_buffers import TensorDictReplayBuffer, \
  LazyTensorStorage, SamplerWithoutReplacement
from torchrl.envs.libs.gym import GymEnv
from torchrl.modules import ProbabilisticActor, ValueOperator, TanhNormal
from torchrl.objectives import ClipPPOLoss
from torchrl.objectives.value import GAE

env = GymEnv("Pendulum-v1") 
model = TensorDictModule(
  nn.Sequential(
      nn.Linear(3, 128), nn.Tanh(),
      nn.Linear(128, 128), nn.Tanh(),
      nn.Linear(128, 128), nn.Tanh(),
      nn.Linear(128, 2),
      NormalParamExtractor()
  ),
  in_keys=["observation"],
  out_keys=["loc", "scale"]
)
critic = ValueOperator(
  nn.Sequential(
      nn.Linear(3, 128), nn.Tanh(),
      nn.Linear(128, 128), nn.Tanh(),
      nn.Linear(128, 128), nn.Tanh(),
      nn.Linear(128, 1),
  ),
  in_keys=["observation"],
)
actor = ProbabilisticActor(
  model,
  in_keys=["loc", "scale"],
  distribution_class=TanhNormal,
  distribution_kwargs={"low": -1.0, "high": 1.0},
  return_log_prob=True
  )
buffer = TensorDictReplayBuffer(
  storage=LazyTensorStorage(1000),
  sampler=SamplerWithoutReplacement(),
  batch_size=50,
  )
collector = SyncDataCollector(
  env,
  actor,
  frames_per_batch=1000,
  total_frames=1_000_000,
)
loss_fn = ClipPPOLoss(actor, critic)
adv_fn = GAE(value_network=critic, average_gae=True, gamma=0.99, lmbda=0.95)
optim = torch.optim.Adam(loss_fn.parameters(), lr=2e-4)

for data in collector:  # collect data
  for epoch in range(10):
      adv_fn(data)  # compute advantage
      buffer.extend(data)
      for sample in buffer:  # consume data
          loss_vals = loss_fn(sample)
          loss_val = sum(
              value for key, value in loss_vals.items() if
              key.startswith("loss")
              )
          loss_val.backward()
          optim.step()
          optim.zero_grad()
  print(f"avg reward: {data['next', 'reward'].mean().item(): 4.4f}")

Here is an example of how the environment API relies on tensordict to carry data from one function to another during a rollout execution: Alt Text

TensorDict makes it easy to re-use pieces of code across environments, models and algorithms.

Code

For instance, here's how to code a rollout in TorchRL:

- obs, done = env.reset()
+ tensordict = env.reset()
policy = SafeModule(
    model,
    in_keys=["observation_pixels", "observation_vector"],
    out_keys=["action"],
)
out = []
for i in range(n_steps):
-     action, log_prob = policy(obs)
-     next_obs, reward, done, info = env.step(action)
-     out.append((obs, next_obs, action, log_prob, reward, done))
-     obs = next_obs
+     tensordict = policy(tensordict)
+     tensordict = env.step(tensordict)
+     out.append(tensordict)
+     tensordict = step_mdp(tensordict)  # renames next_observation_* keys to observation_*
- obs, next_obs, action, log_prob, reward, done = [torch.stack(vals, 0) for vals in zip(*out)]
+ out = torch.stack(out, 0)  # TensorDict supports multiple tensor operations

Using this, TorchRL abstracts away the input / output signatures of the modules, env, collectors, replay buffers and losses of the library, allowing all primitives to be easily recycled across settings.

Code

Here's another example of an off-policy training loop in TorchRL (assuming that a data collector, a replay buffer, a loss and an optimizer have been instantiated):

- for i, (obs, next_obs, action, hidden_state, reward, done) in enumerate(collector):
+ for i, tensordict in enumerate(collector):
-     replay_buffer.add((obs, next_obs, action, log_prob, reward, done))
+     replay_buffer.add(tensordict)
    for j in range(num_optim_steps):
-         obs, next_obs, action, hidden_state, reward, done = replay_buffer.sample(batch_size)
-         loss = loss_fn(obs, next_obs, action, hidden_state, reward, done)
+         tensordict = replay_buffer.sample(batch_size)
+         loss = loss_fn(tensordict)
        loss.backward()
        optim.step()
        optim.zero_grad()

This training loop can be re-used across algorithms as it makes a minimal number of assumptions about the structure of the data.

TensorDict supports multiple tensor operations on its device and shape (the shape of TensorDict, or its batch size, is the common arbitrary N first dimensions of all its contained tensors):

Code
# stack and cat
tensordict = torch.stack(list_of_tensordicts, 0)
tensordict = torch.cat(list_of_tensordicts, 0)
# reshape
tensordict = tensordict.view(-1)
tensordict = tensordict.permute(0, 2, 1)
tensordict = tensordict.unsqueeze(-1)
tensordict = tensordict.squeeze(-1)
# indexing
tensordict = tensordict[:2]
tensordict[:, 2] = sub_tensordict
# device and memory location
tensordict.cuda()
tensordict.to("cuda:1")
tensordict.share_memory_()

TensorDict comes with a dedicated tensordict.nn module that contains everything you might need to write your model with it. And it is functorch and torch.compile compatible!

Code
transformer_model = nn.Transformer(nhead=16, num_encoder_layers=12)
+ td_module = SafeModule(transformer_model, in_keys=["src", "tgt"], out_keys=["out"])
src = torch.rand((10, 32, 512))
tgt = torch.rand((20, 32, 512))
+ tensordict = TensorDict({"src": src, "tgt": tgt}, batch_size=[20, 32])
- out = transformer_model(src, tgt)
+ td_module(tensordict)
+ out = tensordict["out"]

The TensorDictSequential class allows to branch sequences of nn.Module instances in a highly modular way. For instance, here is an implementation of a transformer using the encoder and decoder blocks:

encoder_module = TransformerEncoder(...)
encoder = TensorDictSequential(encoder_module, in_keys=["src", "src_mask"], out_keys=["memory"])
decoder_module = TransformerDecoder(...)
decoder = TensorDictModule(decoder_module, in_keys=["tgt", "memory"], out_keys=["output"])
transformer = TensorDictSequential(encoder, decoder)
assert transformer.in_keys == ["src", "src_mask", "tgt"]
assert transformer.out_keys == ["memory", "output"]

TensorDictSequential allows to isolate subgraphs by querying a set of desired input / output keys:

transformer.select_subsequence(out_keys=["memory"])  # returns the encoder
transformer.select_subsequence(in_keys=["tgt", "memory"])  # returns the decoder

Check TensorDict tutorials to learn more!

Features

  • A common interface for environments which supports common libraries (OpenAI gym, deepmind control lab, etc.)(1) and state-less execution (e.g. Model-based environments). The batched environments containers allow parallel execution(2). A common PyTorch-first class of tensor-specification class is also provided. TorchRL's environments API is simple but stringent and specific. Check the documentation and tutorial to learn more!

    Code
    env_make = lambda: GymEnv("Pendulum-v1", from_pixels=True)
    env_parallel = ParallelEnv(4, env_make)  # creates 4 envs in parallel
    tensordict = env_parallel.rollout(max_steps=20, policy=None)  # random rollout (no policy given)
    assert tensordict.shape == [4, 20]  # 4 envs, 20 steps rollout
    env_parallel.action_spec.is_in(tensordict["action"])  # spec check returns True
    
  • multiprocess and distributed data collectors(2) that work synchronously or asynchronously. Through the use of TensorDict, TorchRL's training loops are made very similar to regular training loops in supervised learning (although the "dataloader" -- read data collector -- is modified on-the-fly):

    Code
    env_make = lambda: GymEnv("Pendulum-v1", from_pixels=True)
    collector = MultiaSyncDataCollector(
        [env_make, env_make],
        policy=policy,
        devices=["cuda:0", "cuda:0"],
        total_frames=10000,
        frames_per_batch=50,
        ...
    )
    for i, tensordict_data in enumerate(collector):
        loss = loss_module(tensordict_data)
        loss.backward()
        optim.step()
        optim.zero_grad()
        collector.update_policy_weights_()
    

    Check our distributed collector examples to learn more about ultra-fast data collection with TorchRL.

  • efficient(2) and generic(1) replay buffers with modularized storage:

    Code
    storage = LazyMemmapStorage(  # memory-mapped (physical) storage
        cfg.buffer_size,
        scratch_dir="/tmp/"
    )
    buffer = TensorDictPrioritizedReplayBuffer(
        alpha=0.7,
        beta=0.5,
        collate_fn=lambda x: x,
        pin_memory=device != torch.device("cpu"),
        prefetch=10,  # multi-threaded sampling
        storage=storage
    )
    

    Replay buffers are also offered as wrappers around common datasets for offline RL:

    Code
    from torchrl.data.replay_buffers import SamplerWithoutReplacement
    from torchrl.data.datasets.d4rl import D4RLExperienceReplay
    data = D4RLExperienceReplay(
        "maze2d-open-v0",
        split_trajs=True,
        batch_size=128,
        sampler=SamplerWithoutReplacement(drop_last=True),
    )
    for sample in data:  # or alternatively sample = data.sample()
        fun(sample)
    
  • cross-library environment transforms(1), executed on device and in a vectorized fashion(2), which process and prepare the data coming out of the environments to be used by the agent:

    Code
    env_make = lambda: GymEnv("Pendulum-v1", from_pixels=True)
    env_base = ParallelEnv(4, env_make, device="cuda:0")  # creates 4 envs in parallel
    env = TransformedEnv(
        env_base,
        Compose(
            ToTensorImage(),
            ObservationNorm(loc=0.5, scale=1.0)),  # executes the transforms once and on device
    )
    tensordict = env.reset()
    assert tensordict.device == torch.device("cuda:0")
    

    Other transforms include: reward scaling (RewardScaling), shape operations (concatenation of tensors, unsqueezing etc.), concatenation of successive operations (CatFrames), resizing (Resize) and many more.

    Unlike other libraries, the transforms are stacked as a list (and not wrapped in each other), which makes it easy to add and remove them at will:

    env.insert_transform(0, NoopResetEnv())  # inserts the NoopResetEnv transform at the index 0
    

    Nevertheless, transforms can access and execute operations on the parent environment:

    transform = env.transform[1]  # gathers the second transform of the list
    parent_env = transform.parent  # returns the base environment of the second transform, i.e. the base env + the first transform
    
  • various tools for distributed learning (e.g. memory mapped tensors)(2);

  • various architectures and models (e.g. actor-critic)(1):

    Code
    # create an nn.Module
    common_module = ConvNet(
        bias_last_layer=True,
        depth=None,
        num_cells=[32, 64, 64],
        kernel_sizes=[8, 4, 3],
        strides=[4, 2, 1],
    )
    # Wrap it in a SafeModule, indicating what key to read in and where to
    # write out the output
    common_module = SafeModule(
        common_module,
        in_keys=["pixels"],
        out_keys=["hidden"],
    )
    # Wrap the policy module in NormalParamsWrapper, such that the output
    # tensor is split in loc and scale, and scale is mapped onto a positive space
    policy_module = SafeModule(
        NormalParamsWrapper(
            MLP(num_cells=[64, 64], out_features=32, activation=nn.ELU)
        ),
        in_keys=["hidden"],
        out_keys=["loc", "scale"],
    )
    # Use a SafeProbabilisticTensorDictSequential to combine the SafeModule with a
    # SafeProbabilisticModule, indicating how to build the
    # torch.distribution.Distribution object and what to do with it
    policy_module = SafeProbabilisticTensorDictSequential(  # stochastic policy
        policy_module,
        SafeProbabilisticModule(
            in_keys=["loc", "scale"],
            out_keys="action",
            distribution_class=TanhNormal,
        ),
    )
    value_module = MLP(
        num_cells=[64, 64],
        out_features=1,
        activation=nn.ELU,
    )
    # Wrap the policy and value funciton in a common module
    actor_value = ActorValueOperator(common_module, policy_module, value_module)
    # standalone policy from this
    standalone_policy = actor_value.get_policy_operator()
    
  • exploration wrappers and modules to easily swap between exploration and exploitation(1):

    Code
    policy_explore = EGreedyWrapper(policy)
    with set_exploration_type(ExplorationType.RANDOM):
        tensordict = policy_explore(tensordict)  # will use eps-greedy
    with set_exploration_type(ExplorationType.DETERMINISTIC):
        tensordict = policy_explore(tensordict)  # will not use eps-greedy
    
  • A series of efficient loss modules and highly vectorized functional return and advantage computation.

    Code

    Loss modules

    from torchrl.objectives import DQNLoss
    loss_module = DQNLoss(value_network=value_network, gamma=0.99)
    tensordict = replay_buffer.sample(batch_size)
    loss = loss_module(tensordict)
    

    Advantage computation

    from torchrl.objectives.value.functional import vec_td_lambda_return_estimate
    advantage = vec_td_lambda_return_estimate(gamma, lmbda, next_state_value, reward, done, terminated)
    
  • a generic trainer class(1) that executes the aforementioned training loop. Through a hooking mechanism, it also supports any logging or data transformation operation at any given time.

  • various recipes to build models that correspond to the environment being deployed.

  • LLM API: Complete framework for language model fine-tuning with unified wrappers for Hugging Face and vLLM backends, conversation management with automatic chat template detection, tool integration (Python execution, function calling), specialized objectives (GRPO, SFT), and high-performance async collectors. Perfect for RLHF, supervised fine-tuning, and tool-augmented training scenarios.

    Code
    from torchrl.envs.llm import ChatEnv
    from torchrl.modules.llm import TransformersWrapper
    from torchrl.envs.llm.transforms import PythonInterpreter
    
    # Create environment with tool execution
    env = ChatEnv(
        tokenizer=tokenizer,
        system_prompt="You can execute Python code.",
        batch_size=[1]
    ).append_transform(PythonInterpreter())
    
    # Wrap language model for training
    llm = TransformersWrapper(
        model=model,
        tokenizer=tokenizer,
        input_mode="history"
    )
    
    # Multi-turn conversation with tool use
    obs = env.reset(TensorDict({"query": "Calculate 2+2"}, batch_size=[1]))
    llm_output = llm(obs)  # Generates response
    obs = env.step(llm_output)  # Environment processes response
    

If you feel a feature is missing from the library, please submit an issue! If you would like to contribute to new features, check our call for contributions and our contribution page.

Examples, tutorials and demos

A series of State-of-the-Art implementations are provided with an illustrative purpose:

Algorithm Compile Support** Tensordict-free API Modular Losses Continuous and Discrete
DQN 1.9x + NA + (through ActionDiscretizer transform)
DDPG 1.87x + + - (continuous only)
IQL 3.22x + + +
CQL 2.68x + + +
TD3 2.27x + + - (continuous only)
TD3+BC untested + + - (continuous only)
A2C 2.67x + - +
PPO 2.42x + - +
SAC 2.62x + - +
REDQ 2.28x + - - (continuous only)
Dreamer v1 untested + + (different classes) - (continuous only)
Decision Transformers untested + NA - (continuous only)
CrossQ untested + + - (continuous only)
Gail untested + NA +
Impala untested + - +
IQL (MARL) untested + + +
DDPG (MARL) untested + + - (continuous only)
PPO (MARL) untested + - +
QMIX-VDN (MARL) untested + NA +
SAC (MARL) untested + - +
RLHF NA + NA NA
LLM API (GRPO) NA + + NA

** The number indicates expected speed-up compared to eager mode when executed on CPU. Numbers may vary depending on architecture and device.

and many more to come!

Code examples displaying toy code snippets and training scripts are also available

Check the examples directory for more details about handling the various configuration settings.

We also provide tutorials and demos that give a sense of what the library can do.

Citation

If you're using TorchRL, please refer to this BibTeX entry to cite this work:

@misc{bou2023torchrl,
      title={TorchRL: A data-driven decision-making library for PyTorch}, 
      author={Albert Bou and Matteo Bettini and Sebastian Dittert and Vikash Kumar and Shagun Sodhani and Xiaomeng Yang and Gianni De Fabritiis and Vincent Moens},
      year={2023},
      eprint={2306.00577},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

Installation

Create a new virtual environment:

python -m venv torchrl
source torchrl/bin/activate  # On Windows use: venv\Scripts\activate

Or create a conda environment where the packages will be installed.

conda create --name torchrl python=3.9
conda activate torchrl

Install dependencies:

PyTorch

Depending on the use of torchrl that you want to make, you may want to install the latest (nightly) PyTorch release or the latest stable version of PyTorch. See here for a detailed list of commands, including pip3 or other special installation instructions.

TorchRL offers a few pre-defined dependencies such as "torchrl[tests]", "torchrl[atari]" etc.

Torchrl

You can install the latest stable release by using

pip3 install torchrl

This should work on linux (including AArch64 machines), Windows 10 and OsX (Metal chips only). On certain Windows machines (Windows 11), one should build the library locally. This can be done in two ways:

# Install and build locally v0.8.1 of the library without cloning
pip3 install git+https://github.com/pytorch/rl@v0.8.1
# Clone the library and build it locally
git clone https://github.com/pytorch/tensordict
git clone https://github.com/pytorch/rl
pip install -e tensordict
pip install -e rl

Note that tensordict local build requires cmake to be installed via homebrew (MacOS) or another package manager such as apt, apt-get, conda or yum but NOT pip, as well as pip install "pybind11[global]".

One can also build the wheels to distribute to co-workers using

python setup.py bdist_wheel

Your wheels will be stored there ./dist/torchrl<name>.whl and installable via

pip install torchrl<name>.whl

The nightly build can be installed via

pip3 install tensordict-nightly torchrl-nightly

which we currently only ship for Linux machines. Importantly, the nightly builds require the nightly builds of PyTorch too. Also, a local build of torchrl with the nightly build of tensordict may fail - install both nightlies or both local builds but do not mix them.

Disclaimer: As of today, TorchRL is roughly compatible with any pytorch version >= 2.1 and installing it will not directly require a newer version of pytorch to be installed. Indirectly though, tensordict still requires the latest PyTorch to be installed and we are working hard to loosen that requirement. The C++ binaries of TorchRL (mainly for prioritized replay buffers) will only work with PyTorch 2.7.0 and above. Some features (e.g., working with nested jagged tensors) may also be limited with older versions of pytorch. It is recommended to use the latest TorchRL with the latest PyTorch version unless there is a strong reason not to do so.

Optional dependencies

The following libraries can be installed depending on the usage one wants to make of torchrl:

# diverse
pip3 install tqdm tensorboard "hydra-core>=1.1" hydra-submitit-launcher

# rendering
pip3 install "moviepy<2.0.0"

# deepmind control suite
pip3 install dm_control

# gym, atari games
pip3 install "gym[atari]" "gym[accept-rom-license]" pygame

# tests
pip3 install pytest pyyaml pytest-instafail

# tensorboard
pip3 install tensorboard

# wandb
pip3 install wandb

Versioning issues can cause error message of the type undefined symbol and such. For these, refer to the versioning issues document for a complete explanation and proposed workarounds.

Asking a question

If you spot a bug in the library, please raise an issue in this repo.

If you have a more generic question regarding RL in PyTorch, post it on the PyTorch forum.

Contributing

Internal collaborations to torchrl are welcome! Feel free to fork, submit issues and PRs. You can checkout the detailed contribution guide here. As mentioned above, a list of open contributions can be found in here.

Contributors are recommended to install pre-commit hooks (using pre-commit install). pre-commit will check for linting related issues when the code is committed locally. You can disable th check by appending -n to your commit command: git commit -m <commit message> -n

Disclaimer

This library is released as a PyTorch beta feature. BC-breaking changes are likely to happen but they will be introduced with a deprecation warranty after a few release cycles.

License

TorchRL is licensed under the MIT License. See LICENSE for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

torchrl-0.9.1-cp313-cp313t-win_amd64.whl (1.4 MB view details)

Uploaded CPython 3.13tWindows x86-64

torchrl-0.9.1-cp313-cp313t-manylinux_2_28_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.13tmanylinux: glibc 2.28+ x86-64

torchrl-0.9.1-cp313-cp313t-manylinux_2_28_aarch64.whl (1.4 MB view details)

Uploaded CPython 3.13tmanylinux: glibc 2.28+ ARM64

torchrl-0.9.1-cp313-cp313t-macosx_11_0_arm64.whl (1.8 MB view details)

Uploaded CPython 3.13tmacOS 11.0+ ARM64

torchrl-0.9.1-cp313-cp313-win_amd64.whl (1.4 MB view details)

Uploaded CPython 3.13Windows x86-64

torchrl-0.9.1-cp313-cp313-manylinux_2_28_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.28+ x86-64

torchrl-0.9.1-cp313-cp313-manylinux_2_28_aarch64.whl (1.4 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.28+ ARM64

torchrl-0.9.1-cp313-cp313-macosx_11_0_arm64.whl (1.7 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

torchrl-0.9.1-cp312-cp312-win_amd64.whl (1.4 MB view details)

Uploaded CPython 3.12Windows x86-64

torchrl-0.9.1-cp312-cp312-manylinux_2_28_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ x86-64

torchrl-0.9.1-cp312-cp312-manylinux_2_28_aarch64.whl (1.4 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ ARM64

torchrl-0.9.1-cp312-cp312-macosx_11_0_arm64.whl (1.7 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

torchrl-0.9.1-cp311-cp311-win_amd64.whl (1.4 MB view details)

Uploaded CPython 3.11Windows x86-64

torchrl-0.9.1-cp311-cp311-manylinux_2_28_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ x86-64

torchrl-0.9.1-cp311-cp311-manylinux_2_28_aarch64.whl (1.4 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ ARM64

torchrl-0.9.1-cp311-cp311-macosx_11_0_arm64.whl (1.7 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

torchrl-0.9.1-cp310-cp310-win_amd64.whl (1.4 MB view details)

Uploaded CPython 3.10Windows x86-64

torchrl-0.9.1-cp310-cp310-manylinux_2_28_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ x86-64

torchrl-0.9.1-cp310-cp310-manylinux_2_28_aarch64.whl (1.4 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ ARM64

torchrl-0.9.1-cp310-cp310-macosx_11_0_arm64.whl (1.7 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

torchrl-0.9.1-cp39-cp39-win_amd64.whl (1.4 MB view details)

Uploaded CPython 3.9Windows x86-64

torchrl-0.9.1-cp39-cp39-manylinux_2_28_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.28+ x86-64

torchrl-0.9.1-cp39-cp39-manylinux_2_28_aarch64.whl (1.4 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.28+ ARM64

torchrl-0.9.1-cp39-cp39-macosx_11_0_arm64.whl (1.7 MB view details)

Uploaded CPython 3.9macOS 11.0+ ARM64

File details

Details for the file torchrl-0.9.1-cp313-cp313t-win_amd64.whl.

File metadata

  • Download URL: torchrl-0.9.1-cp313-cp313t-win_amd64.whl
  • Upload date:
  • Size: 1.4 MB
  • Tags: CPython 3.13t, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.11.13

File hashes

Hashes for torchrl-0.9.1-cp313-cp313t-win_amd64.whl
Algorithm Hash digest
SHA256 4192a4c882525f3fd9f43049e59aeb28bc965bf122ae49fe6c0eb9628b4a9ee6
MD5 ed9656c9d86a390309f050e24441db58
BLAKE2b-256 4cf107d628ccfd3d9cc89b56a1a16ec3883d524260102e1a1362500636969fc3

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp313-cp313t-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp313-cp313t-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 ea844640b32d368c746954993d033e9104489170cf39de22eb87fc367dc2d9c9
MD5 7635a8155ff9972920cd6e8f0a8d3878
BLAKE2b-256 f13a40cedd62b1a7c6ac328d56c7713e0674455504a3312f94adcb4ffbea22ec

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp313-cp313t-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp313-cp313t-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 35eefa31ebc5af64877fa4eeafd6fe8f49398acdd4a86d6babb06b2436ef7653
MD5 e373ab4a900f68bb078174e8b60a09a5
BLAKE2b-256 315cb3a3443a425f2e7e7589e15e210e01ebceda57b9b6e6b52a1683905a5f57

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp313-cp313t-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp313-cp313t-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 60c91911f4feca6b1cf654aa6140829c48ed0d16f36ed010dba92128fba7dc45
MD5 2efdc30369ce7c8f25f8501cea36bef2
BLAKE2b-256 a4654aa08b7db6487b2b3c78aaee657bb90e776cf57f40d37b9e1128069e90b4

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: torchrl-0.9.1-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 1.4 MB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.11.13

File hashes

Hashes for torchrl-0.9.1-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 faa6bda7fee2404ac1834972c06c6e186e660c709a6bff4e4457642856de0050
MD5 9e7e5e9548ea76242e76504cfc73a4aa
BLAKE2b-256 19944942b0464dbfa962766bf3528dc46b55f6e864faf2a49e8845198bdbf53a

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp313-cp313-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp313-cp313-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 62dcf3839a962c61c963081c5c944f42e09ad43a4fba0b21f41d1b9a57518057
MD5 e90a9261a6712fa068ca6f84375c5512
BLAKE2b-256 98d2b674f85338bae18cb7f2805310a93e69fd82e45e9aaad6af464d2e9c4d41

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp313-cp313-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp313-cp313-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 1544b6754d6572953a2b4e85ae97f6d9ce1298f0b1fd02315c25f9543baa70b5
MD5 f4e581f723ec5e5218a0a0f02bf66e7d
BLAKE2b-256 d1ab9cafc26323103d06ea7775cc4fd8162a734dc1483691b2558f9232a2ef75

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 8852f5d5c0c0706f3cc83b72a4ed2aad65337772c7f1df04673584e5f70d0d0f
MD5 f9bd56f4e51f4b5397b640b071707fe8
BLAKE2b-256 4b75a298c41a293b81371e1aa449269cc9f7ec3855ad4a417feb3afbb8e2a9ce

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: torchrl-0.9.1-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 1.4 MB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.11.13

File hashes

Hashes for torchrl-0.9.1-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 0635f101893fba84069022c72715681da3abc9c184c70f9385cd038958eea504
MD5 780fa659e1311c032542e84a333bb4ca
BLAKE2b-256 1c3f49268936cc1e1607b2e108a648b3434206949ddc5c08021afc2f01b00fe1

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp312-cp312-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp312-cp312-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 8dea600531ccc2b060e34d5b6d729fc9441a6dbbde95ab5fa55fc7659a59ca90
MD5 84044bdd0c54ad58d92748852b4b7c1c
BLAKE2b-256 9b112c4af99db0ad6c18c04720f55528e3e2be50c7cbbbd325c844cb1bae89b2

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp312-cp312-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp312-cp312-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 3ffcc3230eefe80ea1b0584ceb7f7bc66c720ee75ed1b70e442d0b0cf31ccd8f
MD5 1902f826e9abc99bdee6735cf2906bb7
BLAKE2b-256 1bf2d25d94ecd764047d56421d9e270633a4d958d99b45fcc17420043e6e3790

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 4bd8115dcd77f4626c430a17c083df8283a1615a630475bdbe181eb7238f57c3
MD5 4229ea99dfa20fd7e77c324a0b1d4954
BLAKE2b-256 3e4c5a77ac812a865c02069b82fdc9253f3a498235ea092f1399dfbb911ce95c

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: torchrl-0.9.1-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 1.4 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.11.13

File hashes

Hashes for torchrl-0.9.1-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 440efa1356c4c29dcb5720109b02233e274e9375c5bb2b6542157f22b2240b47
MD5 31d65fbe2c7c22a94081710c52020ac0
BLAKE2b-256 b38775209f39025b1d42a2c4b7feb92cf5206d8ec4f6f0f5fadb4ca1ff9bd743

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp311-cp311-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp311-cp311-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 2cb6c207fbdf7a104eca6a01dc4cca511d8a6fbf7b48370f6b7c995e29904765
MD5 d5e6dd954d5ee4deaba83d54ef51b101
BLAKE2b-256 36d0c3ab2c4757e060b3716b6574d57c652b116ff2d18e11d1e7107aea0d3418

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp311-cp311-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp311-cp311-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 5ced4e1fb5dee4854782ea44d3ea4c781aa514d61bd8ef8db7d620db73c5f430
MD5 db34681354edad77774c396e2bce5446
BLAKE2b-256 6dacde8e4ddcf6c77869ea356ef041f060c53518910a950008e11e12a02120e7

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 59f812d158bee68de99147d2e229285495f8a364e4e3fa2ac35be6da78481354
MD5 5fda0674ff1e737c796d717366c3a7dc
BLAKE2b-256 f67dbd98456eec289d5edd6bbfe8e55d4f208fa0ffa16439bfa955b0e410a07a

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: torchrl-0.9.1-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 1.4 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.11.13

File hashes

Hashes for torchrl-0.9.1-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 dd84f9d8dd2dd83250b282049953ed6bad6a3345d74b69c5335f1f6fb62a2711
MD5 db5dfbb4fb8ceeb834b5952bb1075b4a
BLAKE2b-256 40e6f7b61788bd53b5d401895bf6b221cd86c1244f1fb01ae4b51824669d293d

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp310-cp310-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 4d0493f0388dbdbcb7b0ddc1712910cedb3b1a9a24be02de641727fb977123fa
MD5 dda0720e7c17b8a320c6369d8a6394b8
BLAKE2b-256 06965a585506334d514b06b7728970209d932cce71135d68806f92ef41cf2f7c

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp310-cp310-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp310-cp310-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 dfd13306813d8b3f2a069692286beabb486ef58e72f6835e72e7041109034999
MD5 235ece93ad3b0e46dca530d0e66099a6
BLAKE2b-256 fbdbcaa812930115e978008c7b7e926dfc80873353da6e7ee79ab24667911785

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 186044d9d6cf6492667bf863ce377245e5eed3f5d903c210f0dfac7d3626fdb0
MD5 6735a987ef5c4288f3852d1a25df211f
BLAKE2b-256 d6ae50fa43a070ae0badd3700900afb0a5f099f3ada2b6bdd5fb5bccaecfa155

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: torchrl-0.9.1-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 1.4 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.11.13

File hashes

Hashes for torchrl-0.9.1-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 a5ede23031051db3d4428555d781c5cf57946ea5297fefd46f5046e5e4dd0afb
MD5 e97ed093e0ac286301219b070c2bf63e
BLAKE2b-256 86a934cb632090f0ffa14af05f3ccd91945535862b0a0f7351639c1cecf74987

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp39-cp39-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp39-cp39-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 af1e1e57e2deb846211adcfdf33081ef7ee05fb6b197fb6e6f8dcdef2fddc717
MD5 777fafcbec2298e4019822056313a45a
BLAKE2b-256 f44612655921c6b0b2c5dce195846117f27458f17c0014d4e6894543789ce66e

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp39-cp39-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp39-cp39-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 b6bc45c5f8208fc0ebd53865c0a3e61616dac64751d3819d3031278a4ef8b5c2
MD5 2657f0e09d862014275bee4cc689eaf0
BLAKE2b-256 cd39144c413d2e795253ce67cd72950aeda74728644e4c0c848855e6da48cacf

See more details on using hashes here.

File details

Details for the file torchrl-0.9.1-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for torchrl-0.9.1-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 2caa98e528e2142c3083283a557af2a2cc7c87ee1cd6faa56a2c0b97835e7333
MD5 390c6258cbe8ce79cb461f2fbcc6cf3d
BLAKE2b-256 e2d523beb791e46c0ce8a7b1351179b589a40248a152574cbe2dd749d6e45e27

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page