Skip to main content

No project description provided

Project description

Unit-tests Documentation Benchmarks codecov Twitter Follow Python version GitHub license pypi version pypi nightly version Downloads Downloads Discord Shield

TorchRL

Documentation | TensorDict | Features | Examples, tutorials and demos | Citation | Installation | Asking a question | Contributing

TorchRL is an open-source Reinforcement Learning (RL) library for PyTorch.

Key features

  • 🐍 Python-first: Designed with Python as the primary language for ease of use and flexibility
  • ⏱️ Efficient: Optimized for performance to support demanding RL research applications
  • 🧮 Modular, customizable, extensible: Highly modular architecture allows for easy swapping, transformation, or creation of new components
  • 📚 Documented: Thorough documentation ensures that users can quickly understand and utilize the library
  • Tested: Rigorously tested to ensure reliability and stability
  • ⚙️ Reusable functionals: Provides a set of highly reusable functions for cost functions, returns, and data processing

Design Principles

  • 🔥 Aligns with PyTorch ecosystem: Follows the structure and conventions of popular PyTorch libraries (e.g., dataset pillar, transforms, models, data utilities)
  • ➖ Minimal dependencies: Only requires Python standard library, NumPy, and PyTorch; optional dependencies for common environment libraries (e.g., OpenAI Gym) and datasets (D4RL, OpenX...)

Read the full paper for a more curated description of the library.

Getting started

Check our Getting Started tutorials for quickly ramp up with the basic features of the library!

Documentation and knowledge base

The TorchRL documentation can be found here. It contains tutorials and the API reference.

TorchRL also provides a RL knowledge base to help you debug your code, or simply learn the basics of RL. Check it out here.

We have some introductory videos for you to get to know the library better, check them out:

Spotlight publications

TorchRL being domain-agnostic, you can use it across many different fields. Here are a few examples:

  • ACEGEN: Reinforcement Learning of Generative Chemical Agents for Drug Discovery
  • BenchMARL: Benchmarking Multi-Agent Reinforcement Learning
  • BricksRL: A Platform for Democratizing Robotics and Reinforcement Learning Research and Education with LEGO
  • OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control
  • RL4CO: an Extensive Reinforcement Learning for Combinatorial Optimization Benchmark
  • Robohive: A unified framework for robot learning

Writing simplified and portable RL codebase with TensorDict

RL algorithms are very heterogeneous, and it can be hard to recycle a codebase across settings (e.g. from online to offline, from state-based to pixel-based learning). TorchRL solves this problem through TensorDict, a convenient data structure(1) that can be used to streamline one's RL codebase. With this tool, one can write a complete PPO training script in less than 100 lines of code!

Code
import torch
from tensordict.nn import TensorDictModule
from tensordict.nn.distributions import NormalParamExtractor
from torch import nn

from torchrl.collectors import SyncDataCollector
from torchrl.data.replay_buffers import TensorDictReplayBuffer, \
  LazyTensorStorage, SamplerWithoutReplacement
from torchrl.envs.libs.gym import GymEnv
from torchrl.modules import ProbabilisticActor, ValueOperator, TanhNormal
from torchrl.objectives import ClipPPOLoss
from torchrl.objectives.value import GAE

env = GymEnv("Pendulum-v1") 
model = TensorDictModule(
  nn.Sequential(
      nn.Linear(3, 128), nn.Tanh(),
      nn.Linear(128, 128), nn.Tanh(),
      nn.Linear(128, 128), nn.Tanh(),
      nn.Linear(128, 2),
      NormalParamExtractor()
  ),
  in_keys=["observation"],
  out_keys=["loc", "scale"]
)
critic = ValueOperator(
  nn.Sequential(
      nn.Linear(3, 128), nn.Tanh(),
      nn.Linear(128, 128), nn.Tanh(),
      nn.Linear(128, 128), nn.Tanh(),
      nn.Linear(128, 1),
  ),
  in_keys=["observation"],
)
actor = ProbabilisticActor(
  model,
  in_keys=["loc", "scale"],
  distribution_class=TanhNormal,
  distribution_kwargs={"low": -1.0, "high": 1.0},
  return_log_prob=True
  )
buffer = TensorDictReplayBuffer(
  storage=LazyTensorStorage(1000),
  sampler=SamplerWithoutReplacement(),
  batch_size=50,
  )
collector = SyncDataCollector(
  env,
  actor,
  frames_per_batch=1000,
  total_frames=1_000_000,
)
loss_fn = ClipPPOLoss(actor, critic)
adv_fn = GAE(value_network=critic, average_gae=True, gamma=0.99, lmbda=0.95)
optim = torch.optim.Adam(loss_fn.parameters(), lr=2e-4)

for data in collector:  # collect data
  for epoch in range(10):
      adv_fn(data)  # compute advantage
      buffer.extend(data)
      for sample in buffer:  # consume data
          loss_vals = loss_fn(sample)
          loss_val = sum(
              value for key, value in loss_vals.items() if
              key.startswith("loss")
              )
          loss_val.backward()
          optim.step()
          optim.zero_grad()
  print(f"avg reward: {data['next', 'reward'].mean().item(): 4.4f}")

Here is an example of how the environment API relies on tensordict to carry data from one function to another during a rollout execution: Alt Text

TensorDict makes it easy to re-use pieces of code across environments, models and algorithms.

Code

For instance, here's how to code a rollout in TorchRL:

- obs, done = env.reset()
+ tensordict = env.reset()
policy = SafeModule(
    model,
    in_keys=["observation_pixels", "observation_vector"],
    out_keys=["action"],
)
out = []
for i in range(n_steps):
-     action, log_prob = policy(obs)
-     next_obs, reward, done, info = env.step(action)
-     out.append((obs, next_obs, action, log_prob, reward, done))
-     obs = next_obs
+     tensordict = policy(tensordict)
+     tensordict = env.step(tensordict)
+     out.append(tensordict)
+     tensordict = step_mdp(tensordict)  # renames next_observation_* keys to observation_*
- obs, next_obs, action, log_prob, reward, done = [torch.stack(vals, 0) for vals in zip(*out)]
+ out = torch.stack(out, 0)  # TensorDict supports multiple tensor operations

Using this, TorchRL abstracts away the input / output signatures of the modules, env, collectors, replay buffers and losses of the library, allowing all primitives to be easily recycled across settings.

Code

Here's another example of an off-policy training loop in TorchRL (assuming that a data collector, a replay buffer, a loss and an optimizer have been instantiated):

- for i, (obs, next_obs, action, hidden_state, reward, done) in enumerate(collector):
+ for i, tensordict in enumerate(collector):
-     replay_buffer.add((obs, next_obs, action, log_prob, reward, done))
+     replay_buffer.add(tensordict)
    for j in range(num_optim_steps):
-         obs, next_obs, action, hidden_state, reward, done = replay_buffer.sample(batch_size)
-         loss = loss_fn(obs, next_obs, action, hidden_state, reward, done)
+         tensordict = replay_buffer.sample(batch_size)
+         loss = loss_fn(tensordict)
        loss.backward()
        optim.step()
        optim.zero_grad()

This training loop can be re-used across algorithms as it makes a minimal number of assumptions about the structure of the data.

TensorDict supports multiple tensor operations on its device and shape (the shape of TensorDict, or its batch size, is the common arbitrary N first dimensions of all its contained tensors):

Code
# stack and cat
tensordict = torch.stack(list_of_tensordicts, 0)
tensordict = torch.cat(list_of_tensordicts, 0)
# reshape
tensordict = tensordict.view(-1)
tensordict = tensordict.permute(0, 2, 1)
tensordict = tensordict.unsqueeze(-1)
tensordict = tensordict.squeeze(-1)
# indexing
tensordict = tensordict[:2]
tensordict[:, 2] = sub_tensordict
# device and memory location
tensordict.cuda()
tensordict.to("cuda:1")
tensordict.share_memory_()

TensorDict comes with a dedicated tensordict.nn module that contains everything you might need to write your model with it. And it is functorch and torch.compile compatible!

Code
transformer_model = nn.Transformer(nhead=16, num_encoder_layers=12)
+ td_module = SafeModule(transformer_model, in_keys=["src", "tgt"], out_keys=["out"])
src = torch.rand((10, 32, 512))
tgt = torch.rand((20, 32, 512))
+ tensordict = TensorDict({"src": src, "tgt": tgt}, batch_size=[20, 32])
- out = transformer_model(src, tgt)
+ td_module(tensordict)
+ out = tensordict["out"]

The TensorDictSequential class allows to branch sequences of nn.Module instances in a highly modular way. For instance, here is an implementation of a transformer using the encoder and decoder blocks:

encoder_module = TransformerEncoder(...)
encoder = TensorDictSequential(encoder_module, in_keys=["src", "src_mask"], out_keys=["memory"])
decoder_module = TransformerDecoder(...)
decoder = TensorDictModule(decoder_module, in_keys=["tgt", "memory"], out_keys=["output"])
transformer = TensorDictSequential(encoder, decoder)
assert transformer.in_keys == ["src", "src_mask", "tgt"]
assert transformer.out_keys == ["memory", "output"]

TensorDictSequential allows to isolate subgraphs by querying a set of desired input / output keys:

transformer.select_subsequence(out_keys=["memory"])  # returns the encoder
transformer.select_subsequence(in_keys=["tgt", "memory"])  # returns the decoder

Check TensorDict tutorials to learn more!

Features

  • A common interface for environments which supports common libraries (OpenAI gym, deepmind control lab, etc.)(1) and state-less execution (e.g. Model-based environments). The batched environments containers allow parallel execution(2). A common PyTorch-first class of tensor-specification class is also provided. TorchRL's environments API is simple but stringent and specific. Check the documentation and tutorial to learn more!

    Code
    env_make = lambda: GymEnv("Pendulum-v1", from_pixels=True)
    env_parallel = ParallelEnv(4, env_make)  # creates 4 envs in parallel
    tensordict = env_parallel.rollout(max_steps=20, policy=None)  # random rollout (no policy given)
    assert tensordict.shape == [4, 20]  # 4 envs, 20 steps rollout
    env_parallel.action_spec.is_in(tensordict["action"])  # spec check returns True
    
  • multiprocess and distributed data collectors(2) that work synchronously or asynchronously. Through the use of TensorDict, TorchRL's training loops are made very similar to regular training loops in supervised learning (although the "dataloader" -- read data collector -- is modified on-the-fly):

    Code
    env_make = lambda: GymEnv("Pendulum-v1", from_pixels=True)
    collector = MultiaSyncDataCollector(
        [env_make, env_make],
        policy=policy,
        devices=["cuda:0", "cuda:0"],
        total_frames=10000,
        frames_per_batch=50,
        ...
    )
    for i, tensordict_data in enumerate(collector):
        loss = loss_module(tensordict_data)
        loss.backward()
        optim.step()
        optim.zero_grad()
        collector.update_policy_weights_()
    

    Check our distributed collector examples to learn more about ultra-fast data collection with TorchRL.

  • efficient(2) and generic(1) replay buffers with modularized storage:

    Code
    storage = LazyMemmapStorage(  # memory-mapped (physical) storage
        cfg.buffer_size,
        scratch_dir="/tmp/"
    )
    buffer = TensorDictPrioritizedReplayBuffer(
        alpha=0.7,
        beta=0.5,
        collate_fn=lambda x: x,
        pin_memory=device != torch.device("cpu"),
        prefetch=10,  # multi-threaded sampling
        storage=storage
    )
    

    Replay buffers are also offered as wrappers around common datasets for offline RL:

    Code
    from torchrl.data.replay_buffers import SamplerWithoutReplacement
    from torchrl.data.datasets.d4rl import D4RLExperienceReplay
    data = D4RLExperienceReplay(
        "maze2d-open-v0",
        split_trajs=True,
        batch_size=128,
        sampler=SamplerWithoutReplacement(drop_last=True),
    )
    for sample in data:  # or alternatively sample = data.sample()
        fun(sample)
    
  • cross-library environment transforms(1), executed on device and in a vectorized fashion(2), which process and prepare the data coming out of the environments to be used by the agent:

    Code
    env_make = lambda: GymEnv("Pendulum-v1", from_pixels=True)
    env_base = ParallelEnv(4, env_make, device="cuda:0")  # creates 4 envs in parallel
    env = TransformedEnv(
        env_base,
        Compose(
            ToTensorImage(),
            ObservationNorm(loc=0.5, scale=1.0)),  # executes the transforms once and on device
    )
    tensordict = env.reset()
    assert tensordict.device == torch.device("cuda:0")
    

    Other transforms include: reward scaling (RewardScaling), shape operations (concatenation of tensors, unsqueezing etc.), concatenation of successive operations (CatFrames), resizing (Resize) and many more.

    Unlike other libraries, the transforms are stacked as a list (and not wrapped in each other), which makes it easy to add and remove them at will:

    env.insert_transform(0, NoopResetEnv())  # inserts the NoopResetEnv transform at the index 0
    

    Nevertheless, transforms can access and execute operations on the parent environment:

    transform = env.transform[1]  # gathers the second transform of the list
    parent_env = transform.parent  # returns the base environment of the second transform, i.e. the base env + the first transform
    
  • various tools for distributed learning (e.g. memory mapped tensors)(2);

  • various architectures and models (e.g. actor-critic)(1):

    Code
    # create an nn.Module
    common_module = ConvNet(
        bias_last_layer=True,
        depth=None,
        num_cells=[32, 64, 64],
        kernel_sizes=[8, 4, 3],
        strides=[4, 2, 1],
    )
    # Wrap it in a SafeModule, indicating what key to read in and where to
    # write out the output
    common_module = SafeModule(
        common_module,
        in_keys=["pixels"],
        out_keys=["hidden"],
    )
    # Wrap the policy module in NormalParamsWrapper, such that the output
    # tensor is split in loc and scale, and scale is mapped onto a positive space
    policy_module = SafeModule(
        NormalParamsWrapper(
            MLP(num_cells=[64, 64], out_features=32, activation=nn.ELU)
        ),
        in_keys=["hidden"],
        out_keys=["loc", "scale"],
    )
    # Use a SafeProbabilisticTensorDictSequential to combine the SafeModule with a
    # SafeProbabilisticModule, indicating how to build the
    # torch.distribution.Distribution object and what to do with it
    policy_module = SafeProbabilisticTensorDictSequential(  # stochastic policy
        policy_module,
        SafeProbabilisticModule(
            in_keys=["loc", "scale"],
            out_keys="action",
            distribution_class=TanhNormal,
        ),
    )
    value_module = MLP(
        num_cells=[64, 64],
        out_features=1,
        activation=nn.ELU,
    )
    # Wrap the policy and value funciton in a common module
    actor_value = ActorValueOperator(common_module, policy_module, value_module)
    # standalone policy from this
    standalone_policy = actor_value.get_policy_operator()
    
  • exploration wrappers and modules to easily swap between exploration and exploitation(1):

    Code
    policy_explore = EGreedyWrapper(policy)
    with set_exploration_type(ExplorationType.RANDOM):
        tensordict = policy_explore(tensordict)  # will use eps-greedy
    with set_exploration_type(ExplorationType.DETERMINISTIC):
        tensordict = policy_explore(tensordict)  # will not use eps-greedy
    
  • A series of efficient loss modules and highly vectorized functional return and advantage computation.

    Code

    Loss modules

    from torchrl.objectives import DQNLoss
    loss_module = DQNLoss(value_network=value_network, gamma=0.99)
    tensordict = replay_buffer.sample(batch_size)
    loss = loss_module(tensordict)
    

    Advantage computation

    from torchrl.objectives.value.functional import vec_td_lambda_return_estimate
    advantage = vec_td_lambda_return_estimate(gamma, lmbda, next_state_value, reward, done, terminated)
    
  • a generic trainer class(1) that executes the aforementioned training loop. Through a hooking mechanism, it also supports any logging or data transformation operation at any given time.

  • various recipes to build models that correspond to the environment being deployed.

If you feel a feature is missing from the library, please submit an issue! If you would like to contribute to new features, check our call for contributions and our contribution page.

Examples, tutorials and demos

A series of State-of-the-Art implementations are provided with an illustrative purpose:

Algorithm Compile Support** Tensordict-free API Modular Losses Continuous and Discrete
DQN 1.9x + NA + (through ActionDiscretizer transform)
DDPG 1.87x + + - (continuous only)
IQL 3.22x + + +
CQL 2.68x + + +
TD3 2.27x + + - (continuous only)
TD3+BC untested + + - (continuous only)
A2C 2.67x + - +
PPO 2.42x + - +
SAC 2.62x + - +
REDQ 2.28x + - - (continuous only)
Dreamer v1 untested + + (different classes) - (continuous only)
Decision Transformers untested + NA - (continuous only)
CrossQ untested + + - (continuous only)
Gail untested + NA +
Impala untested + - +
IQL (MARL) untested + + +
DDPG (MARL) untested + + - (continuous only)
PPO (MARL) untested + - +
QMIX-VDN (MARL) untested + NA +
SAC (MARL) untested + - +
RLHF NA + NA NA

** The number indicates expected speed-up compared to eager mode when executed on CPU. Numbers may vary depending on architecture and device.

and many more to come!

Code examples displaying toy code snippets and training scripts are also available

Check the examples directory for more details about handling the various configuration settings.

We also provide tutorials and demos that give a sense of what the library can do.

Citation

If you're using TorchRL, please refer to this BibTeX entry to cite this work:

@misc{bou2023torchrl,
      title={TorchRL: A data-driven decision-making library for PyTorch}, 
      author={Albert Bou and Matteo Bettini and Sebastian Dittert and Vikash Kumar and Shagun Sodhani and Xiaomeng Yang and Gianni De Fabritiis and Vincent Moens},
      year={2023},
      eprint={2306.00577},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

Installation

Create a conda environment where the packages will be installed.

conda create --name torch_rl python=3.9
conda activate torch_rl

PyTorch

Depending on the use of functorch that you want to make, you may want to install the latest (nightly) PyTorch release or the latest stable version of PyTorch. See here for a detailed list of commands, including pip3 or other special installation instructions.

Torchrl

You can install the latest stable release by using

pip3 install torchrl

This should work on linux, Windows 10 and OsX (Intel or Silicon chips). On certain Windows machines (Windows 11), one should install the library locally (see below).

For AArch64 machines, the binaries are not yet stored on PyPI so you will need to download them directly from the release page or install the library via

pip3 install git+https://github.com/pytorch/rl@v0.8.1

The nightly build can be installed via

pip3 install tensordict-nightly torchrl-nightly

which we currently only ship for Linux machines. Importantly, the nightly builds require the nightly builds of PyTorch too.

To install extra dependencies, call

pip3 install "torchrl[atari,dm_control,gym_continuous,rendering,tests,utils,marl,open_spiel,checkpointing]"

or a subset of these.

To install torchrl with the latest pytorch, use

pip3 install "torchrl[replay_buffer]"

since some features in the replay buffer require PyTorch 2.7.0 or above.

One may also desire to install the library locally. Three main reasons can motivate this:

  • the nightly/stable release isn't available for one's platform (eg, Windows 11, nightlies for Apple Silicon etc.);

  • contributing to the code;

  • install torchrl with a previous version of PyTorch (any version >= 2.1) (note that this should also be doable via a regular install followed by a downgrade to a previous pytorch version -- but the C++ binaries will not be available so some feature will not work,
    such as prioritized replay buffers and the like.)

    Disclaimer: As of today, TorchRL is roughly compatible with any pytorch version >= 2.1 and installing it will not directly require a newer version of pytorch to be installed. Indirectly though, tensordict still requires the latest PyTorch to be installed and we are working hard to loosen that requirement. The C++ binaries of TorchRL (mainly for prioritized replay buffers) will only work with PyTorch 2.7.0 and above. Some features (e.g., working with nested jagged tensors) may also be limited with older versions of pytorch. It is recommended to use the latest TorchRL with the latest PyTorch version unless there is a strong reason not to do so.

To install the library locally, start by cloning the repo:

git clone https://github.com/pytorch/rl

and don't forget to check out the branch or tag you want to use for the build:

git checkout v0.8.1

Go to the directory where you have cloned the torchrl repo and install it (after installing ninja)

cd /path/to/torchrl/
pip3 install ninja -U
python setup.py develop

One can also build the wheels to distribute to co-workers using

python setup.py bdist_wheel

Your wheels will be stored there ./dist/torchrl<name>.whl and installable via

pip install torchrl<name>.whl

Warning: Unfortunately, pip3 install -e . does not currently work. Contributions to help fix this are welcome!

On M1 machines, this should work out-of-the-box with the nightly build of PyTorch. If the generation of this artifact in MacOs M1 doesn't work correctly or in the execution the message (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')) appears, then try

ARCHFLAGS="-arch arm64" python setup.py develop

To run a quick sanity check, leave that directory (e.g. by executing cd ~/) and try to import the library.

python -c "import torchrl"

This should not return any warning or error.

Optional dependencies

The following libraries can be installed depending on the usage one wants to make of torchrl:

# diverse
pip3 install tqdm tensorboard "hydra-core>=1.1" hydra-submitit-launcher

# rendering
pip3 install "moviepy<2.0.0"

# deepmind control suite
pip3 install dm_control

# gym, atari games
pip3 install "gym[atari]" "gym[accept-rom-license]" pygame

# tests
pip3 install pytest pyyaml pytest-instafail

# tensorboard
pip3 install tensorboard

# wandb
pip3 install wandb

Troubleshooting

If a ModuleNotFoundError: No module named ‘torchrl._torchrl errors occurs (or a warning indicating that the C++ binaries could not be loaded), it means that the C++ extensions were not installed or not found.

  • One common reason might be that you are trying to import torchrl from within the git repo location. The following code snippet should return an error if torchrl has not been installed in develop mode:
    cd ~/path/to/rl/repo
    python -c 'from torchrl.envs.libs.gym import GymEnv'
    
    If this is the case, consider executing torchrl from another location.
  • If you're not importing torchrl from within its repo location, it could be caused by a problem during the local installation. Check the log after the python setup.py develop. One common cause is a g++/C++ version discrepancy and/or a problem with the ninja library.
  • If the problem persists, feel free to open an issue on the topic in the repo, we'll make our best to help!
  • On MacOs, we recommend installing XCode first. With Apple Silicon M1 chips, make sure you are using the arm64-built python (e.g. here). Running the following lines of code
    wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
    python collect_env.py
    
    should display
    OS: macOS *** (arm64)
    
    and not
    OS: macOS **** (x86_64)
    

Versioning issues can cause error message of the type undefined symbol and such. For these, refer to the versioning issues document for a complete explanation and proposed workarounds.

Asking a question

If you spot a bug in the library, please raise an issue in this repo.

If you have a more generic question regarding RL in PyTorch, post it on the PyTorch forum.

Contributing

Internal collaborations to torchrl are welcome! Feel free to fork, submit issues and PRs. You can checkout the detailed contribution guide here. As mentioned above, a list of open contributions can be found in here.

Contributors are recommended to install pre-commit hooks (using pre-commit install). pre-commit will check for linting related issues when the code is committed locally. You can disable th check by appending -n to your commit command: git commit -m <commit message> -n

Disclaimer

This library is released as a PyTorch beta feature. BC-breaking changes are likely to happen but they will be introduced with a deprecation warranty after a few release cycles.

License

TorchRL is licensed under the MIT License. See LICENSE for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

torchrl-0.8.1-cp313-cp313t-win_amd64.whl (1.2 MB view details)

Uploaded CPython 3.13tWindows x86-64

torchrl-0.8.1-cp313-cp313t-manylinux_2_28_x86_64.whl (1.2 MB view details)

Uploaded CPython 3.13tmanylinux: glibc 2.28+ x86-64

torchrl-0.8.1-cp313-cp313t-manylinux_2_28_aarch64.whl (1.2 MB view details)

Uploaded CPython 3.13tmanylinux: glibc 2.28+ ARM64

torchrl-0.8.1-cp313-cp313t-macosx_11_0_arm64.whl (1.6 MB view details)

Uploaded CPython 3.13tmacOS 11.0+ ARM64

torchrl-0.8.1-cp313-cp313-win_amd64.whl (1.2 MB view details)

Uploaded CPython 3.13Windows x86-64

torchrl-0.8.1-cp313-cp313-manylinux_2_28_x86_64.whl (1.2 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.28+ x86-64

torchrl-0.8.1-cp313-cp313-manylinux_2_28_aarch64.whl (1.2 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.28+ ARM64

torchrl-0.8.1-cp313-cp313-macosx_11_0_arm64.whl (1.5 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

torchrl-0.8.1-cp312-cp312-win_amd64.whl (1.2 MB view details)

Uploaded CPython 3.12Windows x86-64

torchrl-0.8.1-cp312-cp312-manylinux_2_28_x86_64.whl (1.2 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ x86-64

torchrl-0.8.1-cp312-cp312-manylinux_2_28_aarch64.whl (1.2 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ ARM64

torchrl-0.8.1-cp312-cp312-macosx_11_0_arm64.whl (1.5 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

torchrl-0.8.1-cp311-cp311-win_amd64.whl (1.2 MB view details)

Uploaded CPython 3.11Windows x86-64

torchrl-0.8.1-cp311-cp311-manylinux_2_28_x86_64.whl (1.2 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ x86-64

torchrl-0.8.1-cp311-cp311-manylinux_2_28_aarch64.whl (1.2 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ ARM64

torchrl-0.8.1-cp311-cp311-macosx_11_0_arm64.whl (1.5 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

torchrl-0.8.1-cp310-cp310-win_amd64.whl (1.2 MB view details)

Uploaded CPython 3.10Windows x86-64

torchrl-0.8.1-cp310-cp310-manylinux_2_28_x86_64.whl (1.2 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ x86-64

torchrl-0.8.1-cp310-cp310-manylinux_2_28_aarch64.whl (1.2 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ ARM64

torchrl-0.8.1-cp310-cp310-macosx_11_0_arm64.whl (1.5 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

torchrl-0.8.1-cp39-cp39-win_amd64.whl (1.2 MB view details)

Uploaded CPython 3.9Windows x86-64

torchrl-0.8.1-cp39-cp39-manylinux_2_28_x86_64.whl (1.2 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.28+ x86-64

torchrl-0.8.1-cp39-cp39-manylinux_2_28_aarch64.whl (1.2 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.28+ ARM64

torchrl-0.8.1-cp39-cp39-macosx_11_0_arm64.whl (1.5 MB view details)

Uploaded CPython 3.9macOS 11.0+ ARM64

File details

Details for the file torchrl-0.8.1-cp313-cp313t-win_amd64.whl.

File metadata

  • Download URL: torchrl-0.8.1-cp313-cp313t-win_amd64.whl
  • Upload date:
  • Size: 1.2 MB
  • Tags: CPython 3.13t, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.10.17

File hashes

Hashes for torchrl-0.8.1-cp313-cp313t-win_amd64.whl
Algorithm Hash digest
SHA256 ffee7106bb150fb911e7ba6aa4c521388409d861ea36353a815d60958b846935
MD5 55acc2b1df75f2343197c8bb729b4488
BLAKE2b-256 f6691509ac1d440f30bae7843977625baea8754e5f958dfdb30e271a9b9ac5e5

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp313-cp313t-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp313-cp313t-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 412e1ca70435f8fee3e6ee38250665fa23cc3a9215e9208d8c410159126f9115
MD5 4c078c0fa085535b61cdec91e43290aa
BLAKE2b-256 0ac87c59aa2d64446b2eb6cd9ab2558a704e79d45a091e9900bb7069186e9401

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp313-cp313t-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp313-cp313t-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 ac2f658b0abcd034df357c18ff8d198ce4e7f29e022a551aa26f5484b3387d4a
MD5 b25855b8d4366281961bb5635e2fe038
BLAKE2b-256 534d2809d3ae110d085c80402e87a44a8a928f347313ad541d4c374649300fb6

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp313-cp313t-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp313-cp313t-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f342c1df35c014c0c02602bfc11aa84fecfadb37e0e02779b4182a7f3b9fbc73
MD5 57cdcd13cb5979bd9097e5989a9e9aa6
BLAKE2b-256 9b57c0f93d95baa92292ea0ec4aaf4b22dbb41f58a3464a38ed88744e2a8b9c7

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: torchrl-0.8.1-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 1.2 MB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.10.17

File hashes

Hashes for torchrl-0.8.1-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 002d25218faf48d535d653462d68cd18941d4250cfbf9f5824eaa99cca9d002e
MD5 fc184b35ec7e7f86e2a9ed625b6b6257
BLAKE2b-256 97075c3639a3c07842cc35cd93c7eb7b28ec385f92e8e3c3fc7fe17fa102e75e

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp313-cp313-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp313-cp313-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 0741a8e98245f87e1b0d45b015568618f9e868ace60849e5d591d312af490960
MD5 3df1883040f650e68b7e63d36e71f1ea
BLAKE2b-256 f1390ac51295c67f375028638f277a7de565dfb79bb0eb1c60097ac73613a12f

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp313-cp313-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp313-cp313-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 097e4912d41d2beb931243ddbc0ac48062bed2ffd62006218db2e5a313763e7f
MD5 98354db6ddc8b1e7e554a5e5c0eb09eb
BLAKE2b-256 a73d7d72dcdb5dd0e20473b48a663f8df9130b047ce2aa7ebbbdfd0c8c716c10

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f8589abd730b0b3242d30f550365e4ebc57d6980181613350e4237013de12b76
MD5 48d5d9ace7a73ae3aed16279c8f2d8d8
BLAKE2b-256 c61aedcd8ccae0d7d46411762738b4b03fb44b153bd3091cce49e020cb2d85e6

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: torchrl-0.8.1-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 1.2 MB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.10.17

File hashes

Hashes for torchrl-0.8.1-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 26d51c3c2f08dea3542f83a43cb93439636aa14aae890b5e599c589403ad42a2
MD5 1a135291e354d1c881d71915ca9f46e6
BLAKE2b-256 94d7d523c3d0142b63f5e4b8c20c46909a4826f2ae56ce5108d8eecba69ba345

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp312-cp312-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp312-cp312-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 17a083bb9634158f85cf0fd04596eff03ab820861b6c73eaf7da1a50fbf531ec
MD5 c1bc73203fcf5fcf9ca36cf4636502f8
BLAKE2b-256 6bc5544d204dba357208c4ba1f8cc5ce490cf94c2669e3532ac5274f3a20008c

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp312-cp312-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp312-cp312-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 da60f34d4c84e336cb15d3e753b6f0800e46c60925a4806737f1c95da2e611d1
MD5 4058174f9dd6bb395ece62f85dd977e6
BLAKE2b-256 4c3346cf85306dcc0eed24a317f003ec899769d71d3deee2a2536aaae5129506

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 435a018b51c75f1641297b0870eaa38aeff1e9577000d936cbb020d843e88017
MD5 f351e99a80065d31a1de10226642fc7e
BLAKE2b-256 59fc55a96b3b66e4f4ca74fbd7badfed36098f0dc06eb88cfc1713484d5de775

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: torchrl-0.8.1-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 1.2 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.10.17

File hashes

Hashes for torchrl-0.8.1-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 9a99c92345e43106bd1e77c0f6f267c75e27588d2b3769b1ccd421952fcf8ffd
MD5 11de70b234b7df24c80091935a438ca3
BLAKE2b-256 0fdf37bbf7d068729caad714b1baf88aa53f4f6520d033897641f57af03beac6

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp311-cp311-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp311-cp311-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 ff810bf778c4cef7a92c37671f09a6252df084cb11489753ae4047e4ec93a278
MD5 7b6fb620d9b31714d527495a875c3cf4
BLAKE2b-256 dd508e2582afd60fd5b44c5431718fd1e13b45bde09a1cc3c76003f80e385e0a

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp311-cp311-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp311-cp311-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 b7cb13e6308edada0394b1188ca8536b7ddfa4518382c009077d7b5e847e1fba
MD5 6c2509f01b789251a8610711c80f51ba
BLAKE2b-256 ffab03d767f381d3b5b0fc53c1b863ebc9028a17335669c56a014facbbaad689

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 bffafbad5714a80cbbb3371c096b704c2c51c7566309dc306b7d60eee5582165
MD5 76bfd1fdb20eb1cdd61be0a1af89c92b
BLAKE2b-256 0d20a8f35cb7dde7058ef8371d62f5a1375d7584366fc1ddde473abe85337c36

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: torchrl-0.8.1-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 1.2 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.10.17

File hashes

Hashes for torchrl-0.8.1-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 bc4d6791a60a87a2235f50ef93c0cd5b8bd856c61786592381f1276d85da88fa
MD5 ff32a5c06322dff2f9cf5deee4f952f4
BLAKE2b-256 5260ea9d33512e718fb931aeccadcf693ee3ebf1cb3245c0221b61c5198bd2fb

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp310-cp310-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 03b9b2de965ce9f06bf962c9736d580b8807ef2fdd5a470a23b7fed1e95dedff
MD5 fea79ade5388b50b636aaa00b0934063
BLAKE2b-256 849ffe6e8ddebdb719c86980a3c9529612e0b844f7e88b110048a8a6238b44b1

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp310-cp310-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp310-cp310-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 8c259f3f5d3308a228c4baf4a530af975104f024d58b48f5b679a7b82bc40d88
MD5 a05ded2931a58b1af8635b404248f7f3
BLAKE2b-256 cff65018e005bf0075b7bf9c4c0e7512afa72fdda060db62db44bee86dfd1fd5

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 9e42893a954abe01294ca2ef3a5663f5d0ac692364226780ee8ab97d7c5494e0
MD5 98cd96d20c9708ad317806281e5cda9b
BLAKE2b-256 3ff48c6efa91ea58aa9df77bd405baf82152c8ba2a8923e558523b09cd7068a8

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: torchrl-0.8.1-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 1.2 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.10.17

File hashes

Hashes for torchrl-0.8.1-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 94ff704f2e94197382ed779d9c589788842b35a3981e3567ed5da96474c3bda8
MD5 dbd639863c18c88eeeac43abe71cda3b
BLAKE2b-256 3ec8cf38f9cd95f2a28a654447a6faaedcb341c431187a5abbce76618c8ae087

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp39-cp39-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp39-cp39-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 a2e53e17ef2218478bec56cb72a9f0ada638e765c4e964327ff2ac39cd03aeb2
MD5 f5face2102e867ccd42fa1cae67caae4
BLAKE2b-256 cce7b093f08ab955a92164c7cd78b38f63413006e7d146eade930101211733de

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp39-cp39-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp39-cp39-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 d7256eacff53aa34fed571de0fa530c9039d5019b112274989e58dd8343c9900
MD5 d08973c7b00937ab7f089d3c0fd9cc7a
BLAKE2b-256 5816cd4bde31457fa59ebd1819f296102a144caad4c7d9de182d041142fe9788

See more details on using hashes here.

File details

Details for the file torchrl-0.8.1-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for torchrl-0.8.1-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 a77953c310d7a180705fb3350e0baf3a48a8097c68e92c664c4bf9793a6eca98
MD5 e66e83e43932dc2f59d7e8c7af573de8
BLAKE2b-256 336587f1ededed9fe1f75345e6668ec0c5dc75d5b81497a325a1c0d648dc7da2

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page