Gradient Boosting Reinforcement Learning (GBRL)

GBRL is a Python-based Gradient Boosting Trees (GBT) library, similar to popular packages such as XGBoost, CatBoost, but specifically designed and optimized for reinforcement learning (RL). GBRL is implemented in C++/CUDA aimed to seamlessly integrate within popular RL libraries.

Overview

GBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets. The following diagram illustrates how GBRL uses gradient boosting trees in RL:

GBRL features a shared tree-based structure for policy and value functions, significantly reducing memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.

Key Features:

• GBT Tailored for RL: GBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets.
• Optimized Actor-Critic Architecture: GBRL features a shared tree-based structure for policy and value functions. This significantly reduces memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.
• Hardware Acceleration: GBRL leverages CUDA for hardware-accelerated computation, ensuring efficiency and speed.
• Seamless Integration: GBRL is designed for easy integration with popular RL libraries. We implemented GBT-based actor-critic algorithm implementations (A2C, PPO, and AWR) in stable_baselines3 GBRL_SB3.

Performance

The following results, obtained using the GBRL_SB3 repository, demonstrate the performance of PPO with GBRL compared to neural-networks across various scenarios and environments:

Getting started

Dependencies

• Python 3.9 or higher

Installation

GBRL provides pre-compiled binaries for easy installation. Choose one of the following options:

CPU-only installation (default):
pip install gbrl

GPU-enabled installation (requires CUDA 12 runtime libraries):
pip install gbrl-gpu

For further installation details and dependencies see the documentation.

Usage Example

For a detailed usage example, see tutorial.ipynb

Current Supported Features

Tree Fitting

• Greedy (Depth-wise) tree building - (CPU/GPU)
• Oblivious (Symmetric) tree building - (CPU/GPU)
• L2 split score - (CPU/GPU)
• Cosine split score - (CPU/GPU)
• Uniform based candidate generation - (CPU/GPU)
• Quantile based candidate generation - (CPU/GPU)
• Supervised learning fitting / Multi-iteration fitting - (CPU/GPU)
  • MultiRMSE loss (only)
• Categorical inputs
• Input feature weights - (CPU/GPU)
• Monotonic constraints - (CPU/GPU, oblivious trees; the whole model must use SGD, Adam is not allowed on any output)

GBT Inference

• SGD optimizer - (CPU/GPU)
• ADAM optimizer - (CPU only)
• Control Variates (gradient variance reduction technique) - (CPU only)
• Shared Tree for policy and value function - (CPU/GPU)
• Linear and constant learning rate scheduler - (CPU/GPU, linear scheduler GPU only for Oblivious trees)
• Support for up to two different optimizers (e.g., policy/value) - (CPU/GPU if both are SGD)
• SHAP value calculation (optimizer-aware; return_base=True gives values that reconstruct the prediction)

Documentation

For comprehensive documentation, visit the GBRL documentation.

Contributing

To contribute to GBRL, please review and sign the Contributor License Agreement (CLA) available at: https://github.com/NVlabs/gbrl/blob/master/CLA.md

Citation

@inproceedings{
fuhrer2025gradient,
title={Gradient Boosting Reinforcement Learning},
author={Benjamin Fuhrer and Chen Tessler and Gal Dalal},
booktitle={Forty-second International Conference on Machine Learning},
year={2025},
url={https://arxiv.org/abs/2407.08250}
}

Licenses

Copyright © 2024-2026, NVIDIA Corporation. All rights reserved.

This work is made available under the NVIDIA MIT License. Click here to view a copy of this license.