Dask cuDF - A GPU Backend for Dask DataFrame
Dask cuDF (a.k.a. dask-cudf or dask_cudf) is an extension library for Dask DataFrame that provides a Pandas-like API for parallel and larger-than-memory DataFrame computing on GPUs. When installed, Dask cuDF is automatically registered as the "cudf" dataframe backend for Dask DataFrame.
Using Dask cuDF
Please visit the official documentation page for detailed information about using Dask cuDF.
Installation
See the RAPIDS install page for the most up-to-date information and commands for installing Dask cuDF and other RAPIDS packages.
Resources
- Dask cuDF documentation
- Best practices
- cuDF documentation
- 10 Minutes to cuDF and Dask cuDF
- Dask-CUDA documentation
- Deployment
- RAPIDS Community: Get help, contribute, and collaborate.
Quick-start example
A very common Dask cuDF use case is single-node multi-GPU data processing. These workflows typically use the following pattern:
import dask
import dask.dataframe as dd
from dask_cuda import LocalCUDACluster
from distributed import Client
if __name__ == "__main__":
# Define a GPU-aware cluster to leverage multiple GPUs
client = Client(
LocalCUDACluster(
CUDA_VISIBLE_DEVICES="0,1", # Use two workers (on devices 0 and 1)
rmm_pool_size=0.9, # Use 90% of GPU memory as a pool for faster allocations
enable_cudf_spill=True, # Improve device memory stability
local_directory="/fast/scratch/", # Use fast local storage for spilling
)
)
# Set the default dataframe backend to "cudf"
dask.config.set({"dataframe.backend": "cudf"})
# Create your DataFrame collection from on-disk
# or in-memory data
df = dd.read_parquet("/my/parquet/dataset/")
# Use cudf-like syntax to transform and/or query your data
query = df.groupby('item')['price'].mean()
# Compute, persist, or write out the result
query.head()
If you do not have multiple GPUs available, using LocalCUDACluster is optional. However, it is still a good idea to enable cuDF spilling.
If you wish to scale across multiple nodes, you will need to use a different mechanism to deploy your Dask-CUDA workers. Please see the RAPIDS deployment documentation for more instructions.
Metadata
Release files for dask-cudf-cu12 26.10.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| dask_cudf_cu12-26.10.0-py3-none-any.whl | Python 3 | none | any | Details |
Release files / dask_cudf_cu12-26.10.0-py3-none-any.whl
| Download URL | dask_cudf_cu12-26.10.0-py3-none-any.whl |
|---|---|
| Size | 51.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f9ec1d85f436a6782bd1cddb32c4ee21ecb8a9a69658bf76eee0fdeba84f1f0d
|
|
BLAKE2b-256 checksum How to use checksums |
0121a064f8ece589c5beb87747674d8f54230edc50ccb8ece531a723c9af15ae
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.8
|
