Dask cuDF - A GPU Backend for Dask DataFrame
Dask cuDF - A GPU Backend for Dask DataFrameDask cuDF (a.k.a. dask-cudf or dask_cudf) is an extension library for Dask DataFrame that provides a Pandas-like API for parallel and larger-than-memory DataFrame computing on GPUs. When installed, Dask cuDF is automatically registered as the "cudf" dataframe backend for Dask DataFrame.
[!IMPORTANT] Dask cuDF does not provide support for multi-GPU or multi-node execution on its own. You must also deploy a distributed cluster (ideally with Dask-CUDA) to leverage multiple GPUs efficiently.
Using Dask cuDF
Please visit the official documentation page for detailed information about using Dask cuDF.
Installation
See the RAPIDS install page for the most up-to-date information and commands for installing Dask cuDF and other RAPIDS packages.
Resources
- Dask cuDF documentation
- Best practices
- cuDF documentation
- 10 Minutes to cuDF and Dask cuDF
- Dask-CUDA documentation
- Deployment
- RAPIDS Community: Get help, contribute, and collaborate.
Quick-start example
A very common Dask cuDF use case is single-node multi-GPU data processing. These workflows typically use the following pattern:
import dask
import dask.dataframe as dd
from dask_cuda import LocalCUDACluster
from distributed import Client
if __name__ == "__main__":
# Define a GPU-aware cluster to leverage multiple GPUs
client = Client(
LocalCUDACluster(
CUDA_VISIBLE_DEVICES="0,1", # Use two workers (on devices 0 and 1)
rmm_pool_size=0.9, # Use 90% of GPU memory as a pool for faster allocations
enable_cudf_spill=True, # Improve device memory stability
local_directory="/fast/scratch/", # Use fast local storage for spilling
)
)
# Set the default dataframe backend to "cudf"
dask.config.set({"dataframe.backend": "cudf"})
# Create your DataFrame collection from on-disk
# or in-memory data
df = dd.read_parquet("/my/parquet/dataset/")
# Use cudf-like syntax to transform and/or query your data
query = df.groupby('item')['price'].mean()
# Compute, persist, or write out the result
query.head()
If you do not have multiple GPUs available, using LocalCUDACluster is optional. However, it is still a good idea to enable cuDF spilling.
If you wish to scale across multiple nodes, you will need to use a different mechanism to deploy your Dask-CUDA workers. Please see the RAPIDS deployment documentation for more instructions.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file dask_cudf_cu12-26.8.0-py3-none-any.whl.
File metadata
- Download URL: dask_cudf_cu12-26.8.0-py3-none-any.whl
- Upload date:
- Size: 51.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/7.0.0 CPython/3.14.6
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
298526889d12ec29721bc1f68ba51afe1ac7076fcf11297efea6b27b76d5add7
|
|
| MD5 |
c47a26af08e7276869d0eb27da086465
|
|
| BLAKE2b-256 |
02b9ea0f79824dece96dbaf55b14fe26cd93de72c4806d46b64fac5c745204e6
|
