function to make working in dask_cudf and dask quik-er
Project description
dask-quik
Utilities for transforming data using dask and dask_cudf. Most can take either object, and will still process.
This utility currently has:
- cartesian: The ability to create a sparse matrix of the entire universe of data (creating a cartesian product of your data), and "indexize" your data for future encoding uses
- combine: a shortcut for a left merge, and the ability to prune rows based on what should be grouped by, max, min, and/or avg
- dummy: The ability to run just dask functions, using a dask_cudf dummy class
- split: Splitting your train, validation, and testing datasets with test being the most recent value (for recommendation systems)
- transform: The ability to transform back and forth from pandas, dask, and dask_cudf, and also a workaround to sort by index in dask_cudf
- utils: random utils to setup the dask_cudf cluster, shrink data types when defaulted to 64-bit, and a quick check if there are available GPUs (using nvidia-smi).
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
dask-quik-0.0.4.tar.gz
(11.7 kB
view details)
Built Distribution
File details
Details for the file dask-quik-0.0.4.tar.gz
.
File metadata
- Download URL: dask-quik-0.0.4.tar.gz
- Upload date:
- Size: 11.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8e0552b92a56139f334e9b7c56233a59f0b847946fd5a62904220873d2a5acb3 |
|
MD5 | 8b5d370ca32c0754c335c28a374ef29c |
|
BLAKE2b-256 | 95d6e743ff950d2ddcb2e75a80bcbc6d45b6c203040a8ce95e55a70e2c68ca95 |
File details
Details for the file dask_quik-0.0.4-py2.py3-none-any.whl
.
File metadata
- Download URL: dask_quik-0.0.4-py2.py3-none-any.whl
- Upload date:
- Size: 15.0 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 22dd4e91b54565aac5baffd3f91e7a71af2ad2bc694f4be7da60ed5ddad4e0f1 |
|
MD5 | 459be35b554ead06eb8d78d3c0d93b7d |
|
BLAKE2b-256 | 12a63ae7aab061ce48287bf026440da2bc9cb0c67fd726b8531e662482b19271 |