Data Preparation Toolkit Transforms using Ray
Project description
DPK Ray Transforms
installation
The transforms are delivered as a standard pyton library available on pypi and can be installed using pip install:
python -m pip install data-prep-toolkit-transforms-ray
installing the Ray transforms will also install data_prep_toolkit_transforms
and data-prep-toolkit-ray
List of Ray Transforms availabe in current package
- code
- code2parquet
- proglang_select
- header_cleanser (Not available on MacOS)
- code_quality
- repo_level_ordering
- language
- doc_quality
- doc_chunk
- lang_id
- text_encoder
- pdf2parquet
- universal
- fdedup
- tokenization
- ededup
- profiler
- doc_id
- filter
- resize
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for data_prep_toolkit_transforms_ray-0.2.1.dev0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d2bb1eef83c5bb16e2810d5575b43ab8c81dcbf724471d2c0b485fdce9a064a6 |
|
MD5 | cf1f80bac961aeb9760cb09fc4c7d250 |
|
BLAKE2b-256 | 02265825bddc88b82cce921ccb4315679e421f4a4fa6bc655ace6e29e00d3856 |
Close
Hashes for data_prep_toolkit_transforms_ray-0.2.1.dev0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | eadfdbe7cd83347ecbe00b9c9baad1ace404c5a5584aa8ee0cca5466371b24d4 |
|
MD5 | 3151a8273ad965c89f2a693858b948f5 |
|
BLAKE2b-256 | 09215094b5d3cf73521c4122ea8ae49d8f8706187f3c6112a9c5b61a9b679506 |