Data Preparation Toolkit Transforms using Ray
Project description
DPK Ray Transforms
installation
The transforms are delivered as a standard pyton library available on pypi and can be installed using pip install:
python -m pip install data-prep-toolkit-transforms-ray
installing the Ray transforms will also install data_prep_toolkit_transforms
and data-prep-toolkit-ray
List of Ray Transforms availabe in current package
- code
- code2parquet
- proglang_select
- header_cleanser (Not available on MacOS)
- code_quality
- repo_level_ordering (PR #434)
- language
- doc_quality
- doc_chunk
- lang_id
- text_encoder
- pdf2parquet
- universal
- fdedup
- tokenization
- ededup
- profiler
- doc_id
- filter
- resize
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for data_prep_toolkit_transforms_ray-0.2.1.dev1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 40e52e634aeca3a65310112fc3c37d65ee62fda0e3cb79a7c5133dcd38689961 |
|
MD5 | ce956f5d960bd6987d967ae4a884b747 |
|
BLAKE2b-256 | 446ec71ce76dbf3e0d5e2a641b4b8e4c3706200e6c42a56348bba9d4a61cb9d9 |
Close
Hashes for data_prep_toolkit_transforms_ray-0.2.1.dev1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 51c589bbfef761315f93f0e0e4ec54d2651e3217485a500f6a5bb61888f45361 |
|
MD5 | 76ed00357f54572e676a5723ccad59ee |
|
BLAKE2b-256 | 4abf8120be0795e658c7a39077e3d2a7619e27a367f92b2880e39f7f28fd64e3 |