Data Preparation Toolkit Transforms using Ray
Project description
DPK Ray Transforms
installation
The transforms are delivered as a standard pyton library available on pypi and can be installed using pip install:
python -m pip install data-prep-toolkit-transforms-ray
installing the Ray transforms will also install data_prep_toolkit_transforms
and data-prep-toolkit-ray
List of Ray Transforms availabe in current package
- code
- code2parquet
- proglang_select
- header_cleanser (Not available on MacOS)
- code_quality
- repo_level_ordering
- language
- doc_quality
- doc_chunk
- lang_id
- text_encoder
- pdf2parquet
- universal
- fdedup
- tokenization
- ededup
- profiler
- doc_id
- filter
- resize
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for data_prep_toolkit_transforms_ray-0.2.1.dev2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3f06db00bb58e499019334b6c206c0c6dc905dd6ecd1793bf933c47e796084a5 |
|
MD5 | ebe63c38a249558736272a5efbbb89bd |
|
BLAKE2b-256 | a7cee5acdbb1421919ed6dc2084534ac40109b9b383f8795fffc95c90c87c985 |
Close
Hashes for data_prep_toolkit_transforms_ray-0.2.1.dev2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a0a6028f3901287ce55fc68bb80762fab8475e285dd6ee37b02577f341f0344a |
|
MD5 | 278162d9ee3894aa73a1c8376cee97e7 |
|
BLAKE2b-256 | 864a262b039e8fd642295336cbc7b8dfb52d5180792d3d055f0bd278c112d50a |