Generic Binary Sharding Tool
This tool is a Generic Binary Sharding tool which serializes binaries into a javascript package where each file is at max a specified shard size.
How it Works
This tool works by determining all the files that match the given criterion (extensions and paths). We read all of these files and store the base64 encoding of their bytes in a dictionary. Later we write a bunch of js files having a maximum of shard_size megabytes size. By default this is set to 64MB.
Finally an entrypoint file is provided which requires all the appropriate files.
DCP Modules
This tool was built to aid in development of dcp packages and for publishing extremely large models and binary files. As such, we've also included some dcp based package generation features.
These features take advantage of a feature bravojs module packages have called module.provide. This feature allows users to request packages to load in dynamically that weren't explicitly required by job.requires.
1 maintainer
Metadata
1 maintainer
Release files for generic-binary-sharding 0.1.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| generic_binary_sharding-0.1.5.tar.gz | 6.4 kB | Without provenance | Details for generic_binary_sharding-0.1.5.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| generic_binary_sharding-0.1.5-py3-none-any.whl | Python 3 | none | any | Details for generic_binary_sharding-0.1.5-py3-none-any.whl |
Total release size: 15.4 kB
Release files / generic_binary_sharding-0.1.5.tar.gz
| Download URL | generic_binary_sharding-0.1.5.tar.gz |
|---|---|
| Size | 6.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
fe0b547da0948398e73ef8b5d4cf876ff4736b87f2f8b533abc7f3598c4112cc
|
|
BLAKE2b-256 checksum How to use checksums |
9a0b0a94932f170f6a2a832bd08af42a5ba59200dbd9136a5b3ada0d403cf7c2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.9.2 Linux/5.15.0-58-generic
|
Release files / generic_binary_sharding-0.1.5-py3-none-any.whl
| Download URL | generic_binary_sharding-0.1.5-py3-none-any.whl |
|---|---|
| Size | 8.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
059aa1229aa7bc70f93fc544be83520a664287c529c88e8edcb7cc210b17d659
|
|
BLAKE2b-256 checksum How to use checksums |
767017cba0eb5e2f47460951b8969795b0241c0237122925785d03e468736725
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.9.2 Linux/5.15.0-58-generic
|
Security for generic-binary-sharding 0.1.5
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
