Weightie
Weightie is a little utility library for downloading, serialising (and deserialising) weights for my little deep learning projects.
You probably shouldn't use this if you're not me!
Serialisation
Quick-start usage:
>>> # Say your model weights are stored in some custom structure like so:
>>> from typing import NamedTuple
>>> from numpy.typing import NDArray
>>> class MyWeights(NamedTuple):
... foo: NDArray
... bar: list[NDArray]
... baz: bool # Data need not be entirely NDArrays
>>> # They can be dumped and loaded from disk like so:
>>> from weightie import dump, load
>>> weights = MyWeights(foo=np.zeros(1000), bar=np.zeros(10000), baz=True)
>>> with open("example.weights", "wb") as f:
... dump(weights, f)
>>> with open("example.weights", "rb") as f:
... loaded_weights = load(weights)
Weightie's serialisation format stores Python objects containing Numpy arrays such that, when loaded from disk, the Numpy arrays will be memory mapped directly from the files on disk. This means that data isn't actually read from disk until it is used and is easily swapped out of memory by the operating system when not in use.
See weightie/serialiser.py for details of the
on-disk format (it's pretty simple!).
Weight Downloading
Weightie provides a download function which can be used to download weights
filed published as assets on GitHub. (GitHub has very
generous
usage allowances for release assets!)
Quick-start usage:
>>> from weightie import download
>>> download(
... repository="mossblaser/example",
... asset_filenames=["foo.weights", "bar.weights"],
... target_version="v1.2.3",
... )
{
"foo": Path("/path/to/downloaded/v1.1.0-foo.weights"),
"bar": Path("/path/to/downloaded/v1.1.0-bar.weights"),
}
The download function handles various fiddly aspects of downloading weights
including:
- Downloading to platform-native data directories by default (using platformdirs)
- Re-using already downloaded weights when appropriate.
- Ensuring downloaded weight file versions are compatible with the running software version.
- Keeping multiple weight files versions in sync.
See weightie/downloader.py for the full list of
options.
Metadata
Release files for weightie 0.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| weightie-0.0.1.tar.gz | 51.6 kB | Without provenance | Details for weightie-0.0.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| weightie-0.0.1-py3-none-any.whl | Python 3 | none | any | Details for weightie-0.0.1-py3-none-any.whl |
Total release size: 86.8 kB
Release files / weightie-0.0.1.tar.gz
| Download URL | weightie-0.0.1.tar.gz |
|---|---|
| Size | 51.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a3e6446c3c33ef032280356242638f728fa390df3a6125ded9a6d37be86476bc
|
|
BLAKE2b-256 checksum How to use checksums |
a784eba5259cfd6d1671f518db3b6cfcd9b40d32b94f984b5be39ec3756473af
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.10.9
|
Release files / weightie-0.0.1-py3-none-any.whl
| Download URL | weightie-0.0.1-py3-none-any.whl |
|---|---|
| Size | 35.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
6d788a7b8f35c196d09dbed1103b9a2f7dc525225e76191325d22d69840a2073
|
|
BLAKE2b-256 checksum How to use checksums |
983c520bfc56315c0e0f4c7c6a4624dfbc54558b7b7a6c4d06a23eac9738494b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.10.9
|
Security for weightie 0.0.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.