🤗 hf-xet - xet client tech, used in huggingface_hub
Welcome
hf-xet enables huggingface_hub to utilize xet storage for uploading and downloading to HF Hub. Xet storage provides chunk-based deduplication, efficient storage/retrieval with local disk caching, and backwards compatibility with Git LFS. This library is not meant to be used directly, and is instead intended to be used from huggingface_hub.
Key features
♻ chunk-based deduplication implementation: avoid transferring and storing chunks that are shared across binary files (models, datasets, etc).
🤗 Python bindings: bindings for huggingface_hub package.
↔ network communications: concurrent communication to HF Hub Xet backend services (CAS).
🔖 local disk caching: chunk-based cache that sits alongside the existing huggingface_hub disk cache.
Installation
Install the hf_xet package with pip:
pip install hf_xet
Quick Start
hf_xet is not intended to be run independently as it is expected to be used from huggingface_hub, so to get started with huggingface_hub check out the documentation here.
Contributions (feature requests, bugs, etc.) are encouraged & appreciated 💙💚💛💜🧡❤️
Please join us in making hf-xet better. We value everyone's contributions. Code is not the only way to help. Answering questions, helping each other, improving documentation, filing issues all help immensely. If you are interested in contributing (please do!), check out the contribution guide for this repository.
5 maintainers
Metadata
5 maintainers
Release files for hf-xet 1.7.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| hf_xet-1.7.0.tar.gz | 985.7 kB | Details |
Built distributions (wheels)
Total release size: 64.2 MB
Release files / hf_xet-1.7.0.tar.gz
| Download URL | hf_xet-1.7.0.tar.gz |
|---|---|
| Size | 985.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d406ec79053c0871817f700c2ac8c36ba0d87f9c34b7458b0f0063bb218b0466
|
|
BLAKE2b-256 checksum How to use checksums |
9e2706d899ea7bd721d272f84aac98bdb238de98af4cc767a69056d967d68c71
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp314-cp314t-win_arm64.whl
| Download URL | hf_xet-1.7.0-cp314-cp314t-win_arm64.whl |
|---|---|
| Size | 3.6 MB |
| Tags | CPython 3.14 CPython 3.14 free-threading Windows ARM64 |
|
SHA-256 checksum How to use checksums |
b91569d5f1b61c34b043687da02c05dd3604f3d329e7868510bf3f7971599006
|
|
BLAKE2b-256 checksum How to use checksums |
3dc4310c3c29e5beae7c049e63947bd1923d597883b41c9ec4718589920812c4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp314-cp314t-win_amd64.whl
| Download URL | hf_xet-1.7.0-cp314-cp314t-win_amd64.whl |
|---|---|
| Size | 3.8 MB |
| Tags | CPython 3.14 CPython 3.14 free-threading Windows x86-64 |
|
SHA-256 checksum How to use checksums |
757168feb5679647c0bb13ee5d0faebe799c4dff9051419885a566ebd79f949d
|
|
BLAKE2b-256 checksum How to use checksums |
3d7f832d3ddb49326114175b7bcc50daea8565c09fd21ac03a02b211c09fefb7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp314-cp314t-musllinux_1_2_x86_64.whl
| Download URL | hf_xet-1.7.0-cp314-cp314t-musllinux_1_2_x86_64.whl |
|---|---|
| Size | 4.4 MB |
| Tags | CPython 3.14 CPython 3.14 free-threading Linux musl 1.2+ x86-64 |
|
SHA-256 checksum How to use checksums |
19c0e64f14175ccb6a1aff69e0d2ab9ec5269a560e6687abaf2b3fa4f73de7cd
|
|
BLAKE2b-256 checksum How to use checksums |
def0277f039b7d72027bc2ed277f1b62a2f70f740a5aac2a3e7243e5b6854c5d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp314-cp314t-musllinux_1_2_aarch64.whl
| Download URL | hf_xet-1.7.0-cp314-cp314t-musllinux_1_2_aarch64.whl |
|---|---|
| Size | 4.2 MB |
| Tags | CPython 3.14 CPython 3.14 free-threading Linux musl 1.2+ ARM64 |
|
SHA-256 checksum How to use checksums |
4ee5e05a627f5ab5bad7a86582277d645556ea1e199903aae19e033a392aa13a
|
|
BLAKE2b-256 checksum How to use checksums |
93349d681f0e3dac0b5dae0d7dea748429266f24e52415446523f464fbaa828e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp314-cp314t-manylinux_2_28_aarch64.whl
| Download URL | hf_xet-1.7.0-cp314-cp314t-manylinux_2_28_aarch64.whl |
|---|---|
| Size | 4.0 MB |
| Tags | CPython 3.14 CPython 3.14 free-threading Linux glibc 2.28+ ARM64 |
|
SHA-256 checksum How to use checksums |
b01fe18dbbd151a2403d2c64ed30dc6547b00d6babab9a617d77c7acdb81ee66
|
|
BLAKE2b-256 checksum How to use checksums |
232d225d5b11a9ca7d31b9470a57f2b2be1a5cef8b84325a2146aeb4589e226c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
| Download URL | hf_xet-1.7.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl |
|---|---|
| Size | 4.2 MB |
| Tags | CPython 3.14 CPython 3.14 free-threading Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
87dab080f8f7d32781c2586904e3603f4e60d09bfc727706c3ae419e0829beeb
|
|
BLAKE2b-256 checksum How to use checksums |
e5c20d1eaa5da13bbf9c896badc7f380601c7d973a87a6ffb4d100267c4536c1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp314-cp314t-macosx_11_0_arm64.whl
| Download URL | hf_xet-1.7.0-cp314-cp314t-macosx_11_0_arm64.whl |
|---|---|
| Size | 3.6 MB |
| Tags | CPython 3.14 CPython 3.14 free-threading macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
57bc157b8b7fe3bee9dcb9af7f3da8de41801c3b31a9ef68a77a33c6a6be382f
|
|
BLAKE2b-256 checksum How to use checksums |
ff3a5e8b363391adcbb002e191dbf924dab31464ea9c45adfeb73502afc36d35
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp314-cp314t-macosx_10_12_x86_64.whl
| Download URL | hf_xet-1.7.0-cp314-cp314t-macosx_10_12_x86_64.whl |
|---|---|
| Size | 3.8 MB |
| Tags | CPython 3.14 CPython 3.14 free-threading macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
fa029678be1ba7f953c409b0b27bf15cc69cd1c9b3a674fbd78856ebefca1052
|
|
BLAKE2b-256 checksum How to use checksums |
9f7c3e45174942e6793adde6cba4daa7fb037275cf02a944d9eadfcf9ff33b86
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp38-abi3-win_arm64.whl
| Download URL | hf_xet-1.7.0-cp38-abi3-win_arm64.whl |
|---|---|
| Size | 3.7 MB |
| Tags | CPython 3.8 Windows ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
acc3851cf2576a8fb2ae926da863f4efabe21303cf292e9a44332802ab0dcc6a
|
|
BLAKE2b-256 checksum How to use checksums |
48cd072313585f74fe9d441e2eb5e0a4703c30586cd709810ea369675f61b74e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp38-abi3-win_amd64.whl
| Download URL | hf_xet-1.7.0-cp38-abi3-win_amd64.whl |
|---|---|
| Size | 3.8 MB |
| Tags | CPython 3.8 Windows x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
2b7bb5727889b0f2436dbaaad8fc4c3e66b8240d992716989e0c086b4278b1bc
|
|
BLAKE2b-256 checksum How to use checksums |
4d59b35106dfa71b6eef605dc88bd038fe99c7f86fb132a15b60d0bf2f235b2c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp38-abi3-musllinux_1_2_x86_64.whl
| Download URL | hf_xet-1.7.0-cp38-abi3-musllinux_1_2_x86_64.whl |
|---|---|
| Size | 4.4 MB |
| Tags | CPython 3.8 Linux musl 1.2+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
0a9e802f33bf50c851abe45fc5380e61f959e2d369647d6742b79ad9d6c27cab
|
|
BLAKE2b-256 checksum How to use checksums |
6cdc03a21b89f118664a0926ff25b0f8e44a519bf22724a6a8fc7a9abbc188b6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp38-abi3-musllinux_1_2_aarch64.whl
| Download URL | hf_xet-1.7.0-cp38-abi3-musllinux_1_2_aarch64.whl |
|---|---|
| Size | 4.2 MB |
| Tags | CPython 3.8 Linux musl 1.2+ ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
80f79dae613ce9e0ea1fd1ae15616ca9ac74aed4c770aabc199c4f03ebecc863
|
|
BLAKE2b-256 checksum How to use checksums |
a477e52e4201b1cbf571530a61cc57f70182045a39a230089ee5f1df182a4de2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp38-abi3-manylinux_2_28_aarch64.whl
| Download URL | hf_xet-1.7.0-cp38-abi3-manylinux_2_28_aarch64.whl |
|---|---|
| Size | 4.0 MB |
| Tags | CPython 3.8 Linux glibc 2.28+ ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
fcfd6c22418e57dd5b3aea649e813b2e2cfb2aebf317b210d90f1fe4b3018b52
|
|
BLAKE2b-256 checksum How to use checksums |
490ca2f703a5a78267556e89e03316fa0805c86b72b50829bc67665746e8ebf0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
| Download URL | hf_xet-1.7.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl |
|---|---|
| Size | 4.2 MB |
| Tags | CPython 3.8 Linux glibc 2.17+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
2814a6e999d13464c4d679b788cc5d784eb5a4edfc638a31f10e9a11ab531ef8
|
|
BLAKE2b-256 checksum How to use checksums |
a34c2bf3b66c215d409655f28de1622393dde04c9461280d48c7924bb3b2decd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp38-abi3-macosx_11_0_arm64.whl
| Download URL | hf_xet-1.7.0-cp38-abi3-macosx_11_0_arm64.whl |
|---|---|
| Size | 3.6 MB |
| Tags | CPython 3.8 abi3 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
59fba37039233c7fcbe196817d6cdcf1b40dfb17b410f229d85b0cf0a1848da4
|
|
BLAKE2b-256 checksum How to use checksums |
c347a26ebdce7056a61e931f228439bc0ab08cbec239d1690f965e5e637cba79
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency logRelease files / hf_xet-1.7.0-cp38-abi3-macosx_10_12_x86_64.whl
| Download URL | hf_xet-1.7.0-cp38-abi3-macosx_10_12_x86_64.whl |
|---|---|
| Size | 3.8 MB |
| Tags | CPython 3.8 abi3 macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
e3e88a7a75d7d95cbee1f37dc31341d6201124cf21c6c4b1dfab8ccba9b09e0f
|
|
BLAKE2b-256 checksum How to use checksums |
9c0bb03be21ffaada749ba0d3197d8aefbf1aa698bac149580421c15239b299e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 6, 2026.
Transparency log
