Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

License GitHub release Contributor Covenant

🤗 hf-xet - xet client tech, used in huggingface_hub

Welcome

hf-xet enables huggingface_hub to utilize xet storage for uploading and downloading to HF Hub. Xet storage provides chunk-based deduplication, efficient storage/retrieval with local disk caching, and backwards compatibility with Git LFS. This library is not meant to be used directly, and is instead intended to be used from huggingface_hub.

Key features

chunk-based deduplication implementation: avoid transferring and storing chunks that are shared across binary files (models, datasets, etc).

🤗 Python bindings: bindings for huggingface_hub package.

network communications: concurrent communication to HF Hub Xet backend services (CAS).

🔖 local disk caching: chunk-based cache that sits alongside the existing huggingface_hub disk cache.

Installation

Install the hf_xet package with pip:

pip install hf_xet

Quick Start

hf_xet is not intended to be run independently as it is expected to be used from huggingface_hub, so to get started with huggingface_hub check out the documentation here.

Contributions (feature requests, bugs, etc.) are encouraged & appreciated 💙💚💛💜🧡❤️

Please join us in making hf-xet better. We value everyone's contributions. Code is not the only way to help. Answering questions, helping each other, improving documentation, filing issues all help immensely. If you are interested in contributing (please do!), check out the contribution guide for this repository.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hf_xet-1.6.0rc0.tar.gz (920.6 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

hf_xet-1.6.0rc0-cp314-cp314t-win_arm64.whl (3.8 MB view details)

Uploaded CPython 3.14tWindows ARM64

hf_xet-1.6.0rc0-cp314-cp314t-win_amd64.whl (4.0 MB view details)

Uploaded CPython 3.14tWindows x86-64

hf_xet-1.6.0rc0-cp314-cp314t-musllinux_1_2_x86_64.whl (4.7 MB view details)

Uploaded CPython 3.14tmusllinux: musl 1.2+ x86-64

hf_xet-1.6.0rc0-cp314-cp314t-musllinux_1_2_aarch64.whl (4.5 MB view details)

Uploaded CPython 3.14tmusllinux: musl 1.2+ ARM64

hf_xet-1.6.0rc0-cp314-cp314t-manylinux_2_28_aarch64.whl (4.2 MB view details)

Uploaded CPython 3.14tmanylinux: glibc 2.28+ ARM64

hf_xet-1.6.0rc0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.5 MB view details)

Uploaded CPython 3.14tmanylinux: glibc 2.17+ x86-64

hf_xet-1.6.0rc0-cp314-cp314t-macosx_11_0_arm64.whl (3.9 MB view details)

Uploaded CPython 3.14tmacOS 11.0+ ARM64

hf_xet-1.6.0rc0-cp314-cp314t-macosx_10_12_x86_64.whl (4.1 MB view details)

Uploaded CPython 3.14tmacOS 10.12+ x86-64

hf_xet-1.6.0rc0-cp38-abi3-win_arm64.whl (3.9 MB view details)

Uploaded CPython 3.8+Windows ARM64

hf_xet-1.6.0rc0-cp38-abi3-win_amd64.whl (4.0 MB view details)

Uploaded CPython 3.8+Windows x86-64

hf_xet-1.6.0rc0-cp38-abi3-musllinux_1_2_x86_64.whl (4.7 MB view details)

Uploaded CPython 3.8+musllinux: musl 1.2+ x86-64

hf_xet-1.6.0rc0-cp38-abi3-musllinux_1_2_aarch64.whl (4.5 MB view details)

Uploaded CPython 3.8+musllinux: musl 1.2+ ARM64

hf_xet-1.6.0rc0-cp38-abi3-manylinux_2_28_aarch64.whl (4.3 MB view details)

Uploaded CPython 3.8+manylinux: glibc 2.28+ ARM64

hf_xet-1.6.0rc0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.5 MB view details)

Uploaded CPython 3.8+manylinux: glibc 2.17+ x86-64

hf_xet-1.6.0rc0-cp38-abi3-macosx_11_0_arm64.whl (3.9 MB view details)

Uploaded CPython 3.8+macOS 11.0+ ARM64

hf_xet-1.6.0rc0-cp38-abi3-macosx_10_12_x86_64.whl (4.1 MB view details)

Uploaded CPython 3.8+macOS 10.12+ x86-64

File details

Details for the file hf_xet-1.6.0rc0.tar.gz.

File metadata

  • Download URL: hf_xet-1.6.0rc0.tar.gz
  • Upload date:
  • Size: 920.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.6.0rc0.tar.gz
Algorithm Hash digest
SHA256 ea7c5ec3777a2e28851bc27f1dd00210d9fe2076341b8012875c779cb4bf74ab
MD5 b9b6a953a044ee6fc933da16567df6c1
BLAKE2b-256 28566be7b19b2a0b4dafbf0c0aaee21dc05c43be985823bb649e2955489e18af

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0.tar.gz:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp314-cp314t-win_arm64.whl.

File metadata

  • Download URL: hf_xet-1.6.0rc0-cp314-cp314t-win_arm64.whl
  • Upload date:
  • Size: 3.8 MB
  • Tags: CPython 3.14t, Windows ARM64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.6.0rc0-cp314-cp314t-win_arm64.whl
Algorithm Hash digest
SHA256 1dd444ff218d99bb66afb0899e96e1ab4e4b6abd60df3240cf961f1ea320c4c1
MD5 270265e740ceb8701b2c37be8d3f1ef2
BLAKE2b-256 489d7e688fd7c639f26681c599aef14c415ee1da1e7695cb56ea0e1479535e78

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp314-cp314t-win_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp314-cp314t-win_amd64.whl.

File metadata

  • Download URL: hf_xet-1.6.0rc0-cp314-cp314t-win_amd64.whl
  • Upload date:
  • Size: 4.0 MB
  • Tags: CPython 3.14t, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.6.0rc0-cp314-cp314t-win_amd64.whl
Algorithm Hash digest
SHA256 5474baadbaae44f50125b12d108ec34187e29df2e959e2fcb528976a7e208908
MD5 8e0b90508e2a2f56a9e1b21a91a81294
BLAKE2b-256 eb0e06c868c84651b3c9db70b3c0741f1c82fcc38936003da486cbfe35a010d3

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp314-cp314t-win_amd64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp314-cp314t-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0rc0-cp314-cp314t-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 b8ba08abc681d54a1f9603f41fa31d1a0d33d77bec0099e390d76db1fcf83329
MD5 df1e396fb2697529439610be76834cf5
BLAKE2b-256 3f8edbfc4430a30fab3efb4f0d3f4921314fba664fe7c0a365673daa10e4b8ad

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp314-cp314t-musllinux_1_2_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp314-cp314t-musllinux_1_2_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0rc0-cp314-cp314t-musllinux_1_2_aarch64.whl
Algorithm Hash digest
SHA256 5083960281c5df4f7ca98f0bcceffc861fe092829cef6acc81641ab5e9f1c9dc
MD5 72305abf092f8ebc9f43fedf15741de2
BLAKE2b-256 0997a7e7bebae9257a641b1826dc8758bd58b8632ba1297024208f7c2cc39f0d

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp314-cp314t-musllinux_1_2_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp314-cp314t-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0rc0-cp314-cp314t-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 6034706074f14c7eeda80b4faaf0e8ab7e6bc88f40da0250620238aebe296607
MD5 0f7c089d7ec62f192227b57f03a11ece
BLAKE2b-256 2f0229a317aa489f93218936c434a5fe07347eb80013ad5459f7da529a184a8c

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp314-cp314t-manylinux_2_28_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0rc0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 86a0a024a25c98702eb932414de1fb0af7355fb7ee2e5d2a9eea09db2b621358
MD5 fea26ae9f5b98d9d3978b96d4b2b0b57
BLAKE2b-256 263ec9757e76fd0cb3df19ced48b2ade2149ec84576fa5abbd44c2d5c809c371

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp314-cp314t-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0rc0-cp314-cp314t-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 83c5840942b80d7c5653c6c9473b64e419a32dd9eed461807273b5f255c375b3
MD5 fdbd2b4a8591791ec0524f72fbfe806d
BLAKE2b-256 0af5dca7c2351dc6c88d83f9268cfe9c02ec336984cbf4cd1cfe99df2c4ee2c9

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp314-cp314t-macosx_11_0_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp314-cp314t-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0rc0-cp314-cp314t-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 4cd171587e845039ea509b99048cb065353f3dc79ac5e20e725507d4059a5417
MD5 22dc1fab4193cf877e2b11417a244c1a
BLAKE2b-256 020585009f74afcaffebae8cd6b5d38ad3a9c20166e1b8ee47ddc45808c7c604

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp314-cp314t-macosx_10_12_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp38-abi3-win_arm64.whl.

File metadata

  • Download URL: hf_xet-1.6.0rc0-cp38-abi3-win_arm64.whl
  • Upload date:
  • Size: 3.9 MB
  • Tags: CPython 3.8+, Windows ARM64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.6.0rc0-cp38-abi3-win_arm64.whl
Algorithm Hash digest
SHA256 94380a5290964373e08da3c62dd481fcbf124b3c5ae411f0138482b313d42bb7
MD5 b39b0d116427cf0087c08c2d25e58f3d
BLAKE2b-256 dbf23848bb7967f2d71051a7faf6b406ff0ccc106c45fae2a7f576171048423b

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp38-abi3-win_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp38-abi3-win_amd64.whl.

File metadata

  • Download URL: hf_xet-1.6.0rc0-cp38-abi3-win_amd64.whl
  • Upload date:
  • Size: 4.0 MB
  • Tags: CPython 3.8+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.6.0rc0-cp38-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 026ccc77fdf0ac3035aa1f6d5af59dce3b66b68c3225cd340192ef9057bbc8df
MD5 27c85559d9f706ec020e51a9d04226fd
BLAKE2b-256 f1e9b05f0086939a376feeef53552a9dd8cbcf3a1f8f64ea74742c36d58c0425

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp38-abi3-win_amd64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp38-abi3-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0rc0-cp38-abi3-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 08ae4a60c892a68a3803f687c09115c9ca254f2a8d1490d4d471d772f3ac43f7
MD5 3ef9afc4096ef80b085ba90561a6080d
BLAKE2b-256 0796d53cdd893597a82094632da16d05d9364a4c2cb7c4987c053d927db0b87f

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp38-abi3-musllinux_1_2_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp38-abi3-musllinux_1_2_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0rc0-cp38-abi3-musllinux_1_2_aarch64.whl
Algorithm Hash digest
SHA256 2e2ac34f5de21bf636fae6c571609fe5b0bba38cf65f13b6fc4898d99a98a861
MD5 06e28a7227256893eeef9555d8c6406a
BLAKE2b-256 0598a8bd9c1d5a38791b3ba9b7fbdc7f0c162fcb5aac7c7a111a2b765c456410

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp38-abi3-musllinux_1_2_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp38-abi3-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0rc0-cp38-abi3-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 687cc0ca2a67e149be7234840b05a420222bb80bd9119a6c71c1cbaf171045d3
MD5 d502b0addbf367d0c763a907b248a34f
BLAKE2b-256 7747f1005946589e204732a9818e53f9d517fca779fb60c418bd2c38e0ac3a6c

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp38-abi3-manylinux_2_28_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0rc0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 83a3b7d3017444c0062a2fd9d64682d6f3e2a394ff199ee00cbc23c55e558d03
MD5 fd050b9b4ab917bd49e3853821986627
BLAKE2b-256 8ac369eed09ab918a7460272e510c6fb0f647718353224e2d3b4db8623b9cd44

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp38-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0rc0-cp38-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 e82996f48bb9280cc0f76276d9a1532d752c8f403102af083ffe8837e3b039bb
MD5 4fa5b54d0d52f78fc962c8ea04d75529
BLAKE2b-256 6d899985fb63380922c3384204b7b3e11ec485c1e744d4fd6a32a65cf77d9d7b

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp38-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0rc0-cp38-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0rc0-cp38-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 2408faba136fc611a90790744df14787b76530e76623671379753ad64e3baf68
MD5 9d4fedcb0e48ad8af167719eb42c47ec
BLAKE2b-256 1c3d7b14f3204f6d1bdd7636126ccd25bf5fb160f4aa1f2e39bcb0235e73badb

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0rc0-cp38-abi3-macosx_10_12_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page