Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

License GitHub release Contributor Covenant

🤗 hf-xet - xet client tech, used in huggingface_hub

Welcome

hf-xet enables huggingface_hub to utilize xet storage for uploading and downloading to HF Hub. Xet storage provides chunk-based deduplication, efficient storage/retrieval with local disk caching, and backwards compatibility with Git LFS. This library is not meant to be used directly, and is instead intended to be used from huggingface_hub.

Key features

chunk-based deduplication implementation: avoid transferring and storing chunks that are shared across binary files (models, datasets, etc).

🤗 Python bindings: bindings for huggingface_hub package.

network communications: concurrent communication to HF Hub Xet backend services (CAS).

🔖 local disk caching: chunk-based cache that sits alongside the existing huggingface_hub disk cache.

Installation

Install the hf_xet package with pip:

pip install hf_xet

Quick Start

hf_xet is not intended to be run independently as it is expected to be used from huggingface_hub, so to get started with huggingface_hub check out the documentation here.

Contributions (feature requests, bugs, etc.) are encouraged & appreciated 💙💚💛💜🧡❤️

Please join us in making hf-xet better. We value everyone's contributions. Code is not the only way to help. Answering questions, helping each other, improving documentation, filing issues all help immensely. If you are interested in contributing (please do!), check out the contribution guide for this repository.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hf_xet-1.6.1a0.tar.gz (920.6 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

hf_xet-1.6.1a0-cp314-cp314t-win_arm64.whl (3.8 MB view details)

Uploaded CPython 3.14tWindows ARM64

hf_xet-1.6.1a0-cp314-cp314t-win_amd64.whl (4.0 MB view details)

Uploaded CPython 3.14tWindows x86-64

hf_xet-1.6.1a0-cp314-cp314t-musllinux_1_2_x86_64.whl (4.7 MB view details)

Uploaded CPython 3.14tmusllinux: musl 1.2+ x86-64

hf_xet-1.6.1a0-cp314-cp314t-musllinux_1_2_aarch64.whl (4.5 MB view details)

Uploaded CPython 3.14tmusllinux: musl 1.2+ ARM64

hf_xet-1.6.1a0-cp314-cp314t-manylinux_2_28_aarch64.whl (4.2 MB view details)

Uploaded CPython 3.14tmanylinux: glibc 2.28+ ARM64

hf_xet-1.6.1a0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.5 MB view details)

Uploaded CPython 3.14tmanylinux: glibc 2.17+ x86-64

hf_xet-1.6.1a0-cp314-cp314t-macosx_11_0_arm64.whl (3.9 MB view details)

Uploaded CPython 3.14tmacOS 11.0+ ARM64

hf_xet-1.6.1a0-cp314-cp314t-macosx_10_12_x86_64.whl (4.1 MB view details)

Uploaded CPython 3.14tmacOS 10.12+ x86-64

hf_xet-1.6.1a0-cp38-abi3-win_arm64.whl (3.9 MB view details)

Uploaded CPython 3.8+Windows ARM64

hf_xet-1.6.1a0-cp38-abi3-win_amd64.whl (4.0 MB view details)

Uploaded CPython 3.8+Windows x86-64

hf_xet-1.6.1a0-cp38-abi3-musllinux_1_2_x86_64.whl (4.7 MB view details)

Uploaded CPython 3.8+musllinux: musl 1.2+ x86-64

hf_xet-1.6.1a0-cp38-abi3-musllinux_1_2_aarch64.whl (4.5 MB view details)

Uploaded CPython 3.8+musllinux: musl 1.2+ ARM64

hf_xet-1.6.1a0-cp38-abi3-manylinux_2_28_aarch64.whl (4.3 MB view details)

Uploaded CPython 3.8+manylinux: glibc 2.28+ ARM64

hf_xet-1.6.1a0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.5 MB view details)

Uploaded CPython 3.8+manylinux: glibc 2.17+ x86-64

hf_xet-1.6.1a0-cp38-abi3-macosx_11_0_arm64.whl (3.9 MB view details)

Uploaded CPython 3.8+macOS 11.0+ ARM64

hf_xet-1.6.1a0-cp38-abi3-macosx_10_12_x86_64.whl (4.1 MB view details)

Uploaded CPython 3.8+macOS 10.12+ x86-64

File details

Details for the file hf_xet-1.6.1a0.tar.gz.

File metadata

  • Download URL: hf_xet-1.6.1a0.tar.gz
  • Upload date:
  • Size: 920.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.6.1a0.tar.gz
Algorithm Hash digest
SHA256 a6906102eb169ccf8ae00be3c6a5dedffe30ce6af3b3dc6f1cbd65ecf20ff990
MD5 907015f5fd6cd8456e7a038503875fcc
BLAKE2b-256 33393956c5c93a2ccddf41540350cd71a3f701a3492522f821259cc567edb846

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0.tar.gz:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp314-cp314t-win_arm64.whl.

File metadata

  • Download URL: hf_xet-1.6.1a0-cp314-cp314t-win_arm64.whl
  • Upload date:
  • Size: 3.8 MB
  • Tags: CPython 3.14t, Windows ARM64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.6.1a0-cp314-cp314t-win_arm64.whl
Algorithm Hash digest
SHA256 7df0358351a88e2346d4d9570e42e39dda9c8dafa767a2c1bd6175f3426ce213
MD5 66b1d347671cf79214adbdbfe32989bd
BLAKE2b-256 49a94e041981c0727f41e0bf5f19464abf36776425651331bf39a64b697ef55e

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp314-cp314t-win_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp314-cp314t-win_amd64.whl.

File metadata

  • Download URL: hf_xet-1.6.1a0-cp314-cp314t-win_amd64.whl
  • Upload date:
  • Size: 4.0 MB
  • Tags: CPython 3.14t, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.6.1a0-cp314-cp314t-win_amd64.whl
Algorithm Hash digest
SHA256 388111f83ddbc33b7777b6899afc93af31cfd52022b79e3dd1434015d5b35071
MD5 7d844a2466841b2d134d78bbae5fa98c
BLAKE2b-256 74d9f088d35ab3a488ca4e715074f58b20a2832d03a9b449533258bcd94f4948

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp314-cp314t-win_amd64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp314-cp314t-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.1a0-cp314-cp314t-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 9ea915037099b509fc079dc3e939fba570766ba831f8eb7b2bbb7c478d3a7444
MD5 4b5c0be603310e8962d8706419e89656
BLAKE2b-256 febaf5ba95926ae9b189534ad9e4391cd7e5a1d8f793c6ef254b8075904fe5a2

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp314-cp314t-musllinux_1_2_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp314-cp314t-musllinux_1_2_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.1a0-cp314-cp314t-musllinux_1_2_aarch64.whl
Algorithm Hash digest
SHA256 04310e0d71db4c2ac00b00235a52fd094bd435a1f8555a697a39bc6512e851de
MD5 83ea12e708b51cb95f2ba5f68d3c7caf
BLAKE2b-256 c743ad1d8de84627824a8ab53c5943bbf4ab89260f832adad98c0c749d649420

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp314-cp314t-musllinux_1_2_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp314-cp314t-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.1a0-cp314-cp314t-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 09cd183f3ec466a0501a00c1cbe5af65b9e0219ad95bbd9a0c2c8fe238db5683
MD5 c0eb0902df52232ab4ea96527bb69f81
BLAKE2b-256 dce008702cd7018ffb2d7c29e076b67711f7c8d68f380beebc29226061303ad8

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp314-cp314t-manylinux_2_28_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.1a0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 e90a28d9866680ff3b78f808cb64e4d7bdc99bc6d6291d1980f097e081ea5d76
MD5 649c87ba936b69ea167aa9e018238329
BLAKE2b-256 3d401be02c71bba1c5480f078eca570ad09458bb3c427a4989f6ef359675328e

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp314-cp314t-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.1a0-cp314-cp314t-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 c5a683c4c1c86c47f54ae7238adaeec39cccd47dde07c43f8aca17601e016104
MD5 94e2e264591123141e6ae545ce2c4f2c
BLAKE2b-256 b392597f593019e6c994e23037158408b3ebf064d24c39bded854fe62851a449

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp314-cp314t-macosx_11_0_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp314-cp314t-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.1a0-cp314-cp314t-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 6c5aca923448fe89b951b05e70184ffbf934286695930c9517f3db33332fad99
MD5 9d6ee2e11ef076a6f2ab35b0f2a49845
BLAKE2b-256 507cfe6064c55955dffa80f7f52555b7866ce3b2f2ba642ecf6c6ada51797318

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp314-cp314t-macosx_10_12_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp38-abi3-win_arm64.whl.

File metadata

  • Download URL: hf_xet-1.6.1a0-cp38-abi3-win_arm64.whl
  • Upload date:
  • Size: 3.9 MB
  • Tags: CPython 3.8+, Windows ARM64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.6.1a0-cp38-abi3-win_arm64.whl
Algorithm Hash digest
SHA256 4e05acc70d690b60fbdf763eb17fff9c0158c94b34339a74f85b4a8020f42b16
MD5 cdcbffc09779120021b3c94cafcb05de
BLAKE2b-256 2915a712ae75483db820ee365beb3a19400bf1ca9cc4f740f5e861d9539e5583

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp38-abi3-win_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp38-abi3-win_amd64.whl.

File metadata

  • Download URL: hf_xet-1.6.1a0-cp38-abi3-win_amd64.whl
  • Upload date:
  • Size: 4.0 MB
  • Tags: CPython 3.8+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.6.1a0-cp38-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 5d6fdcac41dfc1b1da8fabd5ec17345cef16ae7d3a039a76c62de32a9d3d525d
MD5 c88e192305bc6961fe02e55c163ab0b9
BLAKE2b-256 dfcec3eb112285d96ed2bd854e6fc03243a086c0f7425bca0ca7e19d8dbafa60

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp38-abi3-win_amd64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp38-abi3-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.1a0-cp38-abi3-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 5c40ae6949bdaf793529aa460bda2e191614f031c9d68c2dc574afc8bda9462f
MD5 a7ee7bfd01b352dbe4766040547b4fae
BLAKE2b-256 682a5adb82978ff36bc5f69be29521a2153a8ba188d3bf7efc28d13f20fd4aee

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp38-abi3-musllinux_1_2_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp38-abi3-musllinux_1_2_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.1a0-cp38-abi3-musllinux_1_2_aarch64.whl
Algorithm Hash digest
SHA256 30ec8cd98515c81099281603ad116b55200270dda30b87cde61456f6f5b94f01
MD5 07efd5413a3056ad2354d2914b04c7b9
BLAKE2b-256 819a39dcc8f4bc4aa19772c5c5200f40a5c6c4ae2833d76a2f717ae83ac0fa6d

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp38-abi3-musllinux_1_2_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp38-abi3-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.1a0-cp38-abi3-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 305ade275a76d214a297578e53efdd83e6cdb09bec6df47d05e441a29e00408e
MD5 79df2b76445cbac045421422bfaa6e0b
BLAKE2b-256 c5d5cd24eabbcc353b3254670bb417d8692a474b353e2bc396921ceead0ba578

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp38-abi3-manylinux_2_28_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.1a0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 a8309120935178037227a03c9c33afb82e64b05d7219d3d1334a7c7a8223b787
MD5 062e9de70a49d955371f421c748bf82c
BLAKE2b-256 3bd820b2259d7d8aeae0dfb524b988c81caa7fb7ce0b870ff1a8e4c1f3cc97c2

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp38-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.1a0-cp38-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 0ad21a52e5be86e6a45f94407347e2b41286f30f52ceda18f0f65033dfe6ff74
MD5 ea5ae0ba2fe6b2e16865cb194afbc745
BLAKE2b-256 4b8d0450d637e5a75ef9cc3b5d2078ffcbb79f186fa347d88a4ce46863708504

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp38-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.1a0-cp38-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.1a0-cp38-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 032a894447f2673ec0047ffc081d4eca3efb938bb47561de43630e35c9c1944f
MD5 4dbe71c6c9d664870f89fed4ca12226e
BLAKE2b-256 6a017f16ba6b4614bc8b8202c9aee4dee183bd1b1b82693ee6fb61e1f0fd9e0d

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.1a0-cp38-abi3-macosx_10_12_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page