Skip to main content

License GitHub release Contributor Covenant

🤗 hf-xet - xet client tech, used in huggingface_hub

Welcome

hf-xet enables huggingface_hub to utilize xet storage for uploading and downloading to HF Hub. Xet storage provides chunk-based deduplication, efficient storage/retrieval with local disk caching, and backwards compatibility with Git LFS. This library is not meant to be used directly, and is instead intended to be used from huggingface_hub.

Key features

chunk-based deduplication implementation: avoid transferring and storing chunks that are shared across binary files (models, datasets, etc).

🤗 Python bindings: bindings for huggingface_hub package.

network communications: concurrent communication to HF Hub Xet backend services (CAS).

🔖 local disk caching: chunk-based cache that sits alongside the existing huggingface_hub disk cache.

Installation

Install the hf_xet package with pip:

pip install hf_xet

Quick Start

hf_xet is not intended to be run independently as it is expected to be used from huggingface_hub, so to get started with huggingface_hub check out the documentation here.

Contributions (feature requests, bugs, etc.) are encouraged & appreciated 💙💚💛💜🧡❤️

Please join us in making hf-xet better. We value everyone's contributions. Code is not the only way to help. Answering questions, helping each other, improving documentation, filing issues all help immensely. If you are interested in contributing (please do!), check out the contribution guide for this repository.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hf_xet-1.6.0.tar.gz (920.5 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

hf_xet-1.6.0-cp314-cp314t-win_arm64.whl (3.8 MB view details)

Uploaded CPython 3.14tWindows ARM64

hf_xet-1.6.0-cp314-cp314t-win_amd64.whl (4.0 MB view details)

Uploaded CPython 3.14tWindows x86-64

hf_xet-1.6.0-cp314-cp314t-musllinux_1_2_x86_64.whl (4.7 MB view details)

Uploaded CPython 3.14tmusllinux: musl 1.2+ x86-64

hf_xet-1.6.0-cp314-cp314t-musllinux_1_2_aarch64.whl (4.5 MB view details)

Uploaded CPython 3.14tmusllinux: musl 1.2+ ARM64

hf_xet-1.6.0-cp314-cp314t-manylinux_2_28_aarch64.whl (4.2 MB view details)

Uploaded CPython 3.14tmanylinux: glibc 2.28+ ARM64

hf_xet-1.6.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.5 MB view details)

Uploaded CPython 3.14tmanylinux: glibc 2.17+ x86-64

hf_xet-1.6.0-cp314-cp314t-macosx_11_0_arm64.whl (3.9 MB view details)

Uploaded CPython 3.14tmacOS 11.0+ ARM64

hf_xet-1.6.0-cp314-cp314t-macosx_10_12_x86_64.whl (4.1 MB view details)

Uploaded CPython 3.14tmacOS 10.12+ x86-64

hf_xet-1.6.0-cp38-abi3-win_arm64.whl (3.9 MB view details)

Uploaded CPython 3.8+Windows ARM64

hf_xet-1.6.0-cp38-abi3-win_amd64.whl (4.0 MB view details)

Uploaded CPython 3.8+Windows x86-64

hf_xet-1.6.0-cp38-abi3-musllinux_1_2_x86_64.whl (4.7 MB view details)

Uploaded CPython 3.8+musllinux: musl 1.2+ x86-64

hf_xet-1.6.0-cp38-abi3-musllinux_1_2_aarch64.whl (4.5 MB view details)

Uploaded CPython 3.8+musllinux: musl 1.2+ ARM64

hf_xet-1.6.0-cp38-abi3-manylinux_2_28_aarch64.whl (4.3 MB view details)

Uploaded CPython 3.8+manylinux: glibc 2.28+ ARM64

hf_xet-1.6.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.5 MB view details)

Uploaded CPython 3.8+manylinux: glibc 2.17+ x86-64

hf_xet-1.6.0-cp38-abi3-macosx_11_0_arm64.whl (3.9 MB view details)

Uploaded CPython 3.8+macOS 11.0+ ARM64

hf_xet-1.6.0-cp38-abi3-macosx_10_12_x86_64.whl (4.1 MB view details)

Uploaded CPython 3.8+macOS 10.12+ x86-64

File details

Details for the file hf_xet-1.6.0.tar.gz.

File metadata

  • Download URL: hf_xet-1.6.0.tar.gz
  • Upload date:
  • Size: 920.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.6.0.tar.gz
Algorithm Hash digest
SHA256 2e58454a340b3556dfa4972d5451aff4fba8dd42a236600ba1a1d2b1514f0fef
MD5 bf1ab4c595afa39ed1391294dc1d9f01
BLAKE2b-256 1bab522a2ab67f27971a9d48ca666d4fca85ef7d5282d142e31fd087e27b1bbe

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0.tar.gz:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp314-cp314t-win_arm64.whl.

File metadata

  • Download URL: hf_xet-1.6.0-cp314-cp314t-win_arm64.whl
  • Upload date:
  • Size: 3.8 MB
  • Tags: CPython 3.14t, Windows ARM64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.6.0-cp314-cp314t-win_arm64.whl
Algorithm Hash digest
SHA256 75765820ce4700db3750c94acc8fe27c5fae4c9ec000a0dbac3ca082acf97765
MD5 4b9acfb34a30507e7b4e295886f630bb
BLAKE2b-256 49f64a9966633c6fef83af997e2cff68ec1963676d412bdfd096df2a93b8e185

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp314-cp314t-win_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp314-cp314t-win_amd64.whl.

File metadata

  • Download URL: hf_xet-1.6.0-cp314-cp314t-win_amd64.whl
  • Upload date:
  • Size: 4.0 MB
  • Tags: CPython 3.14t, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.6.0-cp314-cp314t-win_amd64.whl
Algorithm Hash digest
SHA256 5789835d7c6bc9436962853192082374297fb72d7eff7e7762ec25ceb7e25338
MD5 4af3eba45562ccb3b65c8ff519cc0072
BLAKE2b-256 53a026b717a9d1840e8abf48dcec64b5ed8fbe472671d38ad28d30e147132b33

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp314-cp314t-win_amd64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp314-cp314t-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0-cp314-cp314t-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 35cec30d75c6f9eb9c16a77cef68e85a103b72e24d4b473714ec9ff06428bab9
MD5 a3bb118fb1bac0b1dafcebe199c0bef7
BLAKE2b-256 67570b88af1f194ab6c9c650547d9cc06bfeaab836ae4dcdb331676bfb8be95a

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp314-cp314t-musllinux_1_2_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp314-cp314t-musllinux_1_2_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0-cp314-cp314t-musllinux_1_2_aarch64.whl
Algorithm Hash digest
SHA256 5153e6bb103ad49d6ea9f1b2e230db5a2ea32551ad09a706d2f61d7c7c80d80e
MD5 3a7cecd72682dd066ed3fa94abd6cd1c
BLAKE2b-256 d8261eee8aedb0dafc1ab9717dc9ac602cde33361b232dc06803f1f6ed18b58c

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp314-cp314t-musllinux_1_2_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp314-cp314t-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0-cp314-cp314t-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 948f15d3a9545cfe5932f6bd8b440f6ae630aee108f14b7bd6c561f7c2dcc522
MD5 dddf4c441cda02e278ffbf6a69820436
BLAKE2b-256 6306ad8eab1c9525246650cbaa821caa3cdbaca734ab1a5b8c91bea09cbd8d69

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp314-cp314t-manylinux_2_28_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 f2f7278c05c22fd60cb436cda1269649b3e81db65ecdc8496e5e164aa4143e7b
MD5 d28aec43788b6f22982ae14b6348e4de
BLAKE2b-256 d5ee7c0d7b6ab336167531b1c30af2af003f054af4c749becbd7209ae33a77c3

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp314-cp314t-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0-cp314-cp314t-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 23379c2f9ec8696d952b16414a2bae72cad86a52df869b050698ba60f538c675
MD5 7626fcbe06c0687afb2e15afc45f70f0
BLAKE2b-256 bb1ec0ad437dd267a8e435bef594acf781bbc3874ff0b6435b4962d03ecf7cc4

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp314-cp314t-macosx_11_0_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp314-cp314t-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0-cp314-cp314t-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 70cbb9c896901600128cb9b6f06e132954fbede1db30f31f7c6c63f84cb7c31d
MD5 f6bc8a888858c53ea59936e6a27dfe2c
BLAKE2b-256 41623c062f593bd92ef4e77a0ef39541e3d82a0a1d3947c8a777a02a13a27828

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp314-cp314t-macosx_10_12_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp38-abi3-win_arm64.whl.

File metadata

  • Download URL: hf_xet-1.6.0-cp38-abi3-win_arm64.whl
  • Upload date:
  • Size: 3.9 MB
  • Tags: CPython 3.8+, Windows ARM64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.6.0-cp38-abi3-win_arm64.whl
Algorithm Hash digest
SHA256 3dc3e35441ba395006af5aaacc40ef2e603c51ef46c3530b9156185f00935ea3
MD5 9f9d4769263768601a178eb41d9d9416
BLAKE2b-256 7363ca511b6f802f28cf3489b280fe77475bcca8de85e81a6299d7916b5b5555

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp38-abi3-win_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp38-abi3-win_amd64.whl.

File metadata

  • Download URL: hf_xet-1.6.0-cp38-abi3-win_amd64.whl
  • Upload date:
  • Size: 4.0 MB
  • Tags: CPython 3.8+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.6.0-cp38-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 fb4fadde1b2b70bf4c0c14a6dccbe7194b1c28947fefd5bbe3fed9d940676c3b
MD5 de73779a780248332d5fe4eb7e7b6065
BLAKE2b-256 98b78c59a66d15205024662f1d66968136f13893f96df1ddc5087e2e281fc95f

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp38-abi3-win_amd64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp38-abi3-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0-cp38-abi3-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 8fb4f71cba6129110c3374a33f919001ff130488fc23553698e34cc1c2a1198c
MD5 fa26525f816e4e445c90c36095b69ac1
BLAKE2b-256 ab5f311725e2a905534dfee2dcb5b08414f249147f1f12252bfc2bd24caa075c

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp38-abi3-musllinux_1_2_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp38-abi3-musllinux_1_2_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0-cp38-abi3-musllinux_1_2_aarch64.whl
Algorithm Hash digest
SHA256 4fc74352a17015bd0ee90038bc9efe38db894cde45f268b6712b04fce8cd0acb
MD5 ea5c6ce8cf1a1d622ff5a25257840265
BLAKE2b-256 d13a4f4f2301ade26e404462d3336fa11f7958d914cabbabdd6e03c3c5d5658c

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp38-abi3-musllinux_1_2_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp38-abi3-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0-cp38-abi3-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 0e6e21fa3cdfcdcd76748564bf593870a5e013f47d97cf10aed63aa222cff5b7
MD5 44c462825e8f15ab7415f7a1cbf97bd3
BLAKE2b-256 9a691f0cbc2fb22ae6082d094f743d1b8945a3f36f6089cb95f42b7ee348cda7

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp38-abi3-manylinux_2_28_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 d62671bb130879cef0ee4c9ebe47a14af6c66ec53e6d84dc15936e5ffdfac82f
MD5 8b996bcfd15a12d7fd64caf229fddf48
BLAKE2b-256 674ea28359bf1c1ecf11eba22123168c138698f7cb576ac678f5a2e16cd5da08

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp38-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0-cp38-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f0906082d9932ae0c0057fa194041c22b4e2cdb46b2592ef3b91f020d62a081a
MD5 a5153f3a125bfef9ba95e9e8fab78697
BLAKE2b-256 4b6955b8dcf636142ae660fec1869fcac14c4da2e8412e14d6eee1523be77e9f

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp38-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.6.0-cp38-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.6.0-cp38-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 633dc0cd71d32da58ab8c03ad38e2fac452c15c2b0a2866ebf6ededfe0a5061d
MD5 0fdb749831799cb56fe1b848b660885b
BLAKE2b-256 a2507afa2c9c787405864fc47a0d1bbc02c62e9101947ed43c1f43899fc7d91d

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.6.0-cp38-abi3-macosx_10_12_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page