Fast transfer of large files with the Hugging Face Hub.

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for chaumond from gravatar.com chaumond Avatar for hf-xet from gravatar.com hf-xet Avatar for rajatarya1 from gravatar.com rajatarya1 Avatar for seanses from gravatar.com seanses Avatar for xcid from gravatar.com xcid

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers
Report project as malware

Project description

License GitHub release Contributor Covenant

🤗 hf-xet - xet client tech, used in huggingface_hub

Welcome

hf-xet enables huggingface_hub to utilize xet storage for uploading and downloading to HF Hub. Xet storage provides chunk-based deduplication, efficient storage/retrieval with local disk caching, and backwards compatibility with Git LFS. This library is not meant to be used directly, and is instead intended to be used from huggingface_hub.

Key features

♻ chunk-based deduplication implementation: avoid transferring and storing chunks that are shared across binary files (models, datasets, etc).

🤗 Python bindings: bindings for huggingface_hub package.

↔ network communications: concurrent communication to HF Hub Xet backend services (CAS).

🔖 local disk caching: chunk-based cache that sits alongside the existing huggingface_hub disk cache.

Installation

Install the hf_xet package with pip:

pip install hf_xet

Quick Start

hf_xet is not intended to be run independently as it is expected to be used from huggingface_hub, so to get started with huggingface_hub check out the documentation here.

Contributions (feature requests, bugs, etc.) are encouraged & appreciated 💙💚💛💜🧡❤️

Please join us in making hf-xet better. We value everyone's contributions. Code is not the only way to help. Answering questions, helping each other, improving documentation, filing issues all help immensely. If you are interested in contributing (please do!), check out the contribution guide for this repository.

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for chaumond from gravatar.com chaumond Avatar for hf-xet from gravatar.com hf-xet Avatar for rajatarya1 from gravatar.com rajatarya1 Avatar for seanses from gravatar.com seanses Avatar for xcid from gravatar.com xcid

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

This version

1.5.2

1.5.2rc1 pre-release

1.5.2rc0 pre-release

1.5.1

1.5.1rc1 pre-release

1.5.1rc0 pre-release

1.5.0

1.4.3

1.4.2

1.4.1

1.4.0

1.3.2

1.3.1

1.3.0

1.2.1 yanked

Reason this release was yanked:

This release contains a regression that enables the disk cache by default.

1.2.0

1.1.10

1.1.9

1.1.8

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.5

1.0.4

1.0.3

1.0.2

1.0.1

1.0.0

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hf_xet-1.5.2.tar.gz (903.7 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

hf_xet-1.5.2-cp314-cp314t-win_arm64.whl (3.8 MB view details)

Uploaded CPython 3.14tWindows ARM64

hf_xet-1.5.2-cp314-cp314t-win_amd64.whl (4.0 MB view details)

Uploaded CPython 3.14tWindows x86-64

hf_xet-1.5.2-cp314-cp314t-musllinux_1_2_x86_64.whl (4.6 MB view details)

Uploaded CPython 3.14tmusllinux: musl 1.2+ x86-64

hf_xet-1.5.2-cp314-cp314t-musllinux_1_2_aarch64.whl (4.4 MB view details)

Uploaded CPython 3.14tmusllinux: musl 1.2+ ARM64

hf_xet-1.5.2-cp314-cp314t-manylinux_2_28_aarch64.whl (4.2 MB view details)

Uploaded CPython 3.14tmanylinux: glibc 2.28+ ARM64

hf_xet-1.5.2-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.4 MB view details)

Uploaded CPython 3.14tmanylinux: glibc 2.17+ x86-64

hf_xet-1.5.2-cp314-cp314t-macosx_11_0_arm64.whl (3.8 MB view details)

Uploaded CPython 3.14tmacOS 11.0+ ARM64

hf_xet-1.5.2-cp314-cp314t-macosx_10_12_x86_64.whl (4.1 MB view details)

Uploaded CPython 3.14tmacOS 10.12+ x86-64

hf_xet-1.5.2-cp38-abi3-win_arm64.whl (3.8 MB view details)

Uploaded CPython 3.8+Windows ARM64

hf_xet-1.5.2-cp38-abi3-win_amd64.whl (4.0 MB view details)

Uploaded CPython 3.8+Windows x86-64

hf_xet-1.5.2-cp38-abi3-musllinux_1_2_x86_64.whl (4.6 MB view details)

Uploaded CPython 3.8+musllinux: musl 1.2+ x86-64

hf_xet-1.5.2-cp38-abi3-musllinux_1_2_aarch64.whl (4.4 MB view details)

Uploaded CPython 3.8+musllinux: musl 1.2+ ARM64

hf_xet-1.5.2-cp38-abi3-manylinux_2_28_aarch64.whl (4.2 MB view details)

Uploaded CPython 3.8+manylinux: glibc 2.28+ ARM64

hf_xet-1.5.2-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.4 MB view details)

Uploaded CPython 3.8+manylinux: glibc 2.17+ x86-64

hf_xet-1.5.2-cp38-abi3-macosx_11_0_arm64.whl (3.8 MB view details)

Uploaded CPython 3.8+macOS 11.0+ ARM64

hf_xet-1.5.2-cp38-abi3-macosx_10_12_x86_64.whl (4.0 MB view details)

Uploaded CPython 3.8+macOS 10.12+ x86-64

File details

Details for the file hf_xet-1.5.2.tar.gz.

File metadata

  • Download URL: hf_xet-1.5.2.tar.gz
  • Upload date:
  • Size: 903.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.5.2.tar.gz
Algorithm Hash digest
SHA256 73044bd31bae33c984af832d19c752a0dffb67518fee9ddbd91d616e1101cf47
MD5 18180ecc86f13143b29657c5345f36a7
BLAKE2b-256 633967be8d71f900d9a55761b6022821d6679fb56c64f1b6063d5af2c2606727

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2.tar.gz:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp314-cp314t-win_arm64.whl.

File metadata

  • Download URL: hf_xet-1.5.2-cp314-cp314t-win_arm64.whl
  • Upload date:
  • Size: 3.8 MB
  • Tags: CPython 3.14t, Windows ARM64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.5.2-cp314-cp314t-win_arm64.whl
Algorithm Hash digest
SHA256 d6f9c58549407b84b9a5383afd68db0acc42345326a3159990b36a5ca8a20e4e
MD5 56f53122d74a54e5eda3d90e225ec90f
BLAKE2b-256 ea09b2cdf2a0fb39a08af3222b96092a36bd3b40c54123eef07de4422e870971

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp314-cp314t-win_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp314-cp314t-win_amd64.whl.

File metadata

  • Download URL: hf_xet-1.5.2-cp314-cp314t-win_amd64.whl
  • Upload date:
  • Size: 4.0 MB
  • Tags: CPython 3.14t, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.5.2-cp314-cp314t-win_amd64.whl
Algorithm Hash digest
SHA256 fd3add255549e8ef58fa35b2e42dc016961c050600444e7d77d030ba6b57120e
MD5 60ade6060cdc7ab69e6c6b5a14acb204
BLAKE2b-256 e94bcc682832de4264a03880a2d1b5ec3e1fab3bf307f508817250baafdb9996

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp314-cp314t-win_amd64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp314-cp314t-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.5.2-cp314-cp314t-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 e396ab0faf6298199ad7a95305c3ca8498cb825978a6485be6d00587ee4ec577
MD5 b859e09e58947de63f403f11c33788a3
BLAKE2b-256 7bd99e33981173dbaf194ba0015202b02d467b624d44d4eba89e1bf06c0d2995

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp314-cp314t-musllinux_1_2_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp314-cp314t-musllinux_1_2_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.5.2-cp314-cp314t-musllinux_1_2_aarch64.whl
Algorithm Hash digest
SHA256 bee28c619622d36968056532fd49cf2b35ca75099b1d616c31a618a893491380
MD5 15c070233dd966bbf6e98d0edafea73e
BLAKE2b-256 0c33a968f4e4535037b36941ec00714625fb60e026302407e7e26ca9f3e65f4e

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp314-cp314t-musllinux_1_2_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp314-cp314t-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.5.2-cp314-cp314t-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 580e59e29bf37aece1f2b68537de1e3fb04f43a23d910dcf6f128280b5bfbba4
MD5 7ad901e6142c9d365bd14a347e6835f1
BLAKE2b-256 d9588343f3cb63c8fa058d576136df3871550f7d5214a8f048a7ea2eab6ac906

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp314-cp314t-manylinux_2_28_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.5.2-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 8d7446f72abbf7e01ca5ff131786bc2e74a56393462c17a6bf1e303fbab81db4
MD5 ce40c2a60ce2bb5fceccf417dca75359
BLAKE2b-256 22d72658ac0a5b9f4664ca27ce31bd015044fe9dea50ed455fb5197aba819c11

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp314-cp314t-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for hf_xet-1.5.2-cp314-cp314t-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 8764488197c1d7b1378c8438c18d2eea902e150dbca0b0f0d2d32603fb9b5576
MD5 ce2440c2aa90293bd4dff1fa1cd12628
BLAKE2b-256 c53f699749dd78442480eda4e4fca494284b0e3542e4063cc37654d5fdc929e6

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp314-cp314t-macosx_11_0_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp314-cp314t-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.5.2-cp314-cp314t-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 4a5ecb9cda8512ba2aa8ee5d37c87a1422992165892d653098c7b90247481c3b
MD5 7c77f6159f000f138c72283bcf5e4952
BLAKE2b-256 29be525eabac5d1736b679c39e342ecd4292534012546a2d18f0043c8e3b6021

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp314-cp314t-macosx_10_12_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp38-abi3-win_arm64.whl.

File metadata

  • Download URL: hf_xet-1.5.2-cp38-abi3-win_arm64.whl
  • Upload date:
  • Size: 3.8 MB
  • Tags: CPython 3.8+, Windows ARM64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.5.2-cp38-abi3-win_arm64.whl
Algorithm Hash digest
SHA256 1da28519496eb7c8094c11e4d25509b4a468457a0302d58136099db2fd9a671d
MD5 7ddf7c5f29da6eff1b5d4945fda2358a
BLAKE2b-256 39c6988383e9dc17294d536fcbcd6fd16eed882e411ad16c954984a53e47b09c

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp38-abi3-win_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp38-abi3-win_amd64.whl.

File metadata

  • Download URL: hf_xet-1.5.2-cp38-abi3-win_amd64.whl
  • Upload date:
  • Size: 4.0 MB
  • Tags: CPython 3.8+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for hf_xet-1.5.2-cp38-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 ecf63d1cb69a9a7319910f8f83fcf9b46e7a32dfcf4b8f8eeddb55f647306e65
MD5 d2857c78eae15c466a22500a753a4174
BLAKE2b-256 e9f868b01c5c2edb56ac9a67b3d076ffddcb90867abaee923923eb34e7a14e76

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp38-abi3-win_amd64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp38-abi3-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.5.2-cp38-abi3-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 cde8cd167126bb6109b2ceb19b844433a4988643e8f3e01dd9dd0e4a34535097
MD5 cafd222933606abeb1ebb3cf48191338
BLAKE2b-256 07a9c050bc2743a2bcd68928bfee157b08681667a164a24ec95fbfcfcd717e08

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp38-abi3-musllinux_1_2_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp38-abi3-musllinux_1_2_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.5.2-cp38-abi3-musllinux_1_2_aarch64.whl
Algorithm Hash digest
SHA256 6395cfe3c9cbead4f16b31808b0e67eac428b66c656f856e99636adaddea878f
MD5 59d1a3278dd327975ad2512210d49edc
BLAKE2b-256 9e05ae50f0d34e3254e6c3e208beb2519f6b8673016fc4b3643badaf6450d186

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp38-abi3-musllinux_1_2_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp38-abi3-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for hf_xet-1.5.2-cp38-abi3-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 7db73c810500c54c6760be8c39d4b2e476974de85424c50063efc22fdda13025
MD5 e8f80475db5ad760e4e821754acbb093
BLAKE2b-256 af6b832dd980af4b0c3ae0660e309285f2ffcdff2faa38129390dbb47aa4a3f9

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp38-abi3-manylinux_2_28_aarch64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.5.2-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 db78c39c83d6279daddc98e2238f373ab8980685556d42472b4ec51abcf03e8c
MD5 0f946e0bc033c931e2abd9c238e04aad
BLAKE2b-256 d335db860aa3a0780660324a506ad4b3d322ddc6ecbba4b9340aed0942cbf21c

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp38-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for hf_xet-1.5.2-cp38-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 045f84440c55cdeb659cf1a1dd48c77bcd0d2e93632e2fea8f2c3bdee79f38ed
MD5 861c7a99382317cc30bd4dd56ccafe58
BLAKE2b-256 60acb097a86a1e4a6098f3a79382643ab09d5733d87ccc864877ad1e12b49b70

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp38-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file hf_xet-1.5.2-cp38-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for hf_xet-1.5.2-cp38-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 f922b8f5fb84f1dd3d7ab7a1316354a1bca9b1c73ecfc19c76e51a2a49d29799
MD5 906c8d65521e3eb006028db4c1eccfea
BLAKE2b-256 deba2b70603c7552db82baeb2623e2336898304a17328845151be4fe1f48d420

See more details on using hashes here.

Provenance

The following attestation bundles were made for hf_xet-1.5.2-cp38-abi3-macosx_10_12_x86_64.whl:

Publisher: release.yml on huggingface/xet-core

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page