Skip to main content

Client library to download and publish models on the hub

Project description


Client library to download and publish models and other files on the hub

Build GitHub GitHub release

Do you have an open source ML library? We're looking to partner with a small number of other cool open source ML libraries to provide model hosting + versioning.

Advantages are:

  • versioning is built-in (as hosting is built around git and git-lfs), no lock-in, you can just git clone away.
  • anyone can upload a new model for your library, just need to add the corresponding tag for the model to be discoverable – no more need for a hardcoded list in your code
  • Fast downloads! We use Cloudfront (a CDN) to geo-replicate downloads so they're blazing fast from anywhere on the globe
  • Usage stats and more features to come

Ping us if interested 😎

♻️ Partial list of implementations in third party libraries:

Download files from the hub

Integration inside a library is super simple. We expose two functions, hf_hub_url() and cached_download().


hf_hub_url() takes:

  • a repo id (e.g. a model id like julien-c/EsperBERTo-small i.e. a user or organization name and a repo name, separated by /),
  • a filename (like pytorch_model.bin),
  • and an optional git revision id (can be a branch name, a tag, or a commit hash)

and returns the url we'll use to download the actual files:

If you check out this URL's headers with a HEAD http request (which you can do from the command line with curl -I) for a few different files, you'll see that:

  • small files are returned directly
  • large files (i.e. the ones stored through git-lfs) are returned via a redirect to a Cloudfront URL. Cloudfront is a Content Delivery Network, or CDN, that ensures that downloads are as fast as possible from anywhere on the globe.


cached_download() takes the following parameters, downloads the remote file, stores it to disk (in a versioning-aware way) and returns its local file path.


  • a remote url
  • your library's name and version (library_name and library_version), which will be added to the HTTP requests' user-agent so that we can provide some usage stats.
  • a cache_dir which you can specify if you want to control where on disk the files are cached.

Check out the source code for all possible params (we'll create a real doc page in the future).

Publish models to the hub

Uploading a model to the hub is super simple too:

  • create a model repo directly from the website, at (models can be public or private, and are namespaced under either a user or an organization)
  • clone it with git
  • download and install git lfs if you don't already have it on your machine (you can check by running a simple git lfs)
  • add, commit and push your files, from git, as you usually do.

We are intentionally not wrapping git too much, so that you can go on with the workflow you’re used to and the tools you already know.

👀 To see an example of how we document the model sharing process in transformers, check out

Users add tags into their model cards (e.g. your library_name, a domain tag like audio, etc.) to make sure their models are discoverable.

Documentation about the model hub itself is at

API utilities in

You don't need them for the standard publishing workflow, however, if you need a programmatic way of creating a repo, deleting it (⚠️ caution), or listing models from the hub, you'll find helpers in

We also have an API to query models by specific tags (e.g. if you want to list models compatible to your library)


Those API utilities are also exposed through a CLI:

huggingface-cli login
huggingface-cli logout
huggingface-cli whoami
huggingface-cli repo create

Need to upload large (>5GB) files?

To upload large files (>5GB 🔥), you need to install the custom transfer agent for git-lfs, bundled in this package.

To install, just run:

$ huggingface-cli lfs-enable-largefiles

This should be executed once for each model repo that contains a model file >5GB. If you just try to push a file bigger than 5GB without running that command, you will get an error with a message reminding you to run it.

Finally, there's a huggingface-cli lfs-multipart-upload command but that one is internal (called by lfs directly) and is not meant to be called by the user.

Visual integration into the hub

Finally, we'll implement a few tweaks to improve the UX for your models on the website – let's use Asteroid as an example:


Model authors add an asteroid tag to their model card and they get the advantages of model versioning built-in


We add a custom "Use in Asteroid" button.


When clicked you get a library-specific code sample that you'll be able to specify. 🔥

Feedback (feature requests, bugs, etc.) is super welcome 💙💚💛💜♥️🧡

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for huggingface-hub, version 0.0.6
Filename, size File type Python version Upload date Hashes
Filename, size huggingface_hub-0.0.6-py3-none-any.whl (28.5 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size huggingface_hub-0.0.6.tar.gz (22.8 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page