Skip to main content

Convert LaBSE model from TensorFlow to PyTorch.

Project description

LaBSE

Project

This project is an implementation to convert Google's LaBSE model from TensorFlow to PyTorch. It also offers extensions to convert the smaller-LaBSE model from TensorFlow to PyTorch, and the LEALLA family of models.

The models are uploaded to the HuggingFace Model Hub in the PyTorch HF-compatible (original and safetensors), TensorFlow and Flax formats, alongwith a compatible tokenizer.

Export

To convert and export the models:

poetry install
poetry run convert_labse --output_path /path/to/models

To update the models on the HuggingFace Model Hub:

# Clone the already uploaded models.
cd /path/to/model
git clone https://huggingface.co/setu4993/LaBSE.git

# Export models anew and update.
cd /path/to/repo
poetry install
poetry run convert_labse --output_path /path/to/models/LaBSE --huggingface_path

Export Commands by Model

  1. LaBSE: poetry run convert_labse --output_path /path/to/models/setu4993/LaBSE --huggingface_path
  2. smaller-LaBSE: poetry run convert_labse --output_path /path/to/models/setu4993/smaller-LaBSE --smaller --huggingface_path
  3. LEALLA-base: poetry run convert_lealla --size base --output_path /path/to/models/setu4993/LEALLA-base --huggingface_path
  4. LEALLA-small: poetry run convert_lealla --size small --output_path /path/to/models/setu4993/LEALLA-small --huggingface_path
  5. LEALLA-large: poetry run convert_lealla --size large --output_path /path/to/models/setu4993/LEALLA-large --huggingface_path

Model Cards

See the model-cards directory for a copy of the model cards.

License

This repository and the conversion code is licensed under the MIT license, but the model is distributed with an Apache-2.0 license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

convert_labse_tf_pt-3.0.0.tar.gz (3.4 MB view details)

Uploaded Source

Built Distribution

convert_labse_tf_pt-3.0.0-py3-none-any.whl (3.5 MB view details)

Uploaded Python 3

File details

Details for the file convert_labse_tf_pt-3.0.0.tar.gz.

File metadata

  • Download URL: convert_labse_tf_pt-3.0.0.tar.gz
  • Upload date:
  • Size: 3.4 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.5.1 CPython/3.10.11 Linux/5.15.0-1038-azure

File hashes

Hashes for convert_labse_tf_pt-3.0.0.tar.gz
Algorithm Hash digest
SHA256 07d5818df35440b03806bf3a79a98a19b1f50e43a52d0d2e17f63dbdbffcc117
MD5 51ae0fc95579ec436aceadb8814a4b79
BLAKE2b-256 979fdaed960aa54cfa25bcffe2052cd9d7b34d179e7d3c0edc8c488b989dcbbd

See more details on using hashes here.

File details

Details for the file convert_labse_tf_pt-3.0.0-py3-none-any.whl.

File metadata

File hashes

Hashes for convert_labse_tf_pt-3.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 00368c3e5b29cdbc3ecb533680bdee9ae1a3987163c48e7bfad79a9e2c20433d
MD5 84e9fdac43acaac6fc4fabcde8347332
BLAKE2b-256 60e5a747d2d6ed1948df6d81e1beccb957e0f38ad9baafbc12d011fe07ddbf54

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page