Convert LaBSE model from TensorFlow to PyTorch.
Project description
LaBSE
Project
This project is an implementation to convert Google's LaBSE model from TensorFlow to PyTorch. It also offers extensions to convert the smaller-LaBSE model from TensorFlow to PyTorch, and the LEALLA family of models.
The models are uploaded to the HuggingFace Model Hub in the PyTorch HF-compatible (original and safetensors
), TensorFlow and Flax formats, alongwith a compatible tokenizer.
Export
To convert and export the models:
poetry install
poetry run convert_labse --output_path /path/to/models
To update the models on the HuggingFace Model Hub:
# Clone the already uploaded models.
cd /path/to/model
git clone https://huggingface.co/setu4993/LaBSE.git
# Export models anew and update.
cd /path/to/repo
poetry install
poetry run convert_labse --output_path /path/to/models/LaBSE --huggingface_path
Export Commands by Model
- LaBSE:
poetry run convert_labse --output_path /path/to/models/setu4993/LaBSE --huggingface_path
- smaller-LaBSE:
poetry run convert_labse --output_path /path/to/models/setu4993/smaller-LaBSE --smaller --huggingface_path
- LEALLA-base:
poetry run convert_lealla --size base --output_path /path/to/models/setu4993/LEALLA-base --huggingface_path
- LEALLA-small:
poetry run convert_lealla --size small --output_path /path/to/models/setu4993/LEALLA-small --huggingface_path
- LEALLA-large:
poetry run convert_lealla --size large --output_path /path/to/models/setu4993/LEALLA-large --huggingface_path
Model Cards
See the model-cards
directory for a copy of the model cards.
License
This repository and the conversion code is licensed under the MIT license, but the model is distributed with an Apache-2.0 license.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file convert_labse_tf_pt-3.0.0.tar.gz
.
File metadata
- Download URL: convert_labse_tf_pt-3.0.0.tar.gz
- Upload date:
- Size: 3.4 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.5.1 CPython/3.10.11 Linux/5.15.0-1038-azure
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 07d5818df35440b03806bf3a79a98a19b1f50e43a52d0d2e17f63dbdbffcc117 |
|
MD5 | 51ae0fc95579ec436aceadb8814a4b79 |
|
BLAKE2b-256 | 979fdaed960aa54cfa25bcffe2052cd9d7b34d179e7d3c0edc8c488b989dcbbd |
File details
Details for the file convert_labse_tf_pt-3.0.0-py3-none-any.whl
.
File metadata
- Download URL: convert_labse_tf_pt-3.0.0-py3-none-any.whl
- Upload date:
- Size: 3.5 MB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.5.1 CPython/3.10.11 Linux/5.15.0-1038-azure
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 00368c3e5b29cdbc3ecb533680bdee9ae1a3987163c48e7bfad79a9e2c20433d |
|
MD5 | 84e9fdac43acaac6fc4fabcde8347332 |
|
BLAKE2b-256 | 60e5a747d2d6ed1948df6d81e1beccb957e0f38ad9baafbc12d011fe07ddbf54 |