A simple and 'tiny' implementation of many multimodal models
Project description
Tiny Multimodal
A simple and "tiny" implementation of many multimodal models. It supports training/finetuning/deploying these tiny-sized models. Unlike the popular "large" models, all the models in this repo will be restricted to train on my RTX 3080 Ti so the implementation will not be totally the same to the original papers.
quick start
create environment
conda create -n tinym python=3.12
conda activate tinym
git clone git@github.com:RobinDong/tiny_multimodal.git
cd tiny_multimodal
python -m pip install -r requirements.txt
prepare dataset for training
Download conceptual-12m from Huggingface to directory cc12m-wds.
Use utils/extract_tars.py to convert CC12M to ready-to-use format:
python utils/extract_tars.py --input_path=<YOUR_DIR>/cc12m-wds/ --output_path=<YOUR_OUTPUT_PATH> --jobs=<YOUR_CPU_CORES>
train
python train.py --provider CLIP
acknowledgements
This repo is still in developing. Please be patient for more multi-modal models.
Any issue or pull request is welcome.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file tinymm-0.10.tar.gz.
File metadata
- Download URL: tinymm-0.10.tar.gz
- Upload date:
- Size: 13.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.10.13
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 64e663c2bb0b04de5a25389d67058eeaab2a6bef576d6ae389ae8099156db8f0 |
|
| MD5 | ca672fa2abce8d41856c4bde15ffa595 |
|
| BLAKE2b-256 | 172d6d1fce2148abc9edb703f558f6ba6263a75936ec991092c3bfec0b321b66 |
File details
Details for the file tinymm-0.10-py3-none-any.whl.
File metadata
- Download URL: tinymm-0.10-py3-none-any.whl
- Upload date:
- Size: 16.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.10.13
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 3b40a4226fdb3a6ae68883cdaec2c23f3665330aa5e4f3ed1bf1f22080551c91 |
|
| MD5 | 5365e33ce3217f07efeee93266448eee |
|
| BLAKE2b-256 | bc3447052fe0c98c247104b54d05a49ddfd89c3732a1c8bfa9f90ad565fb165f |
