Skip to main content

Finetuner allows one to tune the weights of any deep neural network for better embedding on search tasks.

Project description

Finetuner logo: Finetuner allows one to finetune any deep Neural Network for better embedding on search tasks. It accompanies Jina to deliver the last mile of performance-tuning for neural search applications.

Finetuning any deep neural network for better embedding on neural search tasks

Python 3.7 3.8 3.9 PyPI

Finetuner allows one to tune the weights of any deep neural network for better embedding on search tasks. It accompanies Jina to deliver the last mile of performance-tuning for neural search applications.

🔱 Powerful yet intuitive: all you need is finetuner.fit(), a one-liner that unlocks rich features such as siamese/triplet network, interactive labeling, layer trimming, weights freezing, dimensionality reduction.

⚛️ Framework-agnostic: promise an identical user experience on Pytorch, Keras or PaddlePaddle deep learning backends.

🧈 Jina integration: buttery smooth integration with Jina, reducing the cost of context-switch between experimenting and production.

Install

Make sure you have Python 3.7+ and one of Pytorch, Keras or PaddlePaddle installed on Linux/MacOS.

pip install finetuner

Documentation

Usage

🪄 Usage Do you have an embedding model?
Yes No
Do you have labeled data? Yes 1️⃣ 2️⃣
No 3️⃣ 4️⃣

1️⃣ Have embedding model and labeled data

Perfect! Now embed_model and train_data are given by you already, simply do:

import finetuner 

finetuner.fit(
    embed_model,
    train_data=train_data
)

2️⃣ Have embedding model and unlabeled data

You have an embed_model to use, but no labeled data for finetuning this model. No worry, that's good enough already! You can use Finetuner to interactive label data and train embed_model as below:

import finetuner

finetuner.fit(
    embed_model,
    train_data=unlabeled_data,
    interactive=True
)

3️⃣ Have general model and labeled data

You have a general_model which does not output embeddings. Luckily you provide some labeled_data for training. No worry, Finetuner can convert your model into an embedding model and train it via:

import finetuner

finetuner.fit(
    general_model,
    train_data=labeled_data,
    to_embedding_model=True,
    output_dim=100
)

4️⃣ Have general model and unlabeled data

You have a general_model which is not for embeddings. Meanwhile, you don't have labeled data for training. But no worries, Finetuner can help you train an embedding model with interactive labeling on-the-fly:

import finetuner

finetuner.fit(
    general_model,
    train_data=labeled_data,
    interactive=True,
    to_embedding_model=True,
    output_dim=100
)

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

finetuner-0.0.3.dev37.tar.gz (34.9 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page