bagelML 0.0.3

BagelML is a library for Bagel's finetuning API.

Bagel Python Client 🥯

Welcome to the Bagel Python Client! Bagel is your platform for peer-to-peer machine learning, fine-tuning open-source models like Llama or Mistral, and using retrieval augmented generation (RAG).

One of the perks? No need to manage complex embeddings or model integrations yourself! The Bagel client handles these processes, saving you time and money. 🥯

Table of Contents

1. Prerequisites
2. Installation
3. Import the necessary modules
4. Define the Bagel server settings
5. Create the Bagel client
6. Ping the Bagel server
7. Get the Bagel server version
8. Create an Asset
9. Delete an Asset
10. Download Model Files
11. Query Asset
12. Update Asset
13. Download file
14. File Upload
15. Fine-tune
16. Get all assets
17. Get all assets by Id
18. Get finetuned model
19. Get job by job id
20. Get job
21. List Job
22. Add data to asset

Prerequisites

• Python 3.6+
• pip package manager
• Asset size limit 500MB (*Create a new issue if you want to increase the limit)

Installation

To install the Bagel Python client, run the following command in your terminal:

pip install bagelML

Import the necessary modules

import uuid
import os
import bagel
from dotenv import load_dotenv

This snippet imports the required modules for using Bagel.

Create the Bagel client

client = bagel.Client()

Create an instance of the Bagel client using the previously defined server settings.

Ping the Bagel server

print(client.ping())

This checks the connectivity to the Bagel server.

Get the Bagel server version

print(client.get_version())

Retrieves and prints the version of the Bagel server.

Create an Asset

Assets in Bagel serve as powerful containers for large datasets, encapsulating embeddings — high-dimensional vectors that represent various data forms, such as text, images, or audio. These Assets enable efficient similarity searches, which are fundamental to a wide range of applications, from recommendation systems and search engines to data analytics tools.

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

payload = {
    "dataset_type": "RAW",
    "title": "",
    "category": "",
    "details": "Testing",
    "tags": ["AI", "DEMO", "TEST"],
    "user_id": 'insert user id'
}

response = client.create_asset(payload, api_key)
print(response)

Note: This code prompts the user to enter their API key securely (without displaying it on the screen) and then sets it as an environment variable named BAGEL_API_KEY

Delete an Asset

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

dataset_id = 'insert dataset/asset id'

response = client.delete_asset(dataset_id, api_key)
print(response)

This method deletes a specific Asset.

Query Asset

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

asset_id = ""

payload = {
    "where": {
        # "category": "Cat2",
    },
    "where_document": {
        # "is_published": True,
    },
    # "query_embeddings": [em],
    "n_results": 1,
    "include": ["metadatas", "documents", "distances"],
    "query_texts": ["insert query text"],
    "padding": False,
}

response = client.query_asset(asset_id, payload, api_key)
print(response)

Queries a specific Asset with detailed parameters.

Update Asset

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

asset_id = ""

payload = {
   "title": "Updated dataset title",
    "category": "Updated category",
    "details": "Updated dataset description.",
    "tags": ["Updated", "Tags"]
}

response = client.update_asset(asset_id, payload, api_key)
print(response)

Updates the details of an existing Asset.

Download file

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

asset_id = ""
file_name = ""

response = client.download_file(asset_id, file_name, api_key)
print(response)

Downloads a specific file from an Asset.

File Upload

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

dataset_id = ""
file_path = ""

response = client.file_upload(file_path, dataset_id, api_key)
print(response)

Uploads a file to a specific Asset.

Fine-tune

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

user_id = "" #insert user id
asset_id = "" #insert RAW asset id containing dataset 
base_model = "" # model id of purchased base model from marketplace 
file_name = "" # name of file in raw asset 
title = ""  # choose a title 

response = client.fine_tune(title=title,
                            user_id=user_id,
                            asset_id = asset_id,
                            file_name = file_name,
                            base_model = base_model,
                            epochs = 3,
                            learning_rate = 0.01
                            )
print(response)

Fine-tunes a model using a specific Asset and provided parameters.

Note: Epochs & learning rate are optional. By default, epochs = 3 & learning rate = 0.001. Users can change it according to their requirements.

Get all assets

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

user_Id = ""

response = client.get_all_asset(user_Id, api_key)
print(response)

Retrieves all assets for a specific user.

Get assets by Id

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

asset_id = ""

response = client.get_asset_by_id(asset_id, api_key)
print(response)

Retrieves a specific asset by its ID.

Get finetuned model

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

asset_id = ""
file_name = "train.txt"

# Call the function
response = client.download_file_by_asset_and_name(asset_id, file_name)
print(response)

Downloads a fine-tuned model by asset ID and file name.

Get job by asset id

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

asset_id = "" # Replace with asset ID of fine-tuned model. e.g "eR57yd73b-7gt4-4a71-b0fd-afKJUIY6578Ge" 

# Function to get job by asset

# Call the function
response = client.get_job_by_asset_id(asset_id, api_key)
print(response)

Retrieves the status of a specific job by asset ID.

Get job

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

job_id = ""  # Replace with the actual job ID

response = client.get_job(job_id, api_key)
print(response)

Retrieves details of a job.

List Job

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

user_id = ""

# Call the function
response = client.list_jobs(user_id, api_key)
print(response)

Lists all jobs for a specific user.

Add data to asset

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

payload = {
  "metadatas": [{ "source": "testing" }],
  "documents": ["Hi man"],
  "ids": ["xxxx-xxxx-xxxx-xxxx--xxxxx"], #manually generated by you
}

response = client.add_data_to_asset(asset_id, payload, api_key)
print(response)

Adds data to an existing asset.

Download Finetuned Model

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

asset_id = "insert asset id"

response = client.download_model(asset_id, api_key)
print(response)

Buy Asset

import os
import bagel
from getpass import getpass

# Create the asset using the API key from environment variables
client = bagel.Client()

# Copy & Paste the API Key from https://bakery.bagel.net/api-key
DEMO_KEY_IN_USE = getpass("Enter your API key: ")

# Set environment variable
api_key = os.environ['BAGEL_API_KEY'] = DEMO_KEY_IN_USE

asset_id = "insert asset id"
user_id = "insert userid"

response = client.buy_asset(asset_id, user_id, api_key)     
print(response)