bytez 3.0.1

Python API client for Bytez service

API Documentation

• API Documentation
  • Basic Usage
  • Installation
  • Authentication and Getting Your Key
  • List Available Models
  • Initialize the Model API
  • Run a Model
  • Run a Model with HuggingFace Params
  • Stream the Response
  • Request a Huggingface Model Not Yet on Bytez
  • Request a Model Not on Huggingface or Bytez
• Examples snippets
  • Feedback

Basic Usage

from bytez import Bytez

client = Bytez("YOUR_BYTEZ_KEY_HERE")

model = client.model("Qwen/Qwen2-7B-Instruct")

input_text = "Once upon a time there was a beautiful home where"

model_params = {"max_new_tokens": 20, "max_new_tokens": 5, "temperature": 0.5}

result = model.run(input_text, model_params=model_params)

print(result.output)

# Access other properties of the result
error = result.error  # Will be None if no error
provider = result.provider  # if a closed source model, the raw output

Streaming usage (only text-generation models support streaming currently)

from bytez import Bytez

client = Bytez("YOUR_BYTEZ_KEY_HERE")

model = client.model("Qwen/Qwen2-7B-Instruct")

input_text = "Once upon a time there was a beautiful home where"

model_params = {"max_new_tokens": 20, "max_new_tokens": 5, "temperature": 0.5}

stream = model.run(
    input_text,
    stream=True,
    model_params=model_params,
)

for chunk in stream:
    print(f"Output: {chunk}")

Installation

pip install bytez

Authentication and Getting Your Key

To use this API, you need an API key. Obtain your key by visiting the Bytez Settings Page

To then use it in code:

from bytez import Bytez

client = Bytez("YOUR_BYTEZ_KEY_HERE")

List Available Models

Lists the currently available models and provides basic information about each one, such as the RAM required to run an instance.

from bytez import Bytez

client = Bytez("YOUR_BYTEZ_KEY_HERE")

# To list all models
result = client.list.models()

print(result.output)

# To list models by task
result = client.list.models({ "task": "object-detection"})

print(result.output)

Initialize the Model API

Initialize a model, so you can check its status, load, run, or shut it down.

model = client.model("openai-community/gpt2")

Run a Model

Run inference.

result = model.run("Once upon a time there was a")

print(result.output)
# Access error if needed
if result.error:
    print(f"Error occurred: {result.error}")
# Access provider information
print(f"Provider: {result.provider}")

Run a Model with HuggingFace Params

Run inference with HuggingFace parameters.

input_text = "Once upon a time there was a small little man who"

model_params = {"max_new_tokens": 20, "temperature": 2}

result = model.run(input_text, model_params=model_params)

print(result.output)
# Access error if needed
if result.error:
    print(f"Error occurred: {result.error}")

Stream the Response

Note: This is only supported for text-generation models.

input_text = "Once upon a time there was a beautiful home where"

model_params = {"max_new_tokens": 20, "max_new_tokens": 5, "temperature": 0.5}

stream = model.run(
    input_text,
    stream=True,
    model_params=model_params,
)

for chunk in stream:
    print(f"Output: {chunk}")

Request a Huggingface Model Not Yet on Bytez

To request a model that exists on Huggingface but not yet on Bytez, you can do the following:

model_id = "openai-community/gpt2"

job_status = client.process(model_id)

print(job_status)

This sends a job to an automated queue. When the job completes, you'll receive an email indicating the model is ready for use with the models API.

Request a Model Not on Huggingface or Bytez

Please reach out to us and we'll do what's necessary to make other models available!

Please join our Discord or contact us via email at [e-mail]

Examples snippets

Open our docs for Python snippets.

Feedback

We value your feedback to improve our documentation and services. If you have any suggestions, please join our Discord or contact us via email at [e-mail]