Skip to main content

A Python client for Monster API v2

Project description

Monsterapi v2

A Python client for interacting with Monster API v2 in .

Installation

pip install monsterapi

Note: For detailed documentation please visit here

Has support to following MonsterAPI services:

Beta Next Gen LLM Services

Supported Models:
1. "TinyLlama/TinyLlama-1.1B-Chat-v1.0", 
2. "microsoft/phi-2"
3. "mistralai/Mistral-7B-Instruct-v0.2" 
4. "HuggingFaceH4/zephyr-7b-beta" 
Highlights:
1. 99% Cheaper 
2. Synchronous results
3. Token based Pricing. 

Service Introduction: here API Reference: here Usage Guide: here

GA LLM Models Old Generation

Supported Models:
1. falcon-7b-instruct
2. mpt-7b-instruct
3. llama2-7b-chat
4. falcon-40b-instruct
5. mpt-30b-instruct
6. codellama-13b-instruct
7. zephyr-7b-beta
8. Monster Deploy LLMs (deploy-llm)

Detailed payloads of models that are supported ? here

  1. QuickServe API: New service from monsterapi deploy popular LLM models into monsterapi compute infrastructure with one request.

    a. How to use client to launch and manage a quickserve deployment ? here

Additional Information link: here

Code Documentation:

Client module code documentation can be found here

Basic Usage to access Hosted AI-Models

Import Module

from monsterapi import client

set MONSTER_API_KEY env variable to your API key.

os.environ["MONSTER_API_KEY"] = <your_api_key>
client = client() # Initialize client

or

pass api_key parameter to client constructor.

client = client(<api_key>) # pass api_key as parameter

Use generate method

result = client.generate(model='falcon-7b-instruct', data={
    "prompt": "Your prompt here",
    # ... other parameters
})

Quick Serve LLM

Launch a llama2-7b model using QuickServe API

Prepare and send payload to launch a LLM deployment. a. Choose Per_GPU_VRAM and GPU_Count based on your model size and batch size. Please see here for detailed list of supported model and infrastructure matrix.

launch_payload = {
    "basemodel_path": "meta-llama/Llama-2-7b-chat",
    "loramodel_path": "",
    "prompt_template": "{prompt}{completion}",
    "api_auth_token": "b6a97d3b-35d0-4720-a44c-59ee33dbc25b",
    "per_gpu_vram": 24,
    "gpu_count": 1
}

# Launch a deployment
ret = client.deploy("llm", launch_payload) 
deployment_id = ret.get("deployment_id")
print(ret)

# Get deployment status
status_ret = client.get_deployment_status(deployment_id)
print(status_ret)

logs_ret = client.get_deployment_logs(deployment_id)
print(logs_ret)

# Terminate Deployment
terminate_return = client.terminate_deployment(deployment_id)
print(terminate_return)

Run tests

Install test dependencies

pip install monsterapi[tests]

Run functional tests involving actual API key

export MONSTER_API_KEY=<your_api_key>
python3 -m pytest tests/ # Run all tests includes functional tests using actual API key

Run unit tests

export MONSTER_API_KEY="dummy"
python3 -m pytest tests/ -m "not slow" # Run only unit tests

PIP package push Instructions

pip install --upgrade setuptools wheel

python setup.py sdist bdist_wheel

pip install twine

twine upload dist/*

LLama Index CLient Usage

pip install llama_index llama-index-core llama-parse llama-index-readers-file 
from monsterapi.LLamaIndexClient import MonsterLLM

model = "meta-llama/Meta-Llama-3-8B-Instruct"
llm = MonsterLLM(
        model=model, temperature=0.1, max_tokens=256
    )

see examples/llama_index_and_chainlit/ for detailed chainlit example

About us

Check us out at monsterapi.ai

Checkout our new MonsterAPI Deploy service here

Check out new no-code finetuning service here

Checkout our Monster-SD Stable Diffusion v1.5 vs XL Comparison space here

Checkout our Monster API LLM comparison space here

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

monsterapi-1.0.9.3.tar.gz (30.0 kB view details)

Uploaded Source

Built Distribution

monsterapi-1.0.9.3-py3-none-any.whl (29.7 kB view details)

Uploaded Python 3

File details

Details for the file monsterapi-1.0.9.3.tar.gz.

File metadata

  • Download URL: monsterapi-1.0.9.3.tar.gz
  • Upload date:
  • Size: 30.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.8.18

File hashes

Hashes for monsterapi-1.0.9.3.tar.gz
Algorithm Hash digest
SHA256 7c592b5c13001f8633b9bc2e15b7aae749cb6104c48037e8a2064b98a336cf07
MD5 6019e830796fcfa7b437489cc5872f52
BLAKE2b-256 cc8ae66ad6d0f4986e246dcadad92413f7cf7596685ca3cb580ac2d437f5059f

See more details on using hashes here.

File details

Details for the file monsterapi-1.0.9.3-py3-none-any.whl.

File metadata

  • Download URL: monsterapi-1.0.9.3-py3-none-any.whl
  • Upload date:
  • Size: 29.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.8.18

File hashes

Hashes for monsterapi-1.0.9.3-py3-none-any.whl
Algorithm Hash digest
SHA256 078a6a1b13268ecd046177f45a1162b56540368b10c397f61fd3dda3ece91a89
MD5 a4ef3a73cb8dc5946075e45af8cb034d
BLAKE2b-256 462c40089909b0593d32d50446feb6344e6829115a5cf2a1d900ae679d385613

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page