A Python client for Monster API v2
Project description
Monsterapi v2
A Python client for interacting with Monster API v2 in .
Installation
pip install monsterapi
Note: For detailed documentation please visit here
Has support to following MonsterAPI services:
Beta Next Gen LLM Services
Supported Models:
1. "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
2. "microsoft/phi-2"
3. "mistralai/Mistral-7B-Instruct-v0.2"
4. "HuggingFaceH4/zephyr-7b-beta"
Highlights:
1. 99% Cheaper
2. Synchronous results
3. Token based Pricing.
Service Introduction: here API Reference: here Usage Guide: here
GA LLM Models Old Generation
Supported Models:
1. falcon-7b-instruct
2. mpt-7b-instruct
3. llama2-7b-chat
4. falcon-40b-instruct
5. mpt-30b-instruct
6. codellama-13b-instruct
7. zephyr-7b-beta
8. Monster Deploy LLMs (deploy-llm)
Detailed payloads of models that are supported ? here
-
QuickServe API: New service from monsterapi deploy popular LLM models into monsterapi compute infrastructure with one request.
a. How to use client to launch and manage a quickserve deployment ? here
Additional Information link: here
Code Documentation:
Client module code documentation can be found here
Basic Usage to access Hosted AI-Models
Import Module
from monsterapi import client
set MONSTER_API_KEY
env variable to your API key.
os.environ["MONSTER_API_KEY"] = <your_api_key>
client = client() # Initialize client
or
pass api_key
parameter to client constructor.
client = client(<api_key>) # pass api_key as parameter
Use generate method
result = client.generate(model='falcon-7b-instruct', data={
"prompt": "Your prompt here",
# ... other parameters
})
Quick Serve LLM
Launch a llama2-7b model using QuickServe API
Prepare and send payload to launch a LLM deployment. a. Choose Per_GPU_VRAM and GPU_Count based on your model size and batch size. Please see here for detailed list of supported model and infrastructure matrix.
launch_payload = {
"basemodel_path": "meta-llama/Llama-2-7b-chat",
"loramodel_path": "",
"prompt_template": "{prompt}{completion}",
"api_auth_token": "b6a97d3b-35d0-4720-a44c-59ee33dbc25b",
"per_gpu_vram": 24,
"gpu_count": 1
}
# Launch a deployment
ret = client.deploy("llm", launch_payload)
deployment_id = ret.get("deployment_id")
print(ret)
# Get deployment status
status_ret = client.get_deployment_status(deployment_id)
print(status_ret)
logs_ret = client.get_deployment_logs(deployment_id)
print(logs_ret)
# Terminate Deployment
terminate_return = client.terminate_deployment(deployment_id)
print(terminate_return)
Run tests
Install test dependencies
pip install monsterapi[tests]
Run functional tests involving actual API key
export MONSTER_API_KEY=<your_api_key>
python3 -m pytest tests/ # Run all tests includes functional tests using actual API key
Run unit tests
export MONSTER_API_KEY="dummy"
python3 -m pytest tests/ -m "not slow" # Run only unit tests
PIP package push Instructions
pip install --upgrade setuptools wheel
python setup.py sdist bdist_wheel
pip install twine
twine upload dist/*
LLama Index CLient Usage
pip install llama_index llama-index-core llama-parse llama-index-readers-file
from monsterapi.LLamaIndexClient import MonsterLLM
model = "meta-llama/Meta-Llama-3-8B-Instruct"
llm = MonsterLLM(
model=model, temperature=0.1, max_tokens=256
)
see examples/llama_index_and_chainlit/ for detailed chainlit example
About us
Check us out at monsterapi.ai
Checkout our new MonsterAPI Deploy service here
Check out new no-code finetuning service here
Checkout our Monster-SD Stable Diffusion v1.5 vs XL Comparison space here
Checkout our Monster API LLM comparison space here
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file monsterapi-1.0.9.3.tar.gz
.
File metadata
- Download URL: monsterapi-1.0.9.3.tar.gz
- Upload date:
- Size: 30.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.8.18
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7c592b5c13001f8633b9bc2e15b7aae749cb6104c48037e8a2064b98a336cf07 |
|
MD5 | 6019e830796fcfa7b437489cc5872f52 |
|
BLAKE2b-256 | cc8ae66ad6d0f4986e246dcadad92413f7cf7596685ca3cb580ac2d437f5059f |
File details
Details for the file monsterapi-1.0.9.3-py3-none-any.whl
.
File metadata
- Download URL: monsterapi-1.0.9.3-py3-none-any.whl
- Upload date:
- Size: 29.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.8.18
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 078a6a1b13268ecd046177f45a1162b56540368b10c397f61fd3dda3ece91a89 |
|
MD5 | a4ef3a73cb8dc5946075e45af8cb034d |
|
BLAKE2b-256 | 462c40089909b0593d32d50446feb6344e6829115a5cf2a1d900ae679d385613 |