tinymagiq-ai-gateway 0.1.0

Simple AI Gateway - 100+ LLM providers, one function call

TinyMagiq AI Gateway

Simple AI Gateway wrapping LiteLLM - 100+ providers, one function call.

Install

pip install tinymagiq-ai-gateway

Quick Start

from tinymagiq_ai_gateway import completion

response = completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response["content"])

Switch Providers Instantly

# OpenAI
response = completion("gpt-4", [{"role": "user", "content": "Hi"}])

# Anthropic
response = completion("anthropic/claude-3-opus", [{"role": "user", "content": "Hi"}])

# Azure
response = completion("azure/gpt-4", [{"role": "user", "content": "Hi"}])

# Bedrock
response = completion("bedrock/anthropic.claude-v2", [{"role": "user", "content": "Hi"}])

Features

Caching (Save Money)

from tinymagiq_ai_gateway import enable_cache, completion

enable_cache(ttl=3600)  # Cache for 1 hour
response = completion("gpt-4", [{"role": "user", "content": "Hello"}])
# Second identical call returns instantly from cache

Fallback (Increase Reliability)

from tinymagiq_ai_gateway import with_fallback

response = with_fallback(
    models=["gpt-4", "claude-3-opus", "gpt-3.5-turbo"],
    messages=[{"role": "user", "content": "Hello"}]
)
# Tries gpt-4 first, falls back to claude-3 if it fails

Load Balancing (Distribute Traffic)

from tinymagiq_ai_gateway import with_load_balance

router = with_load_balance(
    deployments=[
        {"model_name": "gpt4-east", "litellm_params": {"model": "gpt-4"}},
        {"model_name": "gpt4-west", "litellm_params": {"model": "gpt-4"}},
    ],
    strategy="least-busy"
)
response = router.completion(
    model="gpt4-east",
    messages=[{"role": "user", "content": "Hello"}]
)

Guardrails (Block Harmful Content)

from tinymagiq_ai_gateway import add_block_pattern, check_content

add_block_pattern("ssn", r"\d{3}-\d{2}-\d{4}")
add_block_pattern("credit_card", r"\d{4}-\d{4}-\d{4}-\d{4}")

if check_content("My SSN is 123-45-6789"):
    print("Safe to process")
else:
    print("Blocked - contains sensitive data")

Smart Routing (Optimize Cost/Latency)

from tinymagiq_ai_gateway import smart_route

router = smart_route(
    models=[
        {"model_name": "expensive", "litellm_params": {"model": "gpt-4"}},
        {"model_name": "cheap", "litellm_params": {"model": "gpt-4o-mini"}},
    ],
    goal="cost"  # Always pick cheapest
)

Logging (Track Usage)

from tinymagiq_ai_gateway import enable_logging

enable_logging(providers=["console"])
# All API calls are now logged

Available Functions

Function	Description
completion()	Generate text from any LLM
acompletion()	Async version of completion()
embedding()	Convert text to vector
embedding_batch()	Convert multiple texts to vectors
with_fallback()	Try multiple models if one fails
awith_fallback()	Async version of with_fallback()
enable_cache()	Cache responses to save money
disable_cache()	Turn off caching
clear_cache()	Delete cached responses
cache_status()	Check if caching is enabled
with_load_balance()	Distribute requests across deployments
add_block_pattern()	Block content matching regex
remove_block_pattern()	Remove a block rule
check_content()	Check if content is safe
find_matches()	Find blocked patterns in text
clear_patterns()	Remove all block rules
list_patterns()	List configured block rules
smart_route()	Route to cheapest/fastest model
enable_logging()	Log API calls
disable_logging()	Turn off logging
redact_messages()	Hide message content from logs
unredact_messages()	Show message content in logs

License

MIT License - see License for details.

Powered by LiteLLM (MIT License).