everyai-core 1.1.0

Connect to every free AI provider with zero setup

EveryAI Core

Connect to multiple AI providers with zero setup, client-side rate-limit governance, local query caching, automatic fallback failover, and a live web telemetry dashboard.

---

Key Features

• Unified Developer SDK: Query Groq, OpenRouter, Nvidia NIM, Mistral, Cerebras, Cloudflare, and Hugging Face with an identical, clean developer syntax.
• Smart Fallback Orchestration: Chain alternative providers or different keys to handle rate limits and service failures smoothly.
• Local Response Caching: SQLite-backed query cache saves tokens and speeds up repetitive workflows.
• Rate Limit Governor: Client-side RPM (Requests Per Minute) and TPM (Tokens Per Minute) tracking queues and paces outbound requests automatically.
• Telemetry & Web Dashboard: Run a local HTTP dashboard to visualize token counts, response times, and query statistics.
• Local Offline Execution: Load and run open-weights models locally via PyTorch and Transformers using the same API interface.

---

Supported Providers

Use the exact provider names listed below when querying models or configuring presets:

Provider Name (Exact Key)	Environment Variable	Key Features
groq	GROQ_API_KEY	Ultra-fast LPU inference, native token usage logging
openrouter	OPENROUTER_API_KEY	Aggregated model catalog, free tier routes
huggingface	HF_TOKEN or HUGGINGFACE_API_KEY	Serverless API execution of open-weights models
cerebras	CEREBRAS_API_KEY	High-speed wafer-scale engine inference
mistral	MISTRAL_API_KEY	European flagship models, native streaming
cloudflare	CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID	Workers AI serverless endpoints
nvidia	NVIDIA_API_KEY	GPU-optimized microservice inference

---

Installation

Install the core library via pip:

pip install everyai-core

To enable local model inference capabilities (requires PyTorch and Transformers):

pip install everyai-core[local]

---

Quick Start

1. Set Your Environment Variables

EveryAI searches your environment for keys. Add them to your shell or .env file:

export GROQ_API_KEY="gsk_..."
export OPENROUTER_API_KEY="sk-or-..."

2. Basic Non-Streaming Request

from everyai_core import EveryAI

# Initialize client (uses environment keys by default)
client = EveryAI()

# Run a completion
response = client.chat(
    provider="groq",
    model="llama-3.1-8b-instant",
    messages=[
        {"role": "user", "content": "Explain quantum computing in one sentence."}
    ],
    temperature=0.7
)

print(f"[{response.provider.upper()}] {response.choices[0].message['content']}")
print(f"Tokens Used: {response.usage.total_tokens}")

---

Detailed Module & Feature Guide

1. Provider-Specific Direct Clients

You can access provider SDK wrappers directly through client attributes for full autocomplete and static typing support:

from everyai_core import EveryAI

client = EveryAI(api_keys={"groq": "your_custom_key_here"})

# Access Groq direct instance
response = client.groq.chat(
    model="llama-3.1-8b-instant",
    messages=[{"role": "user", "content": "Hi!"}]
)

---

2. Streaming Response Tokens

For real-time responses, pass stream=True to receive a generator yielding chunk responses:

from everyai_core import EveryAI

client = EveryAI()

chunks = client.chat(
    provider="openrouter",
    model="google/gemini-2.5-flash",
    messages=[{"role": "user", "content": "Write a short poem about coding."}],
    stream=True
)

for chunk in chunks:
    content = chunk.choices[0].message.get("content", "")
    print(content, end="", flush=True)

---

3. Fallback Failover

When building AI applications, API providers can fail due to rate limits, server outages, or network hiccups. The fallback feature prevents your application from crashing by automatically trying a list of backup providers and models in order until one succeeds.

How it works:

You provide an ordered list of configurations (each specifying a provider and a model). EveryAI attempts the first configuration in the list. If it raises any exception (like rate limits, authentication issues, or network errors), EveryAI catches the error, prints a warning message, and automatically retries the request using the next provider and model in your list.

Complete Example:

from everyai_core import EveryAI
from dotenv import load_dotenv
import os

load_dotenv()

client = EveryAI(
    api_keys={
        "groq": os.getenv("GROQ_API_KEY"),
        "cerebras": os.getenv("CEREBRAS_API_KEY"),
        "openrouter": os.getenv("OPENROUTER_API_KEY")
    }
)

# If groq fails (e.g. rate limit reached), it falls back to cerebras.
# If cerebras also fails, it falls back to openrouter.
response = client.chat(
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    fallback_chain=[
        {"provider": "groq", "model": "llama-3.3-70b-versatile"},
        {"provider": "cerebras", "model": "llama-3.1-70b"},
        {"provider": "openrouter", "model": "meta-llama/llama-3.1-8b-instruct:free"}
    ]
)

print(f"Success! Response resolved via: {response.provider}")
print(f"Model used: {response.model}")
print(f"Answer: {response.choices[0].message['content']}")

---

4. Autopilot Routing (mode)

Instead of manually defining a fallback chain on every single chat() call, you can configure your fallback chains once when initializing the EveryAI client. This is called Autopilot Routing.

You define named routing profiles (like "fastest", "smartest", or "balanced") using the routing_presets dictionary. When you call .chat(), you simply pass mode="fastest", and EveryAI automatically resolves that mode to the corresponding fallback chain you defined.

Complete Example:

from everyai_core import EveryAI
from dotenv import load_dotenv
import os

load_dotenv()

# Configure your fallback chains once at client initialization
client = EveryAI(
    api_keys={
        "groq": os.getenv("GROQ_API_KEY"),
        "cloudflare": os.getenv("CLOUDFLARE_API_KEY")
    },
    provider_config={
        "cloudflare": {"account_id": os.getenv("CLOUDFLARE_ACCOUNT_ID")}
    },
    routing_presets={
        "fastest": [
            {"provider": "groq", "model": "llama-3.1-8b-instant"},
            {"provider": "cloudflare", "model": "@cf/meta/llama-3.1-8b-instruct"}
        ],
        "smartest": [
            {"provider": "groq", "model": "llama-3.3-70b-versatile"}
        ],
        "balanced": [
            {"provider": "groq", "model": "llama-3.1-8b-instant"}
        ]
    }
)

# Use autopilot mode by passing the mode name
response = client.chat(
    messages=[{"role": "user", "content": "Hello!"}],
    mode="fastest"
)

print(f"Resolved via provider: {response.provider}")

---

5. Client-Side Throttling (Rate Governor)

Pace outbound API requests client-side to prevent hitting provider rate limit caps.

from everyai_core import EveryAI

# Configure 10 requests and 5,000 tokens maximum per minute
client = EveryAI(
    max_requests_per_minute=10,
    max_tokens_per_minute=5000
)

for i in range(12):
    response = client.chat(
        provider="groq",
        model="llama-3.1-8b-instant",
        messages=[{"role": "user", "content": f"Ping number {i}"}]
    )
    # The governor will block and throttle execution if limits are exceeded

---

6. Local Query Cache

Enable local request caching backed by SQLite. Identical prompt inputs with matching model settings are resolved instantly without hitting endpoints, conserving tokens and avoiding rate limits.

from everyai_core import EveryAI

# Enable caching globally
client = EveryAI(cache=True, cache_path="./my_cache.db")

# First call: hits Groq API (takes ~500ms)
res1 = client.chat(
    provider="groq",
    model="llama-3.1-8b-instant",
    messages=[{"role": "user", "content": "Calculate 25 * 40."}]
)

# Second call: resolved instantly from local SQLite cache (~1ms)
res2 = client.chat(
    provider="groq",
    model="llama-3.1-8b-instant",
    messages=[{"role": "user", "content": "Calculate 25 * 40."}]
)

---

7. Local Offline Models

Run open-weights models on your local machine using PyTorch and Hugging Face Transformers.

from everyai_core import EveryAI

client = EveryAI()

# Run OPT-125m locally on CPU/GPU
response = client.chat(
    provider="huggingface",
    model="facebook/opt-125m",
    messages=[{"role": "user", "content": "The weather today is"}],
    local=True  # Enables local transformers inference
)
print(response.choices[0].message["content"])

---

8. Telemetry & Web Dashboard

All token counts, call latencies, success rates, and rate limit errors are recorded in a local SQLite database.

Query Telemetry via Code:

from everyai_core import EveryAI

client = EveryAI()
# Retrieve telemetry summary
summary = client.tracker.get_summary()

print(f"Total Requests: {summary['total_calls']}")
print(f"Cache Hits: {summary['cache_hits_total']}")
print(f"Tokens Saved: {summary['tokens_saved_total']}")

Launch the Web Telemetry Dashboard:

Boot the built-in HTTP server to view real-time statistics, charts, and detailed transaction logs in your web browser:

everyai dashboard --port 8080

Alternatively, print a quick tabular report of statistics directly in your terminal:

everyai stats

---

Error Handling

EveryAI maps raw HTTP status codes from all providers to clean, standardized exceptions:

from everyai_core import EveryAI
from everyai_core.exceptions import RateLimitError, AuthenticationError, ModelNotFoundError

client = EveryAI()

try:
    client.chat(
        provider="groq",
        model="non-existent-model",
        messages=[{"role": "user", "content": "Hello"}]
    )
except ModelNotFoundError as e:
    print(f"Requested model was not found: {e}")
except RateLimitError:
    print("Rate limit reached. Pacing requests...")
except AuthenticationError:
    print("Invalid API credentials. Check your keys.")

---

Security & Packaging Safety

EveryAI is designed with production safety and security in mind:

1. No Leaked Credentials: API keys are handled entirely in-memory and are never written to telemetry databases, logs, or disk.
2. No Prompt Logging: Telemetry logs record only metadata (token count, provider, model name, status, execution duration). Your sensitive prompts and responses are never logged to the telemetry database.
3. Strict Package Bundling: The library's MANIFEST.in explicitly excludes local developer .env files, temporary verification scripts, cache SQLite database stores, and unit tests. You can deploy it to public directories with confidence.

---

License

This project is licensed under the MIT License. See the LICENSE file for the full text.