everyai-core 1.0.0

Connect to every free AI provider with zero setup

EveryAI Core 🚀

Connect to every free and premium AI provider with zero setup, client-side rate-limit governance, local query caching, automatic fallback failover, and a live web telemetry dashboard.

---

🌟 Key Features

• Unified Developer SDK: Query Groq, OpenRouter, Nvidia NIM, Mistral, Cerebras, Cloudflare, and Hugging Face (both cloud REST APIs and local execution) with an identical, clean developer syntax.
• Smart Fallback Orchestration: Chain alternative providers or different keys to handle rate limits and service failures smoothly.
• Local Response Caching: SQLite-backed query cache saves tokens and speeds up repetitive workflows.
• Rate Limit Governor: Client-side RPM (Requests Per Minute) and TPM (Tokens Per Minute) tracking queues and paces outbound requests automatically.
• Telemetry & Web Dashboard: Run everyai dashboard to visualize token counts, response times, and query statistics.
• Local Offline Execution: Load and run open-weights models locally via PyTorch and Transformers using the same API interface.

---

🔌 Supported Providers

Provider	Environment Variable	Default / Preset Models	Key Features
Groq	GROQ_API_KEY	llama3-8b-8192, llama3-70b-8192, mixtral-8x7b-32768	Ultra-fast LPU inference, native token usage logging
OpenRouter	OPENROUTER_API_KEY	google/gemini-2.5-flash, meta-llama/llama-3-8b-instruct:free	Aggregated model catalog, free tier routes
Hugging Face (Cloud)	HF_TOKEN / HUGGINGFACE_API_KEY	meta-llama/Meta-Llama-3-8B-Instruct	Serverless API execution of open-weights models
Hugging Face (Local)	(None required)	gpt2, facebook/opt-125m, any local repo path	Runs models locally on CPU/GPU via PyTorch
Cerebras	CEREBRAS_API_KEY	llama3.1-8b, llama3.1-70b	High-speed wafer-scale engine inference
Mistral	MISTRAL_API_KEY	mistral-large-latest, open-mixtral-8x22b	European flagship models, native streaming
Cloudflare	CLOUDFLARE_API_TOKEN, CLOUDFLARE_ACCOUNT_ID	@cf/meta/llama-3-8b-instruct, @cf/mistral/mistral-7b	Workers AI serverless endpoints
Nvidia NIM	NVIDIA_API_KEY	meta/llama-3.1-405b-instruct	GPU-optimized microservice inference

---

📦 Installation

Install the core library via pip:

pip install everyai-core

To enable local model inference capabilities (requires PyTorch and Transformers):

pip install everyai-core[local]

---

⚡ Quick Start

1. Set Your Environment Variables

EveryAI will automatically search your environment for keys. Add them to your shell or .env file:

export GROQ_API_KEY="gsk_..."
export OPENROUTER_API_KEY="sk-or-..."
export CEREBRAS_API_KEY="cbs_..."

2. Basic Non-Streaming Request

from everyai_core import EveryAI

# Initialize client (uses environment keys by default)
client = EveryAI()

# Run a completion
response = client.chat(
    provider="groq",
    model="llama3-8b-8192",
    messages=[
        {"role": "user", "content": "Explain quantum computing in one sentence."}
    ],
    temperature=0.7
)

print(f"[{response.provider.upper()}] {response.choices[0].message['content']}")
print(f"Tokens Used: {response.usage.total_tokens}")

---

📘 Detailed Module & Feature Guide

1. Provider-Specific Direct Clients

You can access provider SDK wrappers directly through client attributes for full autocomplete and static typing support:

from everyai_core import EveryAI

client = EveryAI(api_keys={"groq": "your_custom_key_here"})

# Access Groq direct instance
response = client.groq.chat(
    model="llama3-8b-8192",
    messages=[{"role": "user", "content": "Hi!"}]
)

---

2. Streaming Response Tokens

For real-time responses, pass stream=True to receive a generator yielding chunk responses:

from everyai_core import EveryAI

client = EveryAI()

chunks = client.chat(
    provider="openrouter",
    model="google/gemini-2.5-flash",
    messages=[{"role": "user", "content": "Write a short poem about coding."}],
    stream=True
)

for chunk in chunks:
    content = chunk.choices[0].message.get("content", "")
    print(content, end="", flush=True)

---

3. Smart Fallback Failovers & Autopilot Modes

To combat rate limits, API downtime, or key exhausts, configure fallback lists. If the first provider fails, EveryAI automatically routes the prompt to the next option.

Manual Fallback Chain

response = client.chat(
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    fallback_chain=[
        {"provider": "groq", "model": "llama3-70b-8192"},
        {"provider": "cerebras", "model": "llama3.1-70b"},
        {"provider": "openrouter", "model": "meta-llama/llama-3-8b-instruct:free"}
    ]
)
print(f"Resolved via: {response.provider}")

Autopilot Presets

Use the built-in preset strings under the mode parameter:

• "fastest": Groq -> OpenRouter -> Cloudflare
• "smartest": Groq Llama-70b -> OpenRouter Gemini-Flash -> Nvidia NIM
• "balanced": Mix of fast and smart models

response = client.chat(
    messages=[{"role": "user", "content": "Compose a script."}],
    mode="fastest"
)

---

4. Client-Side Throttling (Rate Governor)

Pace outbound API requests client-side to prevent hitting provider rate limit caps.

from everyai_core import EveryAI

# Configure 10 requests and 5,000 tokens maximum per minute
client = EveryAI(
    max_requests_per_minute=10,
    max_tokens_per_minute=5000
)

for i in range(12):
    response = client.chat(
        provider="groq",
        model="llama3-8b-8192",
        messages=[{"role": "user", "content": f"Ping number {i}"}]
    )
    # The governor will block and throttle execution if limits are exceeded

---

5. Local Query Cache

Enable local request caching backed by SQLite. Identical prompt inputs with matching model settings are resolved instantly without hitting endpoints, conserving tokens and avoiding rate limits.

from everyai_core import EveryAI

# Enable caching globally
client = EveryAI(cache=True, cache_path="./my_cache.db")

# First call: hits Groq API (takes ~500ms)
res1 = client.chat(
    provider="groq",
    model="llama3-8b-8192",
    messages=[{"role": "user", "content": "Calculate 25 * 40."}]
)

# Second call: resolved instantly from local SQLite cache (~1ms)
res2 = client.chat(
    provider="groq",
    model="llama3-8b-8192",
    messages=[{"role": "user", "content": "Calculate 25 * 40."}]
)

---

6. Local Offline Models

Run open-weights models on your local machine using PyTorch and Hugging Face Transformers.

from everyai_core import EveryAI

client = EveryAI()

# Run OPT-125m locally on CPU/GPU
response = client.chat(
    provider="huggingface",
    model="facebook/opt-125m",
    messages=[{"role": "user", "content": "The weather today is"}],
    local=True  # Enables local transformers inference
)
print(response.choices[0].message["content"])

---

7. Telemetry & Web Dashboard

All token counts, call latencies, success rates, and rate limit errors are recorded in a local SQLite database.

Query Telemetry via Code:

from everyai_core import EveryAI

client = EveryAI()
# Retrieve telemetry summary
summary = client.tracker.get_summary()

print(f"Total Requests: {summary['total_calls']}")
print(f"Cache Hits: {summary['cache_hits_total']}")
print(f"Tokens Saved: {summary['tokens_saved_total']}")

Launch the Web Telemetry Dashboard:

Boot the built-in HTTP server to view real-time statistics, charts, and detailed transaction logs in your web browser:

everyai dashboard --port 8080

Alternatively, print a quick tabular report of statistics directly in your terminal:

everyai stats

---

🛠️ Error Handling

EveryAI automatically maps raw HTTP status codes from all providers to clean, standardized exceptions:

from everyai_core import EveryAI
from everyai_core.exceptions import RateLimitError, AuthenticationError, ModelNotFoundError

client = EveryAI()

try:
    client.chat(
        provider="groq",
        model="non-existent-model",
        messages=[{"role": "user", "content": "Hello"}]
    )
except ModelNotFoundError as e:
    print(f"Requested model was not found: {e}")
except RateLimitError:
    print("Rate limit reached. Pacing requests...")
except AuthenticationError:
    print("Invalid API credentials. Check your keys.")

---

🔒 Security & Packaging Safety

EveryAI is designed with production safety and security in mind:

1. No Leaked Credentials: API keys are handled entirely in-memory and are never written to telemetry databases, logs, or disk.
2. No Prompt Logging: Telemetry logs record only metadata (token count, provider, model name, status, execution duration). Your sensitive prompts and responses are never logged to the telemetry database.
3. Strict Package Bundling: The library's MANIFEST.in explicitly excludes local developer .env files, temporary verification scripts, cache SQLite database stores, and unit tests. You can deploy it to public directories with confidence.

---

📄 License

This project is licensed under the MIT License. See LICENSE for details.