aicortex-core 1.0.4

AI Cortex: A Python toolkit for free access to cloud and local language models. Chat with Gemma, Mistral, Deepseek, Qwen, Llama, and more through community-hosted cloud endpoints or your own Ollama server — completely free, zero-signup, zero API keys. Unified chat API, real-time streaming, multi-server orchestration, and OpenAI-compatible endpoints.

🧠 AI Cortex

A Python toolkit for free access to cloud and local language models.
Zero API keys. Zero signup. Completely free.

---

AI Cortex gives you a single, clean Python interface to hundreds of language models — Llama, Mistral, Gemma, DeepSeek, Qwen, and more — running on community-hosted cloud servers or your own local Ollama instance. No accounts. No credit cards. No rate limits.

from aicortex import chat

response = chat("Explain neural networks like I'm five.")
print(response)

---

✨ Why AI Cortex?

Feature	What it means for you
🆓 100% Free	No API keys, no billing, no subscriptions — ever
🤖 Any Model	Llama, Mistral, Gemma, DeepSeek, Qwen, and more
🌐 Cloud or Local	Community-hosted cloud endpoints or your own Ollama server
⚡ Streaming	Real-time token streaming for responsive UIs
🔌 OpenAI-Compatible	Drop-in replacement for OpenAI client apps
🛡️ Type-Safe	Full type hints, stubs, and IDE autocomplete
🔧 Production Ready	Automatic failover, multi-server routing, error handling
📦 Lightweight	One dependency (ollama) for the core package

---

🚀 Installation

# Core package
pip install aicortex-core

# With OpenAI-compatible server support
pip install aicortex-core[server]

---

💬 Chat

from aicortex import chat

# Simple response
response = chat("What is the speed of light?")
print(response)

# Custom model and parameters
response = chat(
    "Write a Python function to reverse a string.",
    model="llama3.2:3b",
    temperature=0.2,
    max_tokens=200,
)
print(response)

⚡ Streaming

from aicortex import chat

stream = chat("Write a haiku about AI.", stream=True)

for event in stream:
    if event.type == "token":
        print(event.content, end="", flush=True)

🤖 Model Discovery

from aicortex import families, models, get_model_info

# Available families
print(families())   # ['llama', 'mistral', 'gemma', 'deepseek', 'qwen']

# Models in a family
print(models("mistral"))

# Full metadata for a model
info = get_model_info("llama3.2:3b")
print(info['parameter_size'], info['quantization_level'])

🌐 Server Discovery

from aicortex import list_model_servers, get_server_info, get_llm_params

# All servers hosting a model — cloud and local
servers = list_model_servers("llama3.2:3b")
for s in servers:
    print(f"{s['url']} — {s['location']['city']}, {s['location']['country']}")

# Ready-to-use params for LangChain's OllamaLLM
params = get_llm_params("mistral:7b")
# → {'model': 'mistral:7b', 'base_url': 'http://...'}

🖥️ OpenAI-Compatible Server

Run a local proxy that speaks OpenAI's API — drop-in compatible with any OpenAI client:

from aicortex.tools import run_server

run_server(host="127.0.0.1", port=8000, default_model="llama3.2:3b")

# Use with curl
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "llama3.2:3b", "messages": [{"role": "user", "content": "Hello!"}]}'

# Use with the openai Python SDK — just change the base_url
from openai import OpenAI
client = OpenAI(api_key="none", base_url="http://localhost:8000/v1")
response = client.chat.completions.create(
    model="llama3.2:3b",
    messages=[{"role": "user", "content": "Hello!"}]
)

🔧 Model Management Tools

Keep the bundled model database fresh with the four-step pipeline:

from pathlib import Path
from aicortex.tools import (
    find_valid_endpoints,     # Step 1: ping all known IPs
    fetch_models,             # Step 2: pull model lists
    resolve_models,           # Step 3: merge with IP metadata
    apply_valid_models,       # Step 4: write into family JSONs
)

json_dir   = Path("aicortex/models")
valid_urls = find_valid_endpoints(json_dir)                              # Step 1
fetch_models(Path("valid.txt"), Path("fetched.json"))                   # Step 2
resolve_models(Path("fetched.json"), json_dir, Path("resolved.json"))   # Step 3
apply_valid_models(Path("resolved.json"), json_dir, backup=True)        # Step 4

---

📚 Full Documentation

→ aicortex.readthedocs.io

• Installation
• Quick Start
• Basic Usage
• Streaming
• Model Management
• Server Mode
• Tools

---

🤝 Contributing

Contributions are welcome! See CONTRIBUTING.md and the Development Guide.

📄 License

GNU Lesser General Public License v3.0 — free for open-source and commercial use.