identark 1.2.1

The AgentGateway Protocol — secure, scalable AI agent execution infrastructure

identark

The AgentGateway Protocol — secure, scalable AI agent execution infrastructure.

---

The problem

When an AI agent can execute code, call APIs, or access files, it runs in a process. That process has an environment. That environment typically contains everything that can cause serious damage: LLM API keys, database credentials, AWS tokens.

The naive solution — run your agent on the same backend as your REST API — creates two problems at once:

1. Security: The agent can access every secret on the machine.
2. Reliability: A memory-hungry agent degrades your API. Redeploying your API kills all running agents.

identark solves both.

---

How it works

The SDK implements the AgentGateway Protocol — a clean interface between your agent logic and the outside world. Two implementations ship out of the box:

Gateway	When to use	Credentials	History
DirectGateway	Local development, CI evals	Your API key	In-memory
ControlPlaneGateway	Production on IdentArk	Zero — none in the agent	Control plane DB

Your agent code is identical in both environments. The switch is two lines.

---

Quick start

pip install identark[openai]

import asyncio
from openai import AsyncOpenAI
from identark import DirectGateway, Message, Role

async def main():
    gateway = DirectGateway(
        llm_client=AsyncOpenAI(),   # Your API key — not in the agent loop
        model="gpt-4o",
    )

    response = await gateway.invoke_llm(
        new_messages=[Message(role=Role.USER, content="Hello, IdentArk!")]
    )

    print(response.message.content)
    print(f"Cost: ${response.cost_usd:.6f}")

asyncio.run(main())

Moving to production

Change two lines. Your agent logic is untouched.

# Before (local)
from identark import DirectGateway
gateway = DirectGateway(llm_client=AsyncOpenAI(), model="gpt-4o")

# After (production — agent holds zero secrets)
from identark import ControlPlaneGateway
gateway = ControlPlaneGateway()  # auto-detects env vars inside a IdentArk sandbox

---

Installation

# Core SDK only
pip install identark

# With OpenAI support
pip install identark[openai]

# With Anthropic support
pip install identark[anthropic]

# With Google Gemini support
pip install identark[gemini]

# With Mistral AI support (EU provider)
pip install identark[mistral]

# All cloud providers
pip install identark[all]

Requirements: Python 3.10+

---

Data Sovereignty

IdentArk is designed from the ground up to work with any LLM provider, including those that keep your data inside the UK or EU. The AgentGateway Protocol decouples your agent logic from the inference provider — switching providers requires changing one line.

Run fully local with Ollama (zero data egress)

from openai import AsyncOpenAI
from identark import DirectGateway

gateway = DirectGateway(
    llm_client=AsyncOpenAI(
        base_url="http://localhost:11434/v1",
        api_key="ollama",
    ),
    model="llama3.2",
    provider="local",   # forces $0 cost tracking; inference stays on your machine
)

Install Ollama: brew install ollama && ollama pull llama3.2 && ollama serve

Use Mistral AI (EU data residency)

from openai import AsyncOpenAI
from identark import DirectGateway

gateway = DirectGateway(
    llm_client=AsyncOpenAI(
        base_url="https://api.mistral.ai/v1",
        api_key="your-mistral-api-key",
    ),
    model="mistral-small-latest",   # auto-detected as "mistral" provider
)

Mistral AI is a French company. All inference runs in EU data centres, subject to EU data protection law (GDPR). Use this when UK/EU data governance requirements prohibit sending inference traffic to US-based cloud providers.

See examples/ for complete runnable scripts.

---

The AgentGateway Protocol

Any class implementing these four async methods is a valid gateway:

class AgentGateway(Protocol):
    async def invoke_llm(self, new_messages, tools=None, tool_choice="auto") -> LLMResponse: ...
    async def persist_messages(self, messages) -> None: ...
    async def request_file_url(self, file_path, method="PUT") -> PresignedURL: ...
    async def get_session_cost(self) -> float: ...

Write your agent against the protocol. The implementation — local or production — is a runtime detail.

---

Features

• Zero-secret agents — ControlPlaneGateway holds no API keys, database credentials, or cloud tokens
• Stateless by design — conversation history owned by the gateway, not the agent; kill and restart without data loss
• Framework-agnostic — works with LangChain, LlamaIndex, raw API calls, or any custom agent framework
• Built-in cost tracking — every invoke_llm call returns cost_usd; get_session_cost() returns the running total
• OpenAI + Anthropic — both providers supported in DirectGateway out of the box
• MockGateway for testing — no LLM calls in your test suite; full call recording for assertions
• Full type annotations — py.typed marker; works with mypy strict mode

---

Testing your agents

from identark.testing import MockGateway
from identark.models import LLMResponse, Message, Role

async def test_my_agent():
    mock = MockGateway()
    mock.queue_response(LLMResponse(
        message=Message(role=Role.ASSISTANT, content="The answer is 42."),
        cost_usd=0.001,
        model="mock",
        finish_reason="stop",
    ))

    result = await my_agent(gateway=mock)

    assert mock.invoke_llm_call_count == 1
    assert mock.total_messages_sent == 1

---

Supported providers

Provider	Data residency	DirectGateway	GeminiGateway	ControlPlaneGateway
OpenAI (gpt-4o, gpt-4o-mini, …)	US	✓	—	✓ (via control plane)
Anthropic (claude-3-5-sonnet, …)	US	✓	—	✓ (via control plane)
Google Gemini (gemini-1.5-pro, gemini-1.5-flash, …)	US	✓*	✓	Roadmap
Mistral AI (mistral-large, mistral-small, …)	EU 🇪🇺	✓	—	Roadmap
Ollama (llama3.2, mistral, codellama, …)	Local 🏠	✓	—	N/A
Any OpenAI-compatible endpoint	Varies	✓	—	Roadmap

*Gemini via OpenAI-compatible endpoint. Use GeminiGateway for native SDK features.

---

Error handling

from identark.exceptions import CostCapExceededError, RateLimitError, IdentArkError

try:
    response = await gateway.invoke_llm(new_messages=[...])
except CostCapExceededError as e:
    print(f"Cost cap of ${e.cap_usd} reached. Spent: ${e.consumed_usd}")
except RateLimitError as e:
    await asyncio.sleep(e.retry_after_seconds)
except IdentArkError as e:
    # Catch-all for any SDK error
    raise

Full exception hierarchy: IdentArkError > GatewayError > ControlPlaneError > AuthenticationError | CostCapExceededError | SessionNotFoundError

---

Architecture

┌─────────────────────────────────────┐
│            Your Agent Code          │
│   (depends only on AgentGateway)    │
└──────────────┬──────────────────────┘
               │
    ┌──────────▼──────────┐
    │    AgentGateway      │  ← Protocol (interface)
    │      Protocol        │
    └──────┬────────┬──────┘
           │        │
  ┌────────▼─┐  ┌───▼──────────────┐
  │  Direct  │  │  ControlPlane    │
  │ Gateway  │  │    Gateway       │
  │          │  │                  │
  │ Local /  │  │   Production     │
  │  Evals   │  │  (zero secrets)  │
  └──────────┘  └────────┬─────────┘
                         │ HTTP
                ┌────────▼─────────┐
                │  IdentArk        │
                │  Control Plane   │
                │  (holds creds)   │
                └──────────────────┘

---

Community

• Discussions: GitHub Discussions — ask questions, share ideas
• Issues: GitHub Issues — bug reports and feature requests
• Live Demo: identark.vercel.app/demo — try IdentArk in your browser

---

Contributing

Contributions are welcome. Please open an issue before submitting significant changes.

git clone https://github.com/identark/identark.git
cd identark
pip install -e ".[dev]"
pre-commit install
pytest tests/unit/

See CONTRIBUTING.md for full guidelines.

---

Roadmap

• LangChain adapter (IdentArkChatModel)
• LlamaIndex adapter (IdentArkLLM)
• Streaming support (invoke_llm_stream)
• CrewAI integration
• LangGraph integration (IdentArkNode, IdentArkStreamNode)
• Pluggable inference backends (distributed compute)
• identark-cli for one-command control plane deployment

---

License

The IdentArk SDK is licensed under the MIT License — free for any use, including commercial and closed-source projects. See LICENSE.

The IdentArk control plane (hosted service) is proprietary. The SDK works with any AgentGateway backend, including fully self-hosted ones.

---

Built on the control plane pattern described in How We Built Secure, Scalable Agent Sandbox Infrastructure.