Skip to main content

Cognitrix

Cognitrix is an open-source autonomous AI agents orchestrator built in Python. It allows you to create and manage AI agents with ease and integrates seamlessly with large language models (LLMs) from various providers.

Features

  • Agent Creation and Management: Create, list, and load AI agents with customizable names, tasks, and configurations.
  • LLM Integration: Integrates with multiple LLM providers via OpenRouter (200+ models including OpenAI, Anthropic, Google, Meta, Mistral) plus local models via Ollama.
  • Modular Architecture: Easily extensible, allowing the addition of new tools, agents, and LLM integrations.
  • Conversational Interface: Interact with AI agents through a command-line interface, providing queries and receiving responses.
  • Tool Integration: Agents can utilize a variety of tools, including calculators, web searches, file system browsers, and more.
  • Autonomous Agent Mode: Agents can operate autonomously, visually perceiving the screen, interacting with UI elements, and performing tasks.
  • Multimodal Support: Handles both text and image inputs/outputs, enabling multimodal interactions.

Architecture

Cognitrix's architecture is designed to be highly modular and extensible:

  • Agents: The base Agent class is for creating and managing AI agents.
  • LLMs: Provider classes for OpenRouter (primary - 200+ models), OpenAI (direct), and Ollama (local).
  • Tools: A set of tools that agents can utilize. Each tool has a category attribute for grouping and management.
  • Templates: Customizable prompt templates guide the behavior and output formats of LLMs.

More tools can be added by creating new classes that inherit from the Tool base class and specifying a unique category.

Installation

Prerequisites

  • Python 3.11–3.13 (the package will not build on 3.10 or lower).
  • Node.js 18+ and a package manager (this repo uses pnpm) — only needed to build the web UI from source.
  • Poetry — only needed for building from source.
  • Redis — only needed if you use the background task API (Celery worker).

Install with pip:

pip install cognitrix

Build from source:

git clone https://github.com/theonlyamos/cognitrix.git
cd cognitrix/frontend
pnpm install
pnpm run build
cd ..
pip install .

Install directly from github:

pip install https://github.com/theonlyamos/cognitrix/archive/main.zip

Build the Docker Image:

git clone https://github.com/theonlyamos/cognitrix.git
cd cognitrix
docker build -t cognitrix .

Usage

Fill these environment variables as needed

OPENAI_API_KEY=
CO_API_KEY=
TAVILY_API_KEY=
CLARIFAI_ACCESS_TOKEN=
GROQ_API_KEY=
GOOGLE_API_KEY=
NEWSAPI_API_KEY=
ANTHROPIC_API_KEY=
DEEPGRAM_API_KEY=
MINDSDB_API_KEY=
BRAVE_SEARCH_API_KEY=
AIMLAPI_API_KEY=

To run Cognitrix with default settings:

cognitrix

Access the Web UI

To run with web interface

cognitrix --ui web

Open your web browser and go to http://localhost:8000 to access the web UI.

Run the Docker Container

After building the image, you can run the container with:

docker run -p 8000:8000 cognitrix

This command maps port 8000 of the container to port 8000 on your host machine.

To choose an LLM provider, pass --provider (config is read from the environment — AI_PROVIDER, <PROVIDER>_BASE_URL, <PROVIDER>_API_KEY, <PROVIDER>_MODEL):

cognitrix --provider openrouter

To list created agents:

cognitrix agents -l

To list available tools:

cognitrix tools -l

To run Cognitrix with a specific provider:

cognitrix --provider <provider_name>

To run Cognitrix with a specific agent:

cognitrix --agent <agent_name>

To run Cognitrix with a category of tools:

cognitrix --load-tools "web"

To run Cognitrix with categories of tools:

cognitrix --load-tools "web,general"

To create a new agent:

cognitrix agents --new

For more options and usage details, use the help command:

cognitrix --help

Second-model review of risky actions

Tool calls that would normally ask for approval (deleting, overwriting a file, sending something out, shell commands) can be screened by a second model first. It answers allow (runs without asking), block (refused, and the agent is told why) or ask (you get the usual approval prompt). Errors, timeouts and unclear answers count as ask.

COGNITRIX_REVIEWER=unattended          # off | unattended (task runs, auto mode) | always
COGNITRIX_REVIEWER_PROVIDER=anthropic  # defaults to AI_PROVIDER
COGNITRIX_REVIEWER_MODEL=claude-haiku-5-5
COGNITRIX_REVIEWER_MAX_ALLOW=medium    # high lets it allow deletes on its own too

The reviewer never answers for tools you set to "Ask first" or for commands that need the full shell. Task runs record each verdict as an action_reviewed run event.

Code mode and tools on demand

Two switches on an agent's Tools tab (under "Saving tokens") cut what long jobs spend on tools:

  • Code mode gives the agent a Run Code tool. It writes one short JavaScript snippet that calls many tools (await tools.grep({pattern: "TODO"}), loops, Promise.all) and returns only what it needs, instead of one model step per call with every result pasted into the chat. Scripts run in a V8 isolate with no file system or network of their own; each tool call inside goes through the same permission levels and approvals as a direct call.
  • Load tools only when needed sends the model full descriptions of just a few core tools and the names of the rest. It loads the others with Find Tools when it needs them, and they stay loaded for that chat. Worth it from about 12 tools.

COGNITRIX_CODE_MODE_TIMEOUT (seconds, default 600) and COGNITRIX_CODE_MODE_MAX_CALLS (default 200) limit one script.

Use your agents from a code editor (Agent Client Protocol)

cognitrix acp runs Cognitrix as an Agent Client Protocol server, so editors that support ACP (Zed, JetBrains IDEs, Neovim plugins and others) can chat with your agents. The chat works in the editor's project folder, tool calls show up in the editor, and anything that would ask you for approval asks in the editor. Your agents appear in the editor's mode menu, so you can switch between them, and chats are saved like any other.

In Zed, add this to settings.json:

{
  "agent_servers": {
    "Cognitrix": {
      "command": "cognitrix",
      "args": ["acp", "--agent", "Assistant"]
    }
  }
}

With only the desktop app installed, point command at its engine instead: cognitrix-engine with "args": ["acp"] (on Windows it is in the app's install folder under engine\). Add --user you@example.com to use one account's agents and saved sign-ins when several people share the machine.

Web UI

Cognitrix includes a web UI built with React, TypeScript, and Vite. This UI allows users to interact with the AI agents and manage their configurations easily.

Development

For development purposes, you can also run the web UI locally without Docker. Ensure you have Node.js 18+ and pnpm installed, then follow these steps:

  1. Install Dependencies: Navigate to the frontend directory and run:

    pnpm install
    
  2. Start the Development Server: Run the following command to start the development server:

    pnpm run dev
    
  3. Access the Development Server: Open your web browser and go to http://localhost:5173 to view the web UI in development mode.

Docker deployment

docker-compose.yml runs three services: the web app (FastAPI + the in-process schedule loop), a dedicated worker (Celery, executes task/team runs), and Redis as the broker. Web and worker share the SQLite app database over a named volume.

cp .env.example .env          # fill in provider keys + a JWT_SECRET_KEY
docker compose up --build

The UI is then at http://localhost:8000.

  • Provider keys / config come from .env (injected into both web and worker). COGNITRIX_ENV=production is set in compose, so JWT_SECRET_KEY is required — generate one with python -c "import secrets; print(secrets.token_urlsafe(32))".
  • Scale execution with docker compose up --scale worker=3. Keep web at a single replica: the schedule tick loop runs per-process, so multiple web containers would fire the same due schedules more than once.
  • State lives in the cognitrix-data volume (SQLite DB, JWT secret, MCP config) and redis-data (broker). For heavier task concurrency, point DB_* at PostgreSQL or MySQL instead of the shared SQLite file. MongoDB is not supported by the durable task runner because its cross-table lease fencing requires relational atomic predicates.
  • Without CELERY_BROKER_URL, a single container falls back to an in-process filesystem broker + auto-spawned worker — handy for a one-container deploy, but the dedicated-worker compose stack is the recommended setup.

Fly.io

fly.toml deploys the single-container shape to Fly.io: one always-warm machine running a Supervisor-managed web process and solo Celery worker. Configure the external Redis connection with the CELERY_BROKER_URL and TASK_LIMIT_REDIS_URL Fly secrets. SQLite persistence lives on the mounted volume. Keep the web process to one machine while using SQLite because the volume is not shared and the schedule loop is per-process.

API Access

Cognitrix exposes an HTTP API for programmatic use by external apps, scripts, and automation platforms. Create an API key from the API Keys page in the web UI (or via POST /api/v1/api-keys with a session token). A key is shown once at creation — copy the key and its webhook signing secret then.

Keys carry fine-grained permissions:

  • Scopes — chat (call agents), run (start/cancel tasks and teams), read (GET resources), write (create/edit/delete).
  • Allowlists — optional agent/team restrictions (empty = all). Allowlists constrain invoke paths; write is full CRUD.

Send the key as Authorization: Bearer ctx_… or X-API-Key: ctx_…. Missing/invalid/revoked/expired credentials return 401; a valid key lacking a scope or allowlist entry returns 403.

Chat with an agent

curl -X POST http://localhost:8000/api/v1/agents/<agent_id>/generate \
  -H "Authorization: Bearer ctx_…" -H "Content-Type: application/json" \
  -d '{"message": "Hello", "session_id": null}'
# -> {"reply": "...", "session_id": "..."}   (pass session_id back to continue)

Add "stream": true for a Server-Sent-Events token stream (event: chunk … event: done).

Run a team or task

curl -X POST http://localhost:8000/api/v1/teams/<team_id>/run \
  -H "Authorization: Bearer ctx_…" -H "Content-Type: application/json" \
  -d '{"description": "Summarize Q3 metrics", "callback_url": "https://my.app/hook"}'
# -> 202 {"task_id": "..."}

Poll GET /api/v1/tasks/<task_id> and GET /api/v1/tasks/<task_id>/runs, or supply a callback_url to receive a webhook when the run finishes. POST /api/v1/tasks/<task_id>/run starts a pre-created task the same way.

Group chat with a team

Open a team and click Group chat to talk with all of its agents in one conversation (web and desktop):

  • @name asks that agent; mention several and they answer in that order. @all asks everyone.
  • With no mention, the agent who spoke last answers (the team lead starts a new chat).
  • Agents hand work to each other by @mentioning a teammate in their reply. One message gets at most 6 agent replies (COGNITRIX_GROUP_MAX_TURNS), and each agent answers at most twice.

Group chats run in the background like normal chats, so you can leave and come back. They use your signed-in session (/api/v1/teams/<team_id>/chats), not API keys.

Schedule a task

Tasks can run themselves: one-shot ("at a time") or recurring (fixed interval or cron). Schedule fields ride the normal task payload:

# every 6 hours
curl -X POST http://localhost:8000/api/v1/tasks \
  -H "Authorization: Bearer ctx_…" -H "Content-Type: application/json" \
  -d '{"title": "Nightly digest", "description": "Summarize new issues",
       "assigned_agents": ["<agent_id>"], "schedule_interval": 21600}'

# weekdays at 9am (server-local time)
#   "schedule_cron": "0 9 * * 1-5"
# once, at a specific time (any ISO datetime; offsets are normalized to UTC)
#   "schedule_at": "2026-08-01T09:00:00+02:00"

At most one of schedule_at / schedule_interval (seconds, min 60) / schedule_cron may be set. Setting one arms the schedule (schedule_enabled defaults true); the response carries next_run_at (UTC). Pause/resume with POST /api/v1/tasks/<task_id>/schedule {"enabled": false} — resuming recomputes next_run_at.

Semantics: cron is evaluated in the server's local timezone; one-shot times are stored as UTC instants. If the server was down when a run was due, it fires once on startup (no backfill). If a run is still active when the next occurrence comes due, recurring schedules skip that occurrence; one-shots wait and fire when the run ends. Each fire creates a normal TaskRun, and the task's callback_url webhook applies. Scheduling via API key requires the run scope plus agent/team allowlists, same as starting a task directly.

Webhook verification

Deliveries carry X-Cognitrix-Timestamp and X-Cognitrix-Signature: sha256=<hmac>. The HMAC-SHA256 is computed over "{timestamp}.{raw_body}" with the key's webhook secret — recompute and compare in constant time, and reject stale timestamps:

import hmac, hashlib

def verify(body: bytes, timestamp: str, signature: str, secret: str) -> bool:
    expected = "sha256=" + hmac.new(
        secret.encode(), f"{timestamp}.".encode() + body, hashlib.sha256
    ).hexdigest()
    return hmac.compare_digest(expected, signature)

By default webhook targets on loopback/private/link-local addresses are rejected (SSRF guard); set COGNITRIX_WEBHOOK_ALLOW_PRIVATE=1 to allow them in local/dev setups.

OpenAI-compatible endpoint

Point any OpenAI SDK at <host>/v1 with your key as the API key:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="ctx_…")
client.chat.completions.create(
    model="Assistant",  # an agent name (see GET /v1/models)
    messages=[{"role": "user", "content": "Hello"}],
)

Both blocking and stream=True are supported. Requires the chat scope.

Contributing

Cognitrix is open source and contributions are welcome! Please refer to CONTRIBUTING.md for guidelines on how to contribute.

License

This project is licensed under the Apache License 2.0. See LICENSE for more information.

Acknowledgments

Cognitrix was created by Amos Amissah and is inspired by projects like AutoGPT and GPTEngineer. Special thanks to the open-source community and AI companies providing LLM APIs.

Metadata

Release files for cognitrix 0.5.9

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for cognitrix 0.5.9
File Size Uploaded Provenance Actions
cognitrix-0.5.9.tar.gz 1.3 MB Without provenance Details for cognitrix-0.5.9.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for cognitrix 0.5.9
File Interpreter ABI Platform Actions
cognitrix-0.5.9-py3-none-any.whl Python 3 none any Details for cognitrix-0.5.9-py3-none-any.whl

Total release size: 2.9 MB

Release files / cognitrix-0.5.9.tar.gz

Download URL cognitrix-0.5.9.tar.gz
Size 1.3 MB
Tags Source
SHA-256 checksum
How to use checksums
575cda5d2038184739b38f5d861755fca58aa45f71e7d9fab814fc89de0f7fbf
BLAKE2b-256 checksum
How to use checksums
f8a3fdf1a5e98ebf7c7a233d88d50a805f139139aec92f1f65ffff0dd61fcc66
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.9.25

Release files / cognitrix-0.5.9-py3-none-any.whl

Download URL cognitrix-0.5.9-py3-none-any.whl
Size 1.5 MB
Tags Python 3
SHA-256 checksum
How to use checksums
292bbd9a255950f5362f0d5622777f634a4595251a3da839d51cdaf07141b00f
BLAKE2b-256 checksum
How to use checksums
fdd39053d0d22dedb8081f560e9981156f6a3027c649077c1e222c86e0ca08e9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.9.25

Security for cognitrix 0.5.9

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page