Chatty

An advanced AI Chatbot CLI with a rich terminal interface, sandboxed file system tools, multi-provider LLM support (Ollama, OpenRouter, and custom providers via LiteLLM), syntax verification, dynamic skills, and cost/token optimization.

Chatty provides a local terminal loop that allows an LLM agent to interact with your codebase and system tools in a safe, sandboxed directory. It utilizes rich and prompt_toolkit to deliver a premium user experience, complete with formatting, status indicators, and autocompletion.

---

Features

• Multi-Provider Flexibility: Switch seamlessly between local Ollama instances, cloud-based OpenRouter endpoints, or any custom API provider (such as Anthropic, Chatty, DeepSeek, Groq, etc.) via LiteLLM integration.
• Oracle Query Delegation: Consult a more advanced oracle model dynamically during a session via the ask_oracle tool or the /oracle slash command.
• Sandboxed Operations: Restricts file modifications and commands strictly to a sandbox root (default ./sandbox), enhanced by Linux Landlock kernel-level isolation when running on Linux.
• User-Space AST Validation: Parses Python scripts executed via shell commands and blocks direct filesystem operations unless explicitly permitted or whitelisted by the user.
• Interactive UI: Status bars showing the provider, active model, token counter, active loop count, and sandbox directory path. Autocomplete and multiline inputs are fully integrated.
• Slash Commands: Modify settings dynamically mid-session, view tool metrics, or compress history to save tokens.
• Dynamic & Static Skills: Dynamically import system prompts and guidelines from external files or directories via keyword/tag triggers or statically on initialization.
• Context Optimization: Automates context truncation and history compression. Implements ephemeral cache_control tagging to maximize prompt caching efficiency for OpenRouter.
• Google-Style Logging: Standardized process tracking via custom glog styling.

---

Installation & Setup

Prerequisites

• Python 3.8 or higher.
• A running Ollama instance locally (optional, for local models).
• An OpenRouter API Key (optional, for remote models).

Installation

Install directly from PyPI:

pip install chatty-agent

Alternatively, install from the local repository directory:

pip install .

Developer Mode (Editable Install)

To develop or modify the codebase and have changes immediately visible without reinstalling:

pip install -e .

---

CLI Usage & Arguments

Once installed, invoke the chatbot using the chatty command or via Python:

chatty [options]
# OR
python3 -m chatty [options]

Full List of CLI Arguments

Parameter	Short	Type	Default	Description
--provider	-p	string	ollama	Backend provider to use (e.g., ollama, openrouter, anthropic, chatty, etc.). If omitted and --url is set, resolves automatically to the domain of the URL.
--model	-m	string	Auto-resolved	Model identifier(s) to load. Can be specified multiple times or as comma-separated values. The first becomes the active model. Ollama: auto-detects first local model (falls back to qwen2.5-coder:7b). OpenRouter: dynamically defaults to the most popular free model (falls back to google/gemini-2.5-flash:free). Custom: required (no fallback).
--oracle-model	None	string	None	Model identifier to use as the oracle. No default is assumed (the oracle tool is only active if an oracle model is explicitly configured).
--context-size	-c	integer	8192	Target context window length constraint in tokens.
--sandbox	-s	string	./sandbox	Path to the sandboxed folder. All writes and runs are jailed inside this directory.
--skills-path	-k	string	None	Custom directory paths to scan for static/dynamic Skills (can be specified multiple times).
--ondemand-skills-path	-o	string	None	Custom directory paths to scan for on-demand Skills (can be specified multiple times).
--whitelist	-w	string	None	Add an out-of-sandbox path to the initial whitelist. Can end with :ro or :rw to set mode (defaults to ro). Can be specified multiple times.
--static-skills	None	flag	Auto	Load all skills statically into system instructions (defaults to True for OpenRouter, False for Ollama).
--prompt-caching	None	flag	False	Explicitly enable prompt caching for compatible models (adds cache_control tagging).
--max-loops	-l	integer	20	Maximum sequential tool executions allowed in a single user turn.
--config-prompt	-f	string	None	Path to a YAML or plain text file containing custom system prompt guidelines.
--prompt-mode	-d	string	replace	How to apply custom system prompt configuration (replace default prompt, or integrate/append to it).
--api-key	-a	string	None	Client API Key. Overrides OPENROUTER_API_KEY, CUSTOM_API_KEY, or OPENAI_API_KEY environment variables depending on the provider.
--url	-u	string	None	Custom Base URL override (required for custom providers; automatically resolves the provider name to the domain of the URL if --provider is not specified).
--max-read-chars	None	integer	40000	Maximum character limit when reading a text file to prevent context explosion.
--max-grep-results	None	integer	100	Limit on matching results returned from the regular expression search tool.
--max-command-chars	None	integer	16000	Maximum characters returned from stdout/stderr of executing commands.
--max-history-tool-chars	None	integer	1000	Token saving limit: compresses old/historical tool output messages to this size.
--history-keep-messages	None	integer	4	Number of recent messages to keep fully raw and uncompressed.
--max-url-chars	None	integer	24000	Limit on fetched website character outputs.
--max-dir-items	None	integer	200	Maximum number of directory items listed by directory explorer tool.
--log-file	None	string	chatty.log	File path where execution statements are logged. Set to "" to disable.
--log-level	None	string	info	Logging verbosity (debug, info, warning, error).
--headless	None	flag	False	Run the chatbot in headless mode (no console printing or terminal interactive loop).
--max-thinking-chars	None	integer	12000	Maximum internal thinking characters before prompting the user.
--max-thinking-leeway-chars	None	integer	2000	Leeway in characters beyond the maximum before hard-aborting or prompting.
--api-delay	None	float	2.5	Minimum delay in seconds between consecutive API requests.
--api-timeout	None	float	60.0	Timeout in seconds for API requests and streams.

---

Custom OpenAI-Compatible & Multi-Provider Support

Chatty uses LiteLLM under the hood, meaning you can connect to any third-party cloud provider (like Anthropic or Chatty) or any OpenAI-compatible custom endpoint.

1. OpenAI-Compatible Custom Endpoints

To connect to custom endpoints (like DeepSeek, Groq, Together AI, or local vLLM servers), specify --url and --model (along with --api-key if required). The provider name will automatically be resolved to the domain name of your URL:

# Run using DeepSeek V3 (automatically sets provider to 'api.deepseek.com')
chatty --url https://api.deepseek.com/v1 --api-key YOUR_DEEPSEEK_KEY --model deepseek-chat

# Run using Groq (automatically sets provider to 'api.groq.com')
chatty --url https://api.groq.com/openai/v1 --api-key YOUR_GROQ_KEY --model llama-3.3-70b-versatile

# Run using Together AI (automatically sets provider to 'api.together.xyz')
chatty --url https://api.together.xyz/v1 --api-key YOUR_TOGETHER_KEY --model Qwen/Qwen2.5-Coder-32B-Instruct

2. Native Multi-Provider Support via LiteLLM

You can also call other native providers supported by LiteLLM (such as Anthropic or Google Chatty) directly by specifying their provider name and model:

# Run using Anthropic Claude 3.5 Sonnet (expects ANTHROPIC_API_KEY env variable)
chatty --provider anthropic --model claude-3-5-sonnet

# Run using Google Chatty Pro (expects CHATTY_API_KEY env variable)
chatty --provider chatty --model chatty-2.5-pro

---

Interactive Interface & Slash Commands

During a session, you can input direct queries to the model, or use Slash Commands to inspect/adjust configurations on the fly:

Command	Expected Arguments	Description
/help	None	Displays a formatted usage table of all slash commands.
/status	None	Shows active session variables (provider, model, oracle, sandbox, tokens, loops, etc.).
/tool_stats	None	Renders execution statistics (call counts, failures, and breakdowns for tools and binaries).
/provider	[name]	View current provider or switch backend on the fly. Accepts standard or custom provider names (e.g., ollama, openrouter, anthropic, chatty, api.deepseek.com).
/model	[ID|name]	View active model name or switch to another model by name or 1-based index/ID.
/models	[add <name>|remove <ID|name>|available [--refresh]|search <query>|info <ID|name>]	List, add, remove, search, or view details of LLM models.
/oracle	[name]	View active oracle model name or switch to another oracle model by name.
/sandbox	[path]	View sandbox path or change it. Instantly loads any skills found in the new sandbox.
/whitelist / /permissions	[add <path> [ro|rw] | remove <path> | clear]	View or manage whitelisted out-of-sandbox paths.
/skill	[NAME...|clear]	Load on-demand skill(s) or clear explicitly loaded ones.
/context	[tokens]	View or update target context memory window limit in tokens.
/loops	[iterations]	View or modify the limit of sequential agent loops allowed per turn.
/api_key	[key]	Configure your client token dynamically.
/system	[text]	Inspect or update the base system prompt instructions directly.
/load	<path> [append|replace]	Read system instructions from a local YAML or text file, appending or replacing.
/save / /save_session	<path>	Save the whole status of the current conversation/session to a JSON file.
/load_session	<path>	Load a saved conversation/session status from a JSON file.
/history	[N]	Renders message records, estimated token counts, roles, and tool calls, or displays the N-th entry in full detail.
/undo	[count]	Reverts the last conversation turn(s), removing assistant responses, tool outputs, and the user prompt.
/pop	<index>	Truncates the conversation history by deleting all messages from the specified 1-based index onwards.
/backups	<file_path>	Lists all available timestamped backups for a given file.
/restore	<file_path> [index_or_timestamp]	Restores a file to a specific backup version (defaults to the latest).
/tools	None	Lists available sandboxed tools and their schema definitions.
/config	[key=value]	List, view, or change configuration parameters live.
/clear / /reset	None	Clears conversational context history.
/compress	[N]	Directs the model to summarize current conversational state using a structured format, resets older history, and keeps N (default 4) recent messages intact.
/copy / /clip	`[index	all]`
/write / /save_code	`	all]`
/save_response / /save_reply	<path>	Save the entire last AI response to a file.
/exit / /quit	None	Cleanly terminates background processes and exits Chatty.

---

Sandboxed File System Tools

The chatbot uses function-calling to interface with the sandbox workspace. Directly invoking command-line file manipulation programs (e.g. cat, grep, find) in run_command is blocked by safety filters in validate_command_safety. The agent is required to use the appropriate structured tools:

File Manipulation Tools

• list_dir: Explores directories inside the sandbox. Truncates output above --max-dir-items to prevent token flooding.
• read_file: Reads text files. Accepts optional start_line and end_line parameters (1-indexed), supports displaying line numbers, and honors --max-read-chars.
• write_file: Writes full text contents to a file.
• patch_file: Replaces one or more unique blocks of code inside a file using Aider-style SEARCH/REPLACE blocks (or direct search/replace parameters). Highly robust to whitespace and indentation differences (automatically adjusts output indentation to match the file). Supports unique sub-line (intra-line) replacements, chaining multiple blocks sequentially in one patch parameter, detailed mismatch diagnostics, a dry_run simulation mode, and returns applied unified diffs.
• format_file: Styles source files using formatters: black/ruff for Python, clang-format for C/C++, prettier for frontend, or custom JSON/YAML encoders. Displays diff results.
• move_file: Renames or moves files and directories safely inside the sandbox boundaries.
• copy_file: Recursively copies file system structures.
• delete_file: Permanently removes a file. Fails on directories.
• delete_directory: Permanently removes a directory (optionally recursively).
• make_directory: Recursively builds directory trees.
• get_file_info: Retrieves file system metadata (modification dates, sizes, type, and line counts for text documents).
• hex_dump: Performs a hex dump or parses slices of binary files into integers of various widths (8/16/32/64-bit), endianness, and signedness.
• list_file_backups: Lists all available timestamped backups for a file path.
• read_file_backup: Reads the contents of a specific timestamped file backup (with optional range read and line numbering).

Code Search & Diagnostics

• search_grep: Performs recursive regular expression string matching on files. Can report line numbers (line_numbers: true) to aid editing.
• locate_files: Finds files recursively matching glob configurations (e.g., **/*.py).
• run_tests: Runs test scripts (pytest, npm test, custom targets).

Web & Information Retrieval

• search_web: Searches the web for a query and returns titles, URLs, and snippets. Supports multiple backends via environment variables (checked in priority order):
  • Tavily: Set TAVILY_API_KEY (highly recommended for clean, parsed AI search results).
  • Brave Search: Set BRAVE_API_KEY (independent, privacy-focused search).
  • Google Custom Search: Set GOOGLE_API_KEY and GOOGLE_CSE_ID (legacy Google search engine).
  • Serper: Set SERPER_API_KEY (Google search proxy).
  • SerpApi: Set SERPAPI_API_KEY (Google search proxy).
  • Yahoo Scraper: Default fallback if no keys are provided (unreliable for heavy use).
• fetch_url: Fetches the text content of a public URL. Automatically parses both HTML (converting it to clean plain text) and PDF documents. For scanned PDFs (which lack a text layer), it features an optional zero-overhead OCR fallback utilizing pytesseract and pdf2image dynamically if available on the host system.

Command & Background Execution

• run_command: Runs shell commands from the sandbox directory.
  • Safety Restrictions: Monitored by safety checks to block commands that attempt directory escapes, or attempt to circumvent tool guidelines by calling commands like cat, grep, find, sed, awk, less, more, wc, kill, pkill, killall, cp, mv, rm, rmdir, mkdir, ls, dir.
  • Output Controls: Supports output_filter (regex matching), head_lines, and tail_lines parameters to prevent token overflow.
  • Asynchronous Process Execution: Commands that block or run indefinitely are automatically backgrounded by the session, returning a Task ID (e.g., task_1).
• check_background_command: Inspects status, reads output, and checks exit status code of background processes using their task_id.
• peek_task_output: Peeks at the currently accumulated output of a background task without blocking or changing its running status.
• kill_process: Kills a running background subprocess.
• sleep: Pauses execution for a specified number of seconds.
• ask_question: Prompts the user with a question or selectable options to confirm decisions or resolve ambiguity.
• ask_oracle: Consults a more advanced oracle model for advice or suggestions when stuck on a difficult reasoning step, logic problem, or complex code generation.

---

Sandbox & Landlock Security Architecture

Chatty implements a layered security approach to sandbox tool executions and protect the host system.

1. User-Space Safety Filtering

Every command sent to run_command is statically parsed and verified by validate_command_safety. It blocks common shell commands (such as cat, grep, find, cp, mv, rm, ls, etc.) to force the LLM to use structured sandbox API tools rather than arbitrary shell execution.

Additionally, Python script executions (either run inline with python -c or loaded from .py files) are parsed into an Abstract Syntax Tree (AST) to extract their structural signature. If direct filesystem operations (like open, write, os.remove, etc.) are detected, the execution is blocked, and the chatbot prompts the user interactively to allow or deny the script, or whitelist its signature.

2. Kernel-Level Linux Landlock Sandboxing

On Linux (kernel version 5.13+), Chatty provides transparent, compile-on-demand process isolation using the Linux Landlock LSM (Linux Security Module).

Compilation Flow

When a chatbot session is initialized (see ChatbotSession), Chatty checks if:

1. The operating system is Linux.
2. The --sandbox option is enabled.
3. GCC or Clang is installed.

If these conditions are met, compile_landlock_binary builds the C-based wrapper helper landlock_exec.c into an executable binary landlock_exec. The binary is cached in the package directory or under the user's cache folder (~/.cache/chatty/) to avoid compilation overhead on subsequent startup.

Restrictive Access Rules

When executing a shell command via run_command, the execution argument is wrapped using wrap_command_with_landlock, transforming it into:

landlock_exec --ro / --rw <sandbox_dir> --rw <temp_dir> -- /bin/sh -c "<command>"

The landlock_exec binary interacts with the Linux kernel to apply rules before spawning the target process:

• Read-Only Paths (--ro): Allows reading from / (the root filesystem). This enables reading standard system executables, Python runtimes, shared libraries (/lib, /usr/lib), and general command dependencies.
• Read-Write Paths (--rw): Specifically permits file writes, directory creation, and removals only inside the target <sandbox_dir> and the system's temporary directory. Any attempts to write elsewhere on the filesystem will fail with a Permission denied error.
• No New Privileges: Configures prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) so that processes cannot escalate privileges.
• Kernel Enforcement: Enforces the ruleset using the landlock_restrict_self system call, sealing the sandbox rules for the current process and all future sub-processes.

Fallback Behavior

If Landlock is unavailable (non-Linux systems, older kernels, or missing compilers), Chatty automatically falls back to standard user-space process execution limited by the working directory (cwd) configuration and command regex validations.

Out-of-Sandbox Path Whitelisting

To allow the chatbot to access files or directories outside the sandboxed folder:

• CLI Startup Whitelist: Use the --whitelist (or -w) option to whitelist paths at startup.

  chatty -w /usr/include:ro -w /home/user/project
  

  Paths default to Read-Only (ro) unless suffixed with :rw (Read-Write).
• Interactive Whitelisting: If the agent attempts to read or write a file outside the sandbox that is not whitelisted, Chatty will prompt you interactively:
  • [y]es: Allow access for this operation once.
  • [n]o: Deny access.
  • [a]lways: Whitelist the specific file for the rest of the session.
  • [p]arents: Show a menu to select and recursively whitelist a parent directory.
• Session Management: Use the /whitelist (or /permissions) slash command to inspect or manage whitelisted paths on the fly.

---

Extending Chatty with Skills

Skills are modular system prompt extensions and runbooks that give the LLM custom domain knowledge. Under the standard, a skill is structured as a directory containing a required SKILL.md file (for example, see skills/greetings/SKILL.md) and optional helper subfolders:

skills/database_guidelines/
├── SKILL.md          # Required: Main instruction file with YAML frontmatter
├── scripts/          # Optional: Helper scripts and utilities
├── examples/         # Optional: Reference implementations
├── resources/        # Optional: Additional assets or templates
└── references/       # Optional: Detailed documentation or manuals

SKILL.md Structure

The SKILL.md must start with YAML frontmatter containing the name and description fields:

---
name: database-operations
description: Instructions for SQL query formatting and schemas.
---
Always format queries in uppercase... (Rest of system guidelines)

Discovery Locations

Chatty automatically discovers skills and plugins by searching recursive and machine-local customization roots:

1. Workspace Customizations: Searches for folders named .agents/, .agent/, _agents/, _agent/, or .chatty/ by walking up parent directories from the current working directory to the repository root.
2. Global Customizations: Loads customizations globally from ~/.chatty/config/.
3. Built-in Package Skills: Automatically loads default skills packaged with Chatty.
4. Configuration Manifests: Explicitly registers additional paths using skills.json and plugins.json configs placed in customization roots.

Progressive Disclosure

To conserve LLM context window size, Chatty uses progressive disclosure:

• By default, only the names and descriptions of all discovered skills are injected into the system instructions.
• The full content of a skill is loaded into context only when explicitly activated.
• If a skill is running in static mode (or configured to load permanently), all skills can be loaded statically on initialization.

The /skill command

• /skill: Lists all available skills and those currently explicitly loaded.
• /skill name1 name2: Explicitly activates one or more skills.
• /skill clear: Clears all explicitly loaded skills. Loaded skills persist through session clears (via /clear).

---

Context Optimization

To operate efficiently over long conversations, Chatty implements two main context optimization mechanisms:

1. Smart History Pruning

Before sending messages to the LLM, the prune_history method limits the total context:

• Old tool outputs exceeding --max-history-tool-chars characters are truncated in memory and annotated with a [TRUNCATED] note.
• The latest messages (configured via --history-keep-messages) are kept fully raw to ensure the model retains precise local context.

2. Prompt Caching

When --prompt-caching is explicitly enabled or when using static skills with OpenRouter, Chatty injects cache_control: {"type": "ephemeral"} metadata parameters:

• Applied to the system prompt message.
• Applied to the tools schema structure.
• Applied to the last two messages in the conversation queue. This helps minimize token costs and reduces API response latencies for compatible model endpoints.

---

Logging & Diagnostics

Chatty records session activity in a log file (default: chatty.log). The logging mechanism (configured in setup_logging) uses the GlogFormatter to output in standard Google logging syntax:

Lyyyymmdd hh:mm:ss.uuuuuu process file:line] message

• L: Level identifier letter (D for Debug, I for Info, W for Warning, E for Error, F for Critical/Fatal).
• yyyyymmdd hh:mm:ss.uuuuuu: Precise timestamps.
• process: OS Process ID.
• file:line: Source code location reporting the logging output.

To disable file logging, run with --log-file "".

---

Testing

Chatty comes with a comprehensive unittest suite located in tests/. You can execute tests from the project root:

python3 -m unittest discover tests

The test suite validates the following components:

• test_caching_and_repeats.py: Verifies prompt caching efficiency, EPHEMERAL headers, and handling of repeated prompts.
• test_commands.py: Exercises interactive slash commands (switching provider, model, modifying context parameters, system prompts).
• test_cutoff.py: Validation of message token truncation, history pruning, and prompt caching.
• test_format.py: Verification of json, yaml, and clang-format code styling tools.
• test_headless.py: Validates running the chatbot session in headless mode.
• test_landlock.py: Unittests for the Landlock sandboxing mechanism on Linux.
• test_pdf_handling.py: Validates PDF text extraction, encryption handling, scanned PDF detection, and OCR fallback logic.
• test_logging.py: Checks for Google-style Logging (glog) file outputs.
• test_oracle.py: Tests the oracle query delegation logic, the ask_oracle tool, and oracle resolution.
• test_safety.py: Ensures commands and processes are validated for sandboxing.
• test_sandbox_ops.py: Verification of file copy, move, delete, make-dir, search, and info tools.
• test_session_persist.py: Verifies saving and loading session states to/from JSON.
• test_tool_stats.py: Checks tracking metrics for tool and external binary usage.
• test_tools_modularity.py: Checks the registration and separation of modular sandbox tools.
• test_whitelist.py: Asserts correct management of out-of-sandbox directory whitelisting (ro/rw permissions) and interactive whitelist prompts.