AskLLM

A lightweight, cross-platform Python REPL for chatting with LLMs via any OpenAI-compatible API, powered by prompt_toolkit.

Designed to be installed via pip install askllm-cli (or pipx install askllm-cli), and developed using Podman with python:3.12-slim (or run directly with Python 3).

---

Features

• Python Code Execution Tool: Exposes a stateful Python execution tool to the LLM via OpenAI-compatible tool/function calling (-x / --code-execution), allowing models to execute Python code, verify calculations, inspect data, and iterate.
• Modern, Cross-Platform REPL (prompt_toolkit): Robust, native interactive terminal experience across Linux, macOS, and Windows.
• Slash Command Auto-Completion: Interactive auto-completion for slash commands (/help, /model, /tools, /clear, /history, etc.).
• Minimal Dependencies: Uses standard library for networking (urllib.request) and JSON, relying solely on prompt_toolkit for the terminal UI.
• Standard Python Package: Modern PEP 517/621 packaging (pyproject.toml) published as askllm-cli on PyPI.
• Podman Development Ready: Built-in development workflow using python:3.12-slim (./dev.sh).
• Environment Variable Auto-Detection: Automatically picks up standard OpenAI / Azure / local LLM variables.
• Streaming Responses: Real-time token streaming via Server-Sent Events (SSE).
• Graceful Signal Handling:
  • Ctrl+C or Ctrl+D at the beginning of the line exits the REPL.
  • Ctrl+C with text in the buffer cancels the line.
  • Ctrl+C during response streaming halts generation cleanly without exiting.
• Multi-Turn Chat History: Maintains conversation context across turns within the session.
• Automatic /v1/responses Detection & Fallback: Automatically probes and switches to the newer /v1/responses API when supported, seamlessly falling back to /v1/chat/completions if the endpoint returns HTTP 404/405/501 (such as on Ollama, LM Studio, or vLLM). Can also be configured via --api-mode (auto, chat, responses).
• Persistent Input History: History saved across runs in $HOME/.askllm_history or custom file specified via -H / --history-file or ASKLLM_HISTORY_FILE (safely falling back to in-memory history if opening fails).
• Multiline Input Support:
  • Triple quotes """ ... """
  • Trailing backslash \
  • /paste command

---

Development with Podman (python:3.12-slim)

No local Python installation or host dependencies are required. A complete development environment is provided via Podman:

1. Interactive Development Shell

Drop into an interactive bash shell running inside python:3.12-slim with AskLLM installed in editable mode (pip install -e .):

./dev.sh

Inside the shell:

# Run AskLLM directly
askllm --help

# Run tests
pytest

# Test interactive python
python3 -c "import askllm; print(askllm.__version__)"

Any changes made to files in src/askllm on your host are immediately reflected inside the container!

2. Run Tests in Podman

Run the test suite inside the python:3.12-slim container:

./dev.sh test

3. Run AskLLM via Podman

Run AskLLM using Podman directly from the host:

./askllm

or with arguments:

./askllm --model llama3.2 --endpoint http://localhost:11434/v1

4. Dev Container (VS Code / IDEs)

Open this repository in VS Code or any editor supporting Dev Containers to develop seamlessly inside the python:3.12-slim container.

---

Installation via pip / pipx

Install AskLLM from PyPI:

# Recommended for CLI tools
pipx install askllm-cli

# Or via standard pip
pip install askllm-cli

Or install locally in editable mode in any Python virtual environment:

# Editable install
pip install -e .

# With dev dependencies (pytest)
pip install -e ".[dev]"

Once installed, the askllm command is directly available:

askllm --help

Or run as a module:

python3 -m askllm

---

Production Container

Build and run the production image using Podman:

podman build -t askllm .
podman run --rm -it --network=host -e OPENAI_API_KEY askllm

---

Environment Variables

AskLLM automatically checks for standard OpenAI variables, all of which can be overridden by their ASKLLM_ counterparts:

Variable (AskLLM)	Fallback / Alias (OpenAI)	Description	Default
ASKLLM_API_KEY	OPENAI_API_KEY	Your API key	(empty / none)
ASKLLM_BASE_URL / ASKLLM_ENDPOINT / ASKLLM_API_BASE	OPENAI_BASE_URL / OPENAI_ENDPOINT / OPENAI_API_BASE	API base URL or chat completions endpoint	https://api.openai.com/v1
ASKLLM_MODEL / ASKLLM_MODEL_NAME	OPENAI_MODEL / OPENAI_MODEL_NAME / MODEL	Default model name	gpt-4o-mini
ASKLLM_SYSTEM_PROMPT	OPENAI_SYSTEM_PROMPT	Custom system prompt	"You are a helpful assistant."
ASKLLM_CODE_EXECUTION	OPENAI_CODE_EXECUTION	Enable Python code execution tool	false
ASKLLM_HISTORY_FILE	OPENAI_HISTORY_FILE	Path to persistent REPL history file	~/.askllm_history
ASKLLM_API_MODE	OPENAI_API_MODE	API mode (auto, chat, responses)	auto

---

Using with Local LLMs (Ollama, LM Studio, vLLM, etc.)

Because ./askllm and ./dev.sh use host networking (--network=host), you can connect directly to local servers:

Ollama:

ASKLLM_ENDPOINT="http://localhost:11434/v1" ASKLLM_MODEL="llama3.2" ./askllm

LM Studio / LocalAI / vLLM:

ASKLLM_ENDPOINT="http://localhost:1234/v1" ASKLLM_MODEL="local-model" ./askllm

---

REPL Slash Commands

Inside the REPL, type / to access built-in commands:

Command	Action
/help	Display command help and tips
/clear or /reset	Clear session conversation history and code execution namespace
/model [name]	Show or dynamically switch model
/endpoint	Show currently configured endpoint URL and active API mode
/system [prompt]	Show or update the system prompt
/tools [on|off]	Show or toggle Python code execution tool
/history	Show full message history for the session
/paste	Enter multiline paste mode
/exit or /quit	Exit the REPL

---

Command-Line Options

usage: askllm [-h] [-v] [-m MODEL] [-e ENDPOINT] [-x] [-k API_KEY] [-s SYSTEM]
              [-H HISTORY_FILE] [--api-mode {auto,chat,responses}]

options:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  -m MODEL, --model MODEL
                        LLM model name (env: ASKLLM_MODEL, OPENAI_MODEL, default: gpt-4o-mini)
  -e ENDPOINT, --endpoint ENDPOINT
                        API Base URL / Endpoint (env: ASKLLM_BASE_URL, OPENAI_BASE_URL, OPENAI_ENDPOINT, OPENAI_API_BASE)
  -x, --code-execution
                        Enable Python code execution tool for the LLM (env: ASKLLM_CODE_EXECUTION, OPENAI_CODE_EXECUTION)
  -k API_KEY, --api-key API_KEY
                        API Key (env: ASKLLM_API_KEY, OPENAI_API_KEY)
  -s SYSTEM, --system SYSTEM
                        System prompt (env: ASKLLM_SYSTEM_PROMPT, OPENAI_SYSTEM_PROMPT)
  -H HISTORY_FILE, --history-file HISTORY_FILE
                        Path to history file (env: ASKLLM_HISTORY_FILE, OPENAI_HISTORY_FILE, default: ~/.askllm_history)
  --api-mode {auto,chat,responses}
                        API mode: auto (probe /v1/responses then fallback to /v1/chat/completions),
                        chat, or responses (env: ASKLLM_API_MODE, OPENAI_API_MODE, default: auto)