AskLLM

A lightweight, zero-dependency Python REPL for chatting with LLMs via any OpenAI-compatible API.

Designed to be installed via pip install askllm-cli (or pipx install askllm-cli), and developed using Podman with python:3.12-slim (or run directly with Python 3).

---

Features

• Zero Third-Party Runtime Dependencies: Uses standard library only (urllib.request, json, readline).
• Standard Python Package: Modern PEP 517/621 packaging (pyproject.toml) published as askllm-cli on PyPI.
• Podman Development Ready: Built-in development workflow using python:3.12-slim (./dev.sh).
• Environment Variable Auto-Detection: Automatically picks up standard OpenAI / Azure / local LLM variables.
• Streaming Responses: Real-time token streaming via Server-Sent Events (SSE).
• Graceful Signal Handling:
  • Ctrl+C or Ctrl+D at the beginning of the line exits the REPL.
  • Ctrl+C with text in the buffer cancels the line.
  • Ctrl+C during response streaming halts generation cleanly without exiting.
• Multi-Turn Chat History: Maintains conversation context across turns within the session.
• Persistent Input History: Readline history saved across runs in ~/.askllm_history.
• Multiline Input Support:
  • Triple quotes """ ... """
  • Trailing backslash \
  • /paste command

---

Development with Podman (python:3.12-slim)

No local Python installation or host dependencies are required. A complete development environment is provided via Podman:

1. Interactive Development Shell

Drop into an interactive bash shell running inside python:3.12-slim with AskLLM installed in editable mode (pip install -e .):

./dev.sh

Inside the shell:

# Run AskLLM directly
askllm --help

# Run tests
pytest

# Test interactive python
python3 -c "import askllm; print(askllm.__version__)"

Any changes made to files in src/askllm on your host are immediately reflected inside the container!

2. Run Tests in Podman

Run the test suite inside the python:3.12-slim container:

./dev.sh test

3. Run AskLLM via Podman

Run AskLLM using Podman directly from the host:

./askllm

or with arguments:

./askllm --model llama3.2 --endpoint http://localhost:11434/v1

4. Dev Container (VS Code / IDEs)

Open this repository in VS Code or any editor supporting Dev Containers to develop seamlessly inside the python:3.12-slim container.

---

Installation via pip / pipx

Install AskLLM from PyPI:

# Recommended for CLI tools
pipx install askllm-cli

# Or via standard pip
pip install askllm-cli

Or install locally in editable mode in any Python virtual environment:

# Editable install
pip install -e .

# With dev dependencies (pytest)
pip install -e ".[dev]"

Once installed, the askllm command is directly available:

askllm --help

Or run as a module:

python3 -m askllm

---

Production Container

Build and run the production image using Podman:

podman build -t askllm .
podman run --rm -it --network=host -e OPENAI_API_KEY askllm

---

Environment Variables

AskLLM automatically checks for these environment variables:

Variable	Description	Default
OPENAI_API_KEY	Your API key	(empty / none)
OPENAI_ENDPOINT or OPENAI_BASE_URL or OPENAI_API_BASE	API base URL or chat endpoint	https://api.openai.com/v1
OPENAI_MODEL	Default model name	gpt-4o-mini
OPENAI_SYSTEM_PROMPT	Custom system prompt	"You are a helpful assistant."

---

Using with Local LLMs (Ollama, LM Studio, vLLM, etc.)

Because ./askllm and ./dev.sh use host networking (--network=host), you can connect directly to local servers:

Ollama:

OPENAI_ENDPOINT="http://localhost:11434/v1" OPENAI_MODEL="llama3.2" ./askllm

LM Studio / LocalAI / vLLM:

OPENAI_ENDPOINT="http://localhost:1234/v1" OPENAI_MODEL="local-model" ./askllm

---

REPL Slash Commands

Inside the REPL, type / to access built-in commands:

Command	Action
/help	Display command help and tips
/clear or /reset	Clear session conversation history
/model [name]	Show or dynamically switch model
/endpoint	Show currently configured endpoint URL
/system [prompt]	Show or update the system prompt
/history	Show full message history for the session
/paste	Enter multiline paste mode
/exit or /quit	Exit the REPL

---

Command-Line Options

usage: askllm [-h] [-v] [-m MODEL] [-e ENDPOINT] [-k API_KEY] [-s SYSTEM]

options:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  -m MODEL, --model MODEL
                        LLM model name (env: OPENAI_MODEL, default: gpt-4o-mini)
  -e ENDPOINT, --endpoint ENDPOINT
                        API Base URL / Endpoint (env: OPENAI_BASE_URL, OPENAI_ENDPOINT, OPENAI_API_BASE)
  -k API_KEY, --api-key API_KEY
                        API Key (env: OPENAI_API_KEY)
  -s SYSTEM, --system SYSTEM
                        System prompt (env: OPENAI_SYSTEM_PROMPT)