rosa-llm

OpenAI through ChatGPT OAuth and local Ollama, behind one Python interface.

Use it as a standalone client or supply ChatRosaLLM to LangChain and ROSA. The package handles model communication; your application executes tools and owns its agent logic, experiment design, and outcome checks.

v0.1.0 · Python ≥3.11 · Apache-2.0

Install · Provider setup · Python API · LangChain and ROSA · Docker · Configuration · Troubleshooting

Install

In an existing uv project

For the base client:

uv add "rosa-llm==0.1.0"

For LangChain or ROSA, use this dependency instead:

uv add "rosa-llm[langchain]==0.1.0"

These commands install the PyPI release. uv add updates the consuming project's dependencies and lockfile; ==0.1.0 selects this exact version. No PyPI or GitHub sign-in is required to install it. If your project already declares this dependency, use its uv sync --locked; a separate package clone is unnecessary.

Version 0.1.0 renumbers the initial release with the same runtime code previously published as 0.2.0. That earlier upload remains available. Use the explicit pin above: an unpinned install can select 0.2.0 because it sorts higher.

Python compatibility: the base package and adapter require ≥3.11 and have been tested on 3.11 and 3.12 under Linux/WSL x86_64. This checkout and Docker use 3.12. Use Python 3.12 for the installed ROS 2 Jazzy bindings; the package does not install ROS. See verified versions and limits.

uv add "rosa-llm[langchain] @ git+https://github.com/Lelevinson/rosa-llm-package.git@v0.1.0"

uv add --marker "python_version >= '3.11'" \
  "rosa-llm[langchain]==0.1.0"

Run the CLI from a checkout

With Git, Python 3.12, and uv available, run in your Linux or WSL terminal:

git clone https://github.com/Lelevinson/rosa-llm-package.git
cd rosa-llm-package
uv sync --locked
uv run rosa-llm init

git clone downloads the project; cd enters it. uv sync --locked installs the locked environment in .venv; uv run executes a command there. init creates .env and refuses to overwrite it. Skip init if you already have configuration. See uv's sync behavior.

Provider setup

Choose a provider below. YOUR_OLLAMA_MODEL and YOUR_OPENAI_MODEL are placeholders: replace them with names returned by the relevant catalog command.

Ollama

1. Configure the server address in your local .env:

LLM_PROVIDER=ollama
OLLAMA_BASE_URL=http://localhost:11434

In WSL, a Windows-hosted server may need the Windows host address instead. Use the address reachable from your own machine; the client does not switch endpoints, start servers, or download models.

2. Discover a model and make a request:

uv run rosa-llm models --provider ollama --json
uv run rosa-llm ask "Say hello" --provider ollama --model YOUR_OLLAMA_MODEL --json

Save that model as OLLAMA_MODEL in .env to avoid repeating --model. Tool use requires a tool-capable model.

OpenAI — Sign in with ChatGPT

1. Authorize your own account:

uv run rosa-llm login --provider openai
uv run rosa-llm status --provider openai --json

Login opens a browser and listens on 127.0.0.1:1455/auth/callback. Identity login and plan-use permission are separate; inference needs both. status reads local state and does not confirm current remote access.

2. Discover a model, then request it explicitly:

ROSA_LLM_REQUIRE_EXPLICIT_MODEL=false uv run rosa-llm models --provider openai --json
uv run rosa-llm ask "Say hello" --provider openai --model YOUR_OPENAI_MODEL --json

The assignment before models disables the explicit-model guard for that command only, allowing discovery before a model is configured. It does not change .env. --provider and --model override configuration for one command.

3. Save your selection in .env:

LLM_PROVIDER=openai
OPENAI_MODEL=YOUR_OPENAI_MODEL
ROSA_LLM_REQUIRE_EXPLICIT_MODEL=true

Credentials are saved outside the repository. Each teammate signs in separately; GitHub access does not provide model access. An OpenAI API key is not used here.

Python API

The base API needs no ROS, LangChain, database, or agent runtime:

from rosa_llm import create_client

with create_client() as client:
    response = client.generate("Explain a ROS 2 node briefly.")
    print(response.text)
    print(response.model, response.usage)

create_client() loads configuration from the calling directory. For explicit settings, pass Config(...), or use create_client(env_file="path/to/.env"). Keep client use inside with so its HTTP resources close afterward.

Plain text history is portable across providers; opaque provider history is not. OpenAI maps system instructions to developer messages. Ollama accepts system messages and rejects developer messages. Tool arguments are complete only after provider completion; do not execute partial streamed arguments.

Recipes: chat/history · native tool loop · request recording and timing.

LangChain and ROSA

Install with [langchain] as shown above. In this checkout, enable the extra with uv sync --locked --extra langchain; it pins langchain-core==0.3.86.

from rosa_llm import create_client
from rosa_llm.langchain import ChatRosaLLM

with create_client() as client:
    model = ChatRosaLLM(client=client)
    print(model.invoke("Say hello").content)

Use ordinary LangChain tools through model.bind_tools(...) or an agent's normal model interface. LangChain/your application executes tools; this adapter does not. Keep the client open until all agent calls finish.

from rosa import ROSA
from rosa_llm import create_client
from rosa_llm.langchain import ChatRosaLLM

with create_client() as client:
    model = ChatRosaLLM(client=client)
    agent = ROSA(ros_version=2, llm=model, streaming=False)
    print(agent.invoke("List the current ROS 2 nodes."))

Unsupported options fail explicitly. Do not assume temperature/seed controls, strict schemas, hosted tools, images/files, or arbitrary OpenAI parameters are portable across providers. There are no constructor-level decoding defaults on ChatRosaLLM in v0.1.0; supplying per-call options and retaining them through an agent's tool-binding step is the caller's responsibility.

See integration examples, custom LangChain tools, and the compatibility report for exact supported scope.

Docker

Status: Dockerfiles/Compose configuration are provided. Image build, runtime, and credential persistence have not been live-verified in this environment. See verification limits.

The image is a standalone Python 3.12 client, not a complete ROS/Jazzy environment. For an existing ROS container, install the Git dependency into that application's environment. This image defaults to the base client; build with --build-arg INSTALL_LANGCHAIN=true to include the optional adapter.

Create a local .env from .env.example if one does not already exist, then:

docker compose build
docker compose run --rm rosa-llm models --provider ollama --json
docker compose run --rm rosa-llm ask "Say hello" --provider ollama --model YOUR_OLLAMA_MODEL --json

build creates the image. run --rm removes the command container when it exits; the named credential volume persists. Compose loads .env.

OAuth login in Docker

On native Linux, or Docker Desktop with host networking enabled:

docker compose -f compose.yaml -f compose.login.yaml run --rm rosa-llm login --no-browser
docker compose run --rm -e ROSA_LLM_REQUIRE_EXPLICIT_MODEL=false rosa-llm models --provider openai --json
docker compose run --rm rosa-llm ask "Say hello" --provider openai --model YOUR_OPENAI_MODEL --json

-f selects Compose files; the later file overrides the earlier one. -e sets an environment variable inside that command's container. Open the printed URL in the browser on that host while login waits. The listener remains on loopback. Bridge networking and publishing a port do not make a listener bound to the container's 127.0.0.1 reachable from the host. The project does not enable host networking or change host firewalls/Ollama bindings for you.

If host networking is unavailable, use an existing SSH tunnel, or complete WSL login and securely move your own credential/host-identity directory into your private volume with the correct ownership. Move the runtime identity together; do not operate concurrent independent copies. Never share credential volumes between teammates. Use the same Compose project/volume across commands, and avoid docker compose down -v unless credential deletion is intentional (-v deletes named volumes). Native Windows Python is outside this release's tested scope.

Configuration

Edit the .env in the directory from which your application runs. No tokens or API keys belong in this file.

Precedence, highest first: process environment → local .env → user config.env → defaults. The user directory is $XDG_CONFIG_HOME/rosa-llm, normally ~/.config/rosa-llm. uv run rosa-llm init --user creates its config.env. The loader does not change process environment or expand ${...} in dotenv values. For reproducible runs, select a named model rather than relying on catalog order.

CLI reference

All options follow the subcommand; for example, uv run rosa-llm ask --help.

Common overrides: --provider, --model, --profile, --env-file PATH. Diagnostics go to stderr. JSON-mode failures emit an error object and exit 1. Inference never opens a login prompt or automatically retries/falls back. Partial streaming output is not proof of success; check the exit status.

Credentials and transport

• Default storage: ~/.config/rosa-llm/credentials.json and host.json. Profiles belong to this package; it does not reuse Codex credentials. Files are plaintext protected by OS permissions, not encrypted.
• Storage directories use mode 700; credential, host, temporary, and lock files use mode 600. Refreshes atomically rotate tokens under a Linux process lock. Local filesystems are required; NFS/distributed locking is outside scope.
• Expired tokens renew automatically when possible. Revoked sessions need login again; temporary network failures preserve credentials. Keep credentials and authorization URLs private. Never put GitHub/model tokens in Git URLs or logs.
• Redirects and environment proxy variables are disabled on internally created HTTP clients; TLS verification stays enabled. Do not enable HTTP wire/debug logging around OAuth, because it can expose secrets.

Troubleshooting

Development and reference

uv sync --locked --extra langchain
uv run --locked pytest -q
uv run --locked ruff check .
uv run --locked ruff format --check .
uv build

pytest -q runs automated tests with concise output; it does not authorize accounts or run live model requests. Third-party pytest plugin auto-loading is disabled. Optional ROSA executor checks also need the integration group and sourced Jazzy; see the integration guide. ruff check checks code; format --check checks formatting without editing. uv build creates a wheel and source distribution under dist/.