linode-llm-toolkit

Unofficial project. Not affiliated with, endorsed by, or supported by Akamai Technologies, Inc. or Linode LLC. Akamai and Linode are trademarks of their respective owners.

linode-llm-toolkit is an independently maintained, experimental open-source CLI for deploying a small, reviewed catalog of Ollama-hosted LLMs on Linode instances. The compatibility command is linode-llm; w7089 is the maintainer and release authority.

One-command LLM deployment is limited to the reviewed public-alpha instance path described below.

Quickstart: deploy a personal model

You need Python 3.12+, a Linode account, and a Linode API token. Install the exact alpha with uv:

uv tool install 'linode-llm-toolkit==0.1.0a4'

Then deploy with only the token:

export LINODE_TOKEN='…'
linode-llm deploy

In a terminal, deploy shows only reviewed catalog models for which this account currently has a viable live placement and complete price. Select one, or deploy a known catalog model directly:

linode-llm deploy qwen2.5:1.5b

The CLI calculates sizing, cheapest eligible plan, region capacity, and live price behind the scenes, then shows the resolved model, plan, region, and price before creating anything. Non-interactive callers that omit the model use the reviewed Qwen 1.5B default.

When the model is ready, the success output includes values ready for an OpenAI-compatible app:

export OPENAI_BASE_URL='http://<endpoint>/v1'
export OPENAI_MODEL='<selected-model>'
export OPENAI_API_KEY='not-needed'

WebUI and SSH are off. The CLI detects your public IPv4 address and creates a default-deny firewall that allows plain HTTP on port 80 only from that address's /32.

This is a personal prototype boundary, not production security: the endpoint is plain HTTP with no application credential, protected only by source-IP filtering. Do not use it for sensitive prompts or data, shared access, or a public service. Resources remain billable until you destroy them. Save the deployment ID from the success output, then run exactly:

linode-llm destroy <deployment-id>

See Simple deployment for the tested lifecycle, exact boundary, app setup, and cleanup.

Status and limitations

This is not a production service and carries no production or Marketplace support commitment. The instance path is the public-alpha path; the bundled LKE/Helm chart and StackScript are experimental developer-preview paths and are not Marketplace release candidates.

Before using it, understand these limits:

• A deployment needs a live, authenticated Linode API overlay. Cached plan data is for offline planning only and cannot establish account or regional availability.
• Billing can continue until you remove billable resources; a guest shutdown does not make its Linode free.
• The tool records local state, but state can be lost or damaged. Use reconcile and destroy --plan to inspect owned remote resources before cleanup.
• Runtime artifacts are pinned for this experimental profile, but the runtime/deployment release profile remains fail-closed and is not release-eligible. The published production-PyPI alpha 0.1.0a4 is a package-scoped release only; it does not change that runtime boundary. See RELEASING.md.

Installation alternatives

The package-scoped 0.1.0a4 alpha is available from PyPI. If you do not use uv, install it with pip inside a Python 3.12+ virtual environment:

python3.12 -m venv .venv
. .venv/bin/activate
python -m pip install 'linode-llm-toolkit==0.1.0a4'
linode-llm --help

Do not install into the operating system's Python with a bare pip install. Debian and Ubuntu protect that environment under PEP 668; activating a virtual environment and using python -m pip also makes it clear which interpreter receives the CLI.

This package publication does not make the deployment runtime, LKE chart, or StackScript production-ready or Marketplace-supported. The earlier TestPyPI 0.1.0a1 rehearsal remains transport evidence only.

For a source checkout, use uv sync --extra dev and uv run linode-llm --help.

Reviewed catalog

The alpha catalog snapshot is phase0-alpha-2026-08-04. Inspect the installed snapshot rather than copying a model name from older material:

linode-llm catalog
linode-llm catalog --json

Entries with additional acceptance requirements need their exact catalog ID. The permissively licensed simple default does not require a separate acknowledgement:

The models command also shows pre-alpha compatibility records. Those are legacy_unverified, remain discoverable for compatibility, and cannot be planned or deployed. Artifact identities, license evidence, and acceptance requirements are in MODEL_NOTICES.md.

Advanced options

Authenticated TLS, credentials, licenses, and explicit placement

Set a Linode API token only in your environment; never commit or print it. A plan and deployment also require separate inference and administration endpoint credentials. The values are resolved at use time from references and only references are stored in deployment state.

export LINODE_TOKEN='…'
export INFERENCE_KEY='…'
export ADMINISTRATION_KEY='…'

linode-llm plan \
  --model gpt-oss:20b \
  --accept-license gpt-oss-20b-mxfp4 \
  --budget 400 \
  --maximum-lifetime-hours 24 \
  --fqdn llm.example.com

linode-llm deploy \
  --model gpt-oss:20b \
  --accept-license gpt-oss-20b-mxfp4 \
  --budget 400 \
  --maximum-lifetime-hours 24 \
  --fqdn llm.example.com \
  --inference-credential environment:INFERENCE_KEY \
  --administration-credential environment:ADMINISTRATION_KEY

Public mode requires a caller-managed FQDN and Caddy automatic HTTPS. Without a domain, use authenticated, source-restricted developer mode instead:

linode-llm deploy \
  --model gpt-oss:20b \
  --accept-license gpt-oss-20b-mxfp4 \
  --budget 400 \
  --maximum-lifetime-hours 24 \
  --allow-cidr 203.0.113.0/24 \
  --inference-credential environment:INFERENCE_KEY \
  --administration-credential environment:ADMINISTRATION_KEY

When explicitly selecting an advanced security profile, use one of --fqdn, --allow-cidr, or the public-HTTP acknowledgement. Omitting all three selects the source-restricted personal-prototype default. Public anonymous HTTP is experimental and requires the exact acknowledgement I_ACKNOWLEDGE_PUBLIC_HTTP_IS_UNSAFE.

Use the inference credential for /v1/* and the separate administration credential for /api/*. Do not place either value in source code, a shell history you share, or deployment state.

Security

For an advanced internet-facing deployment, use authenticated public TLS or authenticated source-restricted developer mode. Provide an SSH public key with --ssh-key when you need instance login; without one, you cannot log in to the instance. See SECURITY.md for private reporting and the full security boundary.

Cost and planning

For advanced cost policy, run linode-llm plan before deploy. It is mutation-free and emits a stable, itemized JSON plan. It distinguishes live, cached, estimated, and unknown amounts; unknown amounts are never treated as zero. When --budget is supplied, a rejected budget plan cannot proceed to deployment. The simple path displays live compute pricing without requiring this guard.

--budget is a pre-deploy estimate guard, not spend enforcement. --maximum-lifetime-hours is a planning horizon; it does not schedule deletion.

linode-llm cost displays the approved cached snapshot linode-plans-2026-08-04, observed 2026-08-04, for offline planning only. It is not live pricing. The approved cached hourly/monthly values in USD are:

Actual regional pricing, capacity, account restrictions, storage, network, and other charges can differ or be unknown. The plan reports exclusions and unknowns explicitly.

Cleanup and recovery

Inspect before deleting, then remove the deployment and verify it is gone:

linode-llm reconcile
linode-llm destroy --plan <deploy-id>
linode-llm destroy <deploy-id>
linode-llm reconcile

reconcile classifies resources as owned, foreign, or ambiguous. Foreign and ambiguous resources are not automatically adopted or deleted. Check the Linode control plane as well: cleanup is your responsibility, and there is no approved automatic account-side expiry deletion mechanism in this alpha.

Commands

Run linode-llm --help for the authoritative options. Older compatibility-oriented commands may expose records that are not deployable; use catalog and plan for alpha decisions.

Support, compatibility, and security

• SUPPORT.md defines the best-effort community support boundary.
• COMPATIBILITY.md describes the pre-1.0 compatibility and deprecation policy.
• SECURITY.md explains private vulnerability reporting.
• THIRD_PARTY_NOTICES.md and MODEL_NOTICES.md identify reviewed third-party/runtime and model notices.
• RELEASING.md documents the alpha release gate; docs/REPOSITORY_SETTINGS_CHECKLIST.md lists human GitHub/PyPI settings.

For a guided introduction see docs/BEGINNER_GUIDE.md; contributors should start with docs/DEVELOPER_GUIDE.md.