linode-llm-toolkit 0.1.0a2

Unofficial one-command LLM deployment CLI for Akamai/Linode cloud

linode-llm-toolkit

Unofficial project. Not affiliated with, endorsed by, or supported by Akamai Technologies, Inc. or Linode LLC. Akamai and Linode are trademarks of their respective owners.

linode-llm-toolkit is an independently maintained, experimental open-source CLI for planning and deploying a small, reviewed set of Ollama-hosted LLMs on Linode instances. The compatibility command is linode-llm; w7089 is the maintainer and release authority.

One-command LLM deployment is limited to the reviewed public-alpha instance path described below.

Status and limitations

This is not a production service and carries no production or Marketplace support commitment. The instance path is the public-alpha path; the bundled LKE/Helm chart and StackScript are experimental developer-preview paths and are not Marketplace release candidates.

Before using it, understand these limits:

• A deployment needs a live, authenticated Linode API overlay. Cached plan data is for offline planning only and cannot establish account or regional availability.
• --budget is a pre-deploy estimate guard, not spend enforcement. --maximum-lifetime-hours is a planning horizon; it does not schedule deletion. Billing can continue until you remove billable resources.
• The tool records local state, but state can be lost or damaged. Use reconcile and destroy --plan to inspect owned remote resources before cleanup.
• Runtime artifacts are pinned for this experimental profile, but the runtime/deployment release profile remains fail-closed and is not release-eligible. A production-PyPI alpha, when published, is a package-scoped release only; it does not change that runtime boundary. See RELEASING.md.

Install

When a production-PyPI alpha has been published, install that released version with:

uv tool install linode-llm-toolkit
# or
pip install linode-llm-toolkit

linode-llm --help

Publication has not happened merely because a TestPyPI rehearsal exists. The recorded TestPyPI 0.1.0a1 rehearsal is evidence for the same-artifact release procedure, not a claim that a production-PyPI alpha is already available.

For a source checkout, use uv sync --extra dev and uv run linode-llm --help.

Reviewed catalog

The alpha catalog snapshot is phase0-alpha-2026-08-01. Inspect the installed snapshot rather than copying a model name from older material:

linode-llm catalog
linode-llm catalog --json

Deployable entries require acceptance of their exact catalog ID:

Catalog ID	Alias	Quantization	Context	Preferred reviewed plan
qwen2.5-1.5b-instruct-q4-k-m	qwen2.5:1.5b	Q4_K_M	32,768 tokens	g6-dedicated-16
gpt-oss-20b-mxfp4	gpt-oss:20b	MXFP4	131,072 tokens	g2-gpu-rtx4000a1-s
gpt-oss-120b-mxfp4	gpt-oss:120b	MXFP4	131,072 tokens	g3-gpu-rtxpro6000-blackwell-1

The models command also shows pre-alpha compatibility records. Those are legacy_unverified, remain discoverable for compatibility, and cannot be planned or deployed. Artifact identities, license evidence, and acceptance requirements are in MODEL_NOTICES.md.

Secure quickstart

Set a Linode API token only in your environment; never commit or print it. A plan and deployment also require separate inference and administration endpoint credentials. The values are resolved at use time from references and only references are stored in deployment state.

export LINODE_TOKEN='…'
export INFERENCE_KEY='…'
export ADMINISTRATION_KEY='…'

linode-llm plan \
  --model gpt-oss:20b \
  --accept-license gpt-oss-20b-mxfp4 \
  --budget 400 \
  --maximum-lifetime-hours 24 \
  --fqdn llm.example.com

linode-llm deploy \
  --model gpt-oss:20b \
  --accept-license gpt-oss-20b-mxfp4 \
  --budget 400 \
  --maximum-lifetime-hours 24 \
  --fqdn llm.example.com \
  --inference-credential environment:INFERENCE_KEY \
  --administration-credential environment:ADMINISTRATION_KEY

Public mode requires a caller-managed FQDN and Caddy automatic HTTPS. Without a domain, use authenticated, source-restricted developer mode instead:

linode-llm deploy \
  --model gpt-oss:20b \
  --accept-license gpt-oss-20b-mxfp4 \
  --budget 400 \
  --maximum-lifetime-hours 24 \
  --allow-cidr 203.0.113.0/24 \
  --inference-credential environment:INFERENCE_KEY \
  --administration-credential environment:ADMINISTRATION_KEY

Exactly one of --fqdn, --allow-cidr, or the explicit public-HTTP acknowledgement is required. Public HTTP is deliberately excluded from this quickstart; it is experimental and requires the exact acknowledgement I_ACKNOWLEDGE_PUBLIC_HTTP_IS_UNSAFE.

Use the inference credential for /v1/* and the separate administration credential for /api/*. Do not place either value in source code, a shell history you share, or deployment state.

Security

The secure default is authenticated public TLS or authenticated source-restricted developer mode. Provide an SSH public key with --ssh-key when you need instance login; without one, you cannot log in to the instance. Do not disable the default boundary to obtain a simpler example. See SECURITY.md for private reporting and the full security boundary.

Cost and planning

Run linode-llm plan before deploy. It is mutation-free and emits a stable, itemized JSON plan. It distinguishes live, cached, estimated, and unknown amounts; unknown amounts are never treated as zero. A rejected budget plan cannot proceed to deployment.

linode-llm cost displays the approved cached snapshot linode-plans-2026-08-01, observed 2026-08-01T15:20:40Z, for offline planning only. It is not live pricing. The approved cached hourly/monthly values in USD are:

Plan	Hardware	Hourly	Monthly	Availability note
g6-dedicated-16	CPU	$0.432	$288.00	Live overlay still required
g2-gpu-rtx4000a1-s	NVIDIA RTX 4000 Ada (20 GiB)	$0.52	$350.00	Live overlay still required
g1-gpu-rtx6000-1	NVIDIA Quadro RTX 6000 (24 GiB)	$1.50	$1,000.00	Limited availability
g3-gpu-rtxpro6000-blackwell-1	NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GiB)	$2.50	$1,665.00	Limited availability; must appear in the authenticated live API for the account and region

Actual regional pricing, capacity, account restrictions, storage, network, and other charges can differ or be unknown. The plan reports exclusions and unknowns explicitly.

Cleanup and recovery

Inspect before deleting, then remove the deployment and verify it is gone:

linode-llm reconcile
linode-llm destroy --plan <deploy-id>
linode-llm destroy <deploy-id>
linode-llm reconcile

reconcile classifies resources as owned, foreign, or ambiguous. Foreign and ambiguous resources are not automatically adopted or deleted. Check the Linode control plane as well: cleanup is your responsibility, and there is no approved automatic account-side expiry deletion mechanism in this alpha.

Commands

Run linode-llm --help for the authoritative options. Older compatibility-oriented commands may expose records that are not deployable; use catalog and plan for alpha decisions.

Command	Purpose
linode-llm bench	Benchmark a deployed LLM.
linode-llm catalog	Inspect the reviewed catalog snapshot.
linode-llm chat	Open an interactive chat session with a deployment.
linode-llm cost	Display dated cached cost estimates.
linode-llm deploy	Validate and create an instance deployment.
linode-llm destroy	Preview or remove a deployment.
linode-llm init	Create an application starter.
linode-llm logs	Show deployment diagnostics.
linode-llm models	List deployable and legacy compatibility records.
linode-llm plan	Create a mutation-free deployment plan.
linode-llm ps	List models on a deployment.
linode-llm pull	Pull an additional model on a deployment.
linode-llm reconcile	Inspect owned, foreign, and ambiguous resources.
linode-llm rm	Remove a model from a deployment.
linode-llm status	Check deployment status.
linode-llm templates	List application starters.

Support, compatibility, and security

• SUPPORT.md defines the best-effort community support boundary.
• COMPATIBILITY.md describes the pre-1.0 compatibility and deprecation policy.
• SECURITY.md explains private vulnerability reporting.
• THIRD_PARTY_NOTICES.md and MODEL_NOTICES.md identify reviewed third-party/runtime and model notices.
• RELEASING.md documents the alpha release gate; docs/REPOSITORY_SETTINGS_CHECKLIST.md lists human GitHub/PyPI settings.

For a guided introduction see docs/BEGINNER_GUIDE.md; contributors should start with docs/DEVELOPER_GUIDE.md.