kiapi

English | 日本語

Summary

kiapi is an API server for providing the following capabilities to LLM agents on a Mac Studio M4 Max 128GB.

• Chat:
  • OpenAI Chat Completions API compatible
  • text + image + audio + video input support
  • tool call + tool choice (auto, any, specific) + parallel tool calls + streaming support
• Embedding:
  • text + image input support
• Image generation:
  • text2image, image2image, image editing, and LoRA training support
• Music and sound-effect generation:
  • text2audio, cover, repaint, and extract support
• Video generation:
  • text2video, image2video, and audio2video support
• Web:
  • search + fetch support

To provide every capability stably from a single PC, kiapi has these properties.

• GPU work is queued and executed one job at a time
• Application memory is managed to avoid overcommit failures

kiapi is also designed so LLM agents can understand and operate its capabilities.

• The API server can explain how to use itself to an LLM
• Asynchronous task progress can be observed
• Generation tasks can run in both sync / async modes

[!IMPORTANT]

kiapi itself is MIT-licensed OSS, but the packages and models it provides have various licenses. Before use, check the dependency packages and models on each capability page and confirm the license of the model you use.

Model and Dependency Licenses

The table below summarizes the upstream licenses for the default models and runtime resources that kiapi can activate. It is a convenience checklist, not legal advice. License labels and gating status can change upstream, so always check the linked source before commercial use, redistribution, or offering a hosted service.

Review date: 2026-06-23.

API

API Docs

• kiapi API Docs
  • OpenAPI JSON
  • Swagger UI
  • ReDoc

Requirements

• macOS / Apple Silicon
• Python >=3.12,<3.13
• uv (optional, recommended for isolated CLI installs and faster venv/package setup in kiapi activate)
• mise (used for development)
• Docker (when using the Web capability)
• Enough disk capacity for model weights and Docker images

kiapi is developed mainly for personal use on a Mac Studio M4 Max 128GB. Some or all features may work on other Apple Silicon environments, but they are not the primary verification target.

The memory budget can be specified with KIAPI_MEMORY_LIMIT_GB. If omitted, kiapi automatically uses 80% of installed memory as the effective budget on startup. If a model's required memory does not fit in that budget, requests return 503 as an insufficient memory budget error.

kiapi activate --all uses a little under 600GB of disk capacity, including model weights and Docker images. At first, it is recommended to use kiapi activate to set up only the capabilities you need.

Quick Start

From installation to agent integration:

# Install kiapi itself
python3.12 -m pip install --upgrade kiapi  # when uv is unavailable
uv tool install --python 3.12 kiapi  # when uv is available

# Change default host, port, or memory budget if needed
kiapi config init
kiapi config edit

# Check setup status
kiapi status

# Explicit setup for model weights, Docker images, and dedicated venvs
kiapi activate  # select targets from the displayed list
kiapi activate --all  # set up everything (a little under 600GB)
kiapi activate --family acestep  # set up only one family

# Verify behavior
kiapi check  # select targets from the displayed list
kiapi check --all  # check everything

# Start the API server
kiapi run  # starts on 127.0.0.1:8000
kiapi run --host 0.0.0.0 --port 8500  # specify host and port

# Example agent integration
codex e "
Please understand http://localhost:8000/openapi.json.
Using the music generation API, generate a 20-second BGM themed 'a person walking in the rain' at ~/Downloads/bgm.wav.
"

# Check the generated file
open ~/Downloads/bgm.wav

Run as a background service:

# Register the service
kiapi service install

# Start the service
kiapi service start

# Check service status and log tail
kiapi service status

# Stop the service
kiapi service stop

# Remove the service
kiapi service uninstall

Remote Job Relay

The optional GCP relay lets a kiapi node inside a closed network receive API work without exposing an inbound socket. Firebase Realtime Database carries small notifications, while Cloud Storage carries request and response bodies. Install kiapi with the relay-gcp extra to enable it:

python3.12 -m pip install --upgrade "kiapi[relay-gcp]"
uv tool install --python 3.12 "kiapi[relay-gcp]"

export KIAPI_RELAY_GCP_NODE_ID="studio-1"
export KIAPI_RELAY_GCP_DATABASE_URL="https://PROJECT.firebaseio.com"
export KIAPI_RELAY_GCP_BUCKET="PRIVATE_RELAY_BUCKET"
export KIAPI_RELAY_GCP_PREFIX="private/kiapi"

# Uses Application Default Credentials by default.
kiapi run --relay gcp

The requester writes {prefix}/sessions/{session_id}/request.json in GCS, then writes a notification to {prefix}/nodes/{node_id}/requests/{session_id} in RTDB. The relay reports queued, running, and the terminal result below the requester node's responses path.

• Requests are dispatched directly to the in-process FastAPI app and handled one at a time by the relay.
• JSON responses use response.json. Binary responses write response.body before response.json.
• response.json uses a GCS create-only generation precondition. A completed response found after restart is reported without executing the API again.
• The terminal RTDB response and request deletion use one atomic multi-location update.
• Startup installs a prefix-scoped lifecycle rule that deletes session objects after one day. Set KIAPI_RELAY_GCP_MANAGE_BUCKET_LIFECYCLE=false when infrastructure manages this rule.

Use a dedicated bucket and narrowly scoped RTDB/GCS permissions. Google credentials are configured through kiarina-lib-google.

See GCP Relay setup for resource creation, IAM, authentication, configuration, and verification steps.

For local relay verification without GCP, use local. It uses the same in-process dispatch path but stores notifications and payloads under a local directory:

export KIAPI_RELAY_LOCAL_NODE_ID="studio-1"
export KIAPI_RELAY_LOCAL_ROOT="/tmp/kiapi/relay"
export KIAPI_RELAY_LOCAL_PREFIX="private/kiapi"

kiapi run --relay local

The requester writes {root}/{prefix}/sessions/{session_id}/request.json, then writes {root}/{prefix}/nodes/{node_id}/requests/{session_id}.json with {"session_id":"...","source_node_id":"..."}. The relay writes bridge status to {root}/{prefix}/nodes/{source_node_id}/responses/{session_id}.json and stores the committed response in the session directory.

Architecture

[!NOTE]

For kiapi architecture details, see ARCHITECTURE.md.

Local Storage

kiapi mainly writes to these local paths at runtime.

Other model weights and library caches are managed by Hugging Face, mflux, Docker, or each library/tool. kiapi generally does not move them into its own storage location.

Project Status

kiapi is OSS developed mainly for personal use. The API, supported models, and setup steps may change in the future.

Issues and Pull Requests are welcome, but this is a personal project and support is best-effort.

Security

By default, kiapi run starts on 127.0.0.1:8000. When --host 0.0.0.0 is specified, the server may be reachable from other machines, so use it only on trusted networks.

Development

# Install dependencies, download test data, and create the venv environment
make init

# Sync dependencies
make update

# Upgrade dependencies
make upgrade

# ... implement

# Format, type-check, and regenerate documentation under public/
make

# unit test
make test

# Start the development server (auto-reload supported)
make dev

# GPU feature tests / regression tests
make verify  # run all
make verify-fast  # run only light tests for all capabilities
make verify-one  # run one capability

Release

kiapi releases follow the same flow as pydantic-settings-manager: update the version, update the changelog, then push a tag that triggers GitHub Release and PyPI publishing.

# Update the version and release entry in CHANGELOG.md
make bump-version

# Or pass the version explicitly
mise run bump-version 0.2.0

# Local verification
make test
make
make build

# Release commit and tag
git add pyproject.toml CHANGELOG.md
git commit -m "chore(release): prepare v0.2.0"
git tag v0.2.0
git push origin main --tags

When a v*.*.* tag is pushed, the GitHub Actions release workflow builds the package, extracts release notes from CHANGELOG.md, creates a GitHub Release, and publishes to PyPI.