kiapi

Summary

kiapi is an API server that uses a Mac Studio M4 Max with 128GB of memory at home to provide the following capabilities.

See: API Documents

Resources

To provide its capabilities, kiapi downloads and uses selected resources from the list below. Each resource is governed by its upstream license. Always review the upstream license to confirm the terms and whether commercial use is permitted.

Design

Reliably provide every capability:

• Queue non-administrative requests and process them one at a time
• Manage API server memory to prevent overcommit failures

Support interactive integration with LLM agents:

• Provide LLMs with tips as well as I/O specifications through openapi.json
• Run generation tasks in both sync and async modes
• Make asynchronous task progress observable

Provide secure external access to kiapi inside a closed network:

• kiapi binds to 127.0.0.1 by default and never exposes an inbound socket by itself
• For access from other machines, put it behind your own private network layer, for example tailscale serve

API

See: kiapi API Docs

Requirements

• macOS / Apple Silicon
• Python >=3.12,<3.13
• uv (optional, recommended for isolated CLI installs and faster venv/package setup in kiapi activate)
• mise (used for development)
• Docker (when using the Web capability)
• Enough disk capacity for model weights and Docker images

kiapi is developed mainly for personal use on a Mac Studio M4 Max 128GB. Some or all features may work on other Apple Silicon environments, but they are not the primary verification target.

The memory budget can be specified with KIAPI_MEMORY_LIMIT_GB. If omitted, kiapi automatically uses 80% of installed memory as the effective budget on startup. If a model's required memory does not fit in that budget, requests return 503 as an insufficient memory budget error.

kiapi activate --all uses a little under 600GB of disk capacity, including model weights and Docker images. At first, it is recommended to use kiapi activate to set up only the capabilities you need.

Quick Start

Set up kiapi:

# Install kiapi
python3.12 -m pip install --upgrade kiapi  # If you cannot use uv
uv tool install --python 3.12 kiapi        # If you can use uv

# Change the default host, port, or memory budget if needed
kiapi config init  # Create the configuration file
kiapi config edit  # Edit the configuration file in an editor

# Check the current setup state
kiapi status

# Prepare model weights, Docker images, and dedicated venv environments
kiapi activate                   # Choose targets from the interactive list
kiapi activate --all             # Set up everything (just under 600GB)
kiapi activate --family acestep  # Set up only the specified family

# Verify the setup
kiapi check        # Choose targets from the interactive list
kiapi check --all  # Verify everything

Use from an LLM agent:

# Start the kiapi server
kiapi run                             # Start based on the configuration file (default: 127.0.0.1:8000)
kiapi run --host 0.0.0.0 --port 8500  # Start on a specific host and port

# Example integration with an agent
codex e "
Please inspect http://localhost:8000/openapi.json.
Use the music generation API to create a 20-second BGM track at ~/Downloads/bgm.wav
with the theme 'a person walking in the rain'.
"

# Inspect the generated file
open ~/Downloads/bgm.wav

Run as a background service:

# kiapi
kiapi service install    # Register
kiapi service show       # Show the installed plist
kiapi service start      # Start
kiapi service status     # Check status and the tail of logs
kiapi service stop       # Stop
kiapi service uninstall  # Remove

Access from other machines: kiapi binds to 127.0.0.1 by default. To reach it from other machines, expose it over your own private network layer. For example, with Tailscale on the kiapi machine:

tailscale serve --bg --https=8500 8500  # TLS endpoint reachable only inside your tailnet

Local Storage

kiapi mainly writes to these local paths at runtime.

Other model weights and library caches are managed by Hugging Face, mflux, Docker, or each library/tool. kiapi generally does not move them into its own storage location.

Security

By default, kiapi run starts on 127.0.0.1:8000. When --host 0.0.0.0 is specified, the server may be reachable from other machines, so use it only on trusted networks.

Development

make init     # Install dependencies, download test data, and create venv environments
make update   # Sync dependencies
make upgrade  # Upgrade dependencies

# ... implement

make       # Format, type-check, and regenerate dynamic documentation
make test  # Unit tests
make dev   # Start the development server with auto-reload

# GPU-backed functional and regression tests
make verify        # Choose the capability family interactively
make verify-fast   # Interactive choice, light tests only
make verify-kiapi  # Run every capability non-interactively

Release

Releases to PyPI are automated by GitHub Actions workflows.

Project Status