apple-fm-cli 0.2.0

CLI and OpenAI-compatible HTTP server for Apple Foundation Models (Apple Intelligence)

apple-fm-cli

A small command-line interface and local HTTP server that drive Apple’s on-device Foundation Models (Apple Intelligence) from Python. The repo bundles a ctypes-based apple_fm_sdk wrapper around the system model plus apple_fm_cli, which adds prompting, optional tools, JSON-schema-guided output, and an OpenAI-shaped API for clients like Codex.

Requirements

• macOS with Foundation Models available (the SDK checks availability and reports a reason if not).
• Python ≥ 3.14

Install

From PyPI (macOS, Python 3.14+):

pip install apple-fm-cli
# or: uv pip install apple-fm-cli

From a git checkout (editable):

pip install -e .
# or: uv pip install -e .

Entry point: apple-fm-cli.

The published wheel includes a prebuilt libFoundationModels.dylib (Apple silicon / arm64). To rebuild it after changing foundation-models-c, run swift build -c release in src/apple_fm_sdk/foundation-models-c and copy .build/*/release/libFoundationModels.dylib to src/apple_fm_sdk/lib/ before building distributions.

Publishing to PyPI (maintainers)

Automated (recommended): .github/workflows/publish.yml runs when you push a version tag v* (for example v0.1.3). It checks that the tag matches version in pyproject.toml, builds, uploads to PyPI, then creates a GitHub Release (with generated notes) if one does not already exist. One-time: configure PyPI trusted publishing for workflow publish.yml.

Typical release steps:

1. Bump version in pyproject.toml and run uv lock if you track uv.lock in git.

2. Rebuild and commit src/apple_fm_sdk/lib/libFoundationModels.dylib if the Swift bridge changed.

3. Commit and push to main, then tag and push the tag:

   git tag v0.1.3
   git push origin v0.1.3
   

Manual dispatch: Actions → Publish to PyPI → Run workflow still works for the current branch (no tag check); use sparingly.

Manual upload: uv run --with build python -m build then uv run --with twine twine upload dist/* using an API token. Prefer trusted publishing in CI over storing long-lived tokens in ~/.pypirc or GitHub secrets.

CLI

Query the model (plain text or structured JSON):

apple-fm-cli query "Summarize this idea in one sentence."
apple-fm-cli query --format json --schema '{"type":"object","properties":{"title":{"type":"string"}}}' "Name this topic."

Optional tools (comma-separated): bash (local shell), google_search (DuckDuckGo lite + page fetch).

apple-fm-cli query --tools bash,google_search "What’s in README.md in the cwd?"

Legacy-style flags are still accepted: -q / --query, --output, --output-schema.

Server

Starts a FastAPI app that mimics parts of the OpenAI Chat Completions and Responses APIs (including SSE for streaming), so tools that expect those endpoints can point at your machine instead of a cloud provider. Large agent system prompts are truncated heuristically to fit smaller local context windows.

apple-fm-cli server --host 0.0.0.0 --port 8000

• POST /v1/chat/completions
• POST /v1/responses

Codex

1. Start the server (see above), e.g. on port 8000.

2. Add a provider and profile to ~/.codex/config.toml:

   [model_providers.apple]
   name = "apple"
   base_url = "http://localhost:8000/v1"
   env_key = "OPENAI_API_KEY"
   
   [profiles.apple]
   model = "fm"
   model_provider = "apple"
   model_context_window = 4096
   

3. Run Codex with that profile:

   codex -p apple
   

env_key is the environment variable Codex uses for the bearer token. The local server does not need a real OpenAI key; set OPENAI_API_KEY to any non-empty placeholder if your Codex build requires it to be present.

Other agent harnesses

Anything that can target an OpenAI-compatible HTTP API (Chat Completions and/or Responses, including SSE) can point base_url at http://<host>:<port>/v1 and use a model id string of your choice—the server echoes the requested model name. Prefer a context window that matches what the on-device model can handle (4096 is a reasonable default for local sessions). If the client insists on an API key, keep using a dummy value in the configured env var unless you add auth in front of the server yourself.

Layout

Path	Role
src/apple_fm_sdk/	Session, tools, guided generation, tokenizer, native bridge bindings
src/apple_fm_cli/	query / server, built-in tools, schema → Generable helpers
notes/	Design notes (Responses SSE lifecycle, native bridge, e2e testing)

Licensing

apple_fm_sdk source files carry Apple Inc. copyright headers; refer to any accompanying license text distributed with that SDK.