apple-fm-cli 0.1.2

CLI and OpenAI-compatible HTTP server for Apple Foundation Models (Apple Intelligence)

apple-fm-cli

A small command-line interface and local HTTP server that drive Apple’s on-device Foundation Models (Apple Intelligence) from Python. The repo bundles a ctypes-based apple_fm_sdk wrapper around the system model plus apple_fm_cli, which adds prompting, optional tools, JSON-schema-guided output, and an OpenAI-shaped API for clients like Codex.

Requirements

• macOS with Foundation Models available (the SDK checks availability and reports a reason if not).
• Python ≥ 3.14

Install

From PyPI (macOS, Python 3.14+):

pip install apple-fm-cli
# or: uv pip install apple-fm-cli

From a git checkout (editable):

pip install -e .
# or: uv pip install -e .

Entry point: apple-fm-cli.

The published wheel includes a prebuilt libFoundationModels.dylib (Apple silicon / arm64). To rebuild it after changing foundation-models-c, run swift build -c release in src/apple_fm_sdk/foundation-models-c and copy .build/*/release/libFoundationModels.dylib to src/apple_fm_sdk/lib/ before building distributions.

Publishing to PyPI (maintainers)

CI (recommended): GitHub Actions workflow .github/workflows/publish.yml runs on published releases and on manual dispatch. One-time setup: in PyPI → Account → Publishing, add a trusted publisher for this repo with workflow file publish.yml (no GitHub Environment unless you configure both sides to use one). Then:

1. Bump the version in pyproject.toml and merge.
2. Rebuild and commit src/apple_fm_sdk/lib/libFoundationModels.dylib if the Swift bridge changed.
3. Create a GitHub Release from a tag (or use Actions → Publish to PyPI → Run workflow after a tag points at the release commit).

Manual upload: uv run --with build python -m build then uv run --with twine twine upload dist/* using an API token. Prefer trusted publishing in CI over storing long-lived tokens in ~/.pypirc or GitHub secrets.

CLI

Query the model (plain text or structured JSON):

apple-fm-cli query "Summarize this idea in one sentence."
apple-fm-cli query --format json --schema '{"type":"object","properties":{"title":{"type":"string"}}}' "Name this topic."

Optional tools (comma-separated): bash (local shell), google_search (DuckDuckGo lite + page fetch).

apple-fm-cli query --tools bash,google_search "What’s in README.md in the cwd?"

Legacy-style flags are still accepted: -q / --query, --output, --output-schema.

Server

Starts a FastAPI app that mimics parts of the OpenAI Chat Completions and Responses APIs (including SSE for streaming), so tools that expect those endpoints can point at your machine instead of a cloud provider. Large agent system prompts are truncated heuristically to fit smaller local context windows.

apple-fm-cli server --host 0.0.0.0 --port 8000

• POST /v1/chat/completions
• POST /v1/responses

Codex

1. Start the server (see above), e.g. on port 8000.

2. Add a provider and profile to ~/.codex/config.toml:

   [model_providers.apple]
   name = "apple"
   base_url = "http://localhost:8000/v1"
   env_key = "OPENAI_API_KEY"
   
   [profiles.apple]
   model = "fm"
   model_provider = "apple"
   model_context_window = 4096
   

3. Run Codex with that profile:

   codex -p apple
   

env_key is the environment variable Codex uses for the bearer token. The local server does not need a real OpenAI key; set OPENAI_API_KEY to any non-empty placeholder if your Codex build requires it to be present.

Other agent harnesses

Anything that can target an OpenAI-compatible HTTP API (Chat Completions and/or Responses, including SSE) can point base_url at http://<host>:<port>/v1 and use a model id string of your choice—the server echoes the requested model name. Prefer a context window that matches what the on-device model can handle (4096 is a reasonable default for local sessions). If the client insists on an API key, keep using a dummy value in the configured env var unless you add auth in front of the server yourself.

Layout

Path	Role
src/apple_fm_sdk/	Session, tools, guided generation, tokenizer, native bridge bindings
src/apple_fm_cli/	query / server, built-in tools, schema → Generable helpers
notes/	Design notes (Responses SSE lifecycle, native bridge, e2e testing)

Licensing

apple_fm_sdk source files carry Apple Inc. copyright headers; refer to any accompanying license text distributed with that SDK.