Tinkerbox for Apple Neural Engine

Run LiquidAI/d1-3B on Apple's Neural Engine through Core AI. FP16 backbone, 128-token chunks, 32,768-token context. Tokenization, embeddings and answer readout run on the CPU.

Requires Apple Silicon (M4 or newer), macOS 27, and Python 3.12 or 3.13. Earlier generations (M1–M3) are not supported due to Neural Engine hardware and offline compiler constraints.

Quickstart

Install the wheel from the GitHub release (Xcode is not required):

python -m pip install ./tinkerbox_ane-*.whl

from tinkerbox_ane import d1

with d1.load() as model:
    result = model.system_one(
        "I was charged twice this month.",
        {"refund": {"type": "noul", "instructions": "Is this a refund request?"}},
    )
    print(result)

load() downloads the matching Core AI architecture from wegotomars/d1-3B-CoreAI-ANE (v1.0.0) into ~/Library/Caches/tinkerbox-ane/d1/models/. Pass a directory path to load(path) to select a local AOT bundle.

The API exposes system_one, system_one_batch, probabilities, and probabilities_batch for binary, choice, and score queries. Requests run sequentially. Prompts and images exceeding 32,768 rendered tokens are rejected.

SwiftUI Demo

A desktop demo is available in examples/swiftui. To build and launch it from source:

uv run --no-project --python 3.12 python examples/swiftui/run.py

Pass --build-only to create artifacts/swiftui/Tinkerbox.app and the release ZIP without launching. The prebuilt app manages its own Python environment and dependencies via an embedded uv binary.

Architecture & Staging

The Swift host manages recurrent state and allocates attention cache buffers dynamically. Initial execution compiles and specializes the AOT asset for the local ANE hardware, after which Core AI reuses its persistent cache.

Weight staging uses an APFS copy-on-write workaround to eliminate redundant disk writes during specialization.

Building from Source

Building requires Xcode 27 / Swift 6.4 and uv:

uv sync --locked
uv build

The wheel in dist/ bundles the native host and links against macOS system frameworks. Attach it to the GitHub release. Source and editable installs compile the host during installation.

To export and compile model weights locally:

uv run python -m tinkerbox_ane.d1.export
uv run python -m tinkerbox_ane.d1.compile

Export writes intermediate artifacts to artifacts/build/d1_fp16_fused128_ctx32768, and compilation produces the final deployment bundle in artifacts/d1_fp16_fused128_ctx32768. Load the local result with d1.load(d1.DEFAULT_ASSET).

Key compiler options:

• --force: Rebuild an existing bundle. The compiler verifies the new artifact before replacing the previous build.
• --architecture <arch>: Target a specific architecture instead of the current machine.
• --allow-unsupported: Attempt compilation for an architecture outside the verified support list.

Compilation verifies source identity, function manifests, and full ANE placement. Without cached sudo credentials, worker cancellation on failure or timeout is best effort and emits a warning.

Reusable Core AI compilation and Hugging Face publishing utilities are provided in tinkerbox_ane.coreai and tinkerbox_ane.publish. See the model publishing guide for compiling and publishing additional models.

Publishing d1 Releases

Use the d1 publishing CLI to build and upload multi-architecture release bundles:

# Compile bundles for specific architectures without uploading
uv run python -m tinkerbox_ane.d1.publish --arch h17s h16g --no-upload

# Publish prepared bundles and create a release tag
uv run python -m tinkerbox_ane.d1.publish --arch h17s h16g --upload-only --tag v1.0.1

Omitting --arch targets all supported architectures. The upload synchronizes the remote macos27 tree with the selected architectures, pruning omitted targets and obsolete files while keeping existing release tags intact.

Common publishing options:

• --arch: Target architectures to build (defaults to all supported architectures).
• --tag: New release tag to create on Hugging Face (must not already exist).
• --source, --staging, --repo: Source export directory, local staging path, and Hugging Face repository.
• --metadata: Directory containing release metadata files (LICENSE, NOTICE, README.md).

When releasing a new version, update ARTIFACT_REVISION in src/tinkerbox_ane/d1/__init__.py so d1.load() resolves the new tag by default. (The legacy script scripts/publish_all_chips.py forwards to this CLI.)

Validation

uv run python scripts/build_reference.py
uv run python -m tinkerbox_ane.d1.guard artifacts/validation/check -- uv run python scripts/validate.py
uv run python -m tinkerbox_ane.d1.guard artifacts/validation/smoke -- uv run pytest -q

Reference generation downloads benchmark inputs and runs the official FP32 CPU model. Validation checks prompt tokens and probabilities across all nine cache capacities. The two public API smoke tests require a local bundle and cached checkpoint.

Weights and upstream code are pinned to LiquidAI/d1-3B@051bcc464b01b9f92942b364d9586b0ef5912432 under the LFM Open License v1.0. See NOTICE.