Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

Get Started with Microsoft Agent Framework Foundry Local

Requires Python 3.11-3.14 and Foundry Local SDK 2.3.1 or newer, below 3.0. Supported platforms are Windows (x64/ARM64), Linux (x86_64/aarch64), and macOS (Apple Silicon/arm64). Intel macOS is not supported; the standard Agent Framework install skips this integration on unsupported platforms. Install the integration:

pip install agent-framework-foundry-local --pre

and see the README for more information.

Usage

from agent_framework import Agent
from agent_framework.foundry import FoundryLocalClient

client = FoundryLocalClient(model="phi-4-mini")
agent = Agent(client=client, instructions="You are a helpful assistant.")
response = await agent.run("Why is the sky blue?")
print(response.text)

The client initializes the Foundry Local SDK singleton, prepares the selected model, and starts its built-in OpenAI-compatible HTTP service. Streaming, middleware, and function invocation continue to use Agent Framework's OpenAI transport.

All clients in a process share the SDK manager and web service. Concurrent client construction is serialized through model preparation and failed-setup cleanup. Deferred preparation is serialized with setup too; HTTP inference does not hold that lock. To customize cache directories or other SDK settings, pass configuration=Configuration(app_name="my-app", ...) when creating the first client, or initialize FoundryLocalManager yourself before creating clients. Register accelerated execution providers through the SDK manager before selecting GPU/NPU variants. A device value of "CPU", "GPU", or "NPU" selects a matching variant; an exact model ID is never replaced by a sibling variant.

bootstrap=False requires an already initialized SDK manager with a running web service in the same process. prepare_model=False defers native download/load until the first inference request for the configured model. The adapter prepares the exact selected variant on a worker thread before sending HTTP inference; an already loaded variant is reused. This preserves lazy first-call loading even though the HTTP service does not load models automatically. Applications can still download/load the variant manually beforehand to avoid first-request preparation latency. timeout applies to HTTP inference requests, not native preparation.

download_progress_callback receives download percentages from 0 to 100, and download_cancel_event accepts a threading.Event to cancel the download. Both apply to eager and deferred preparation. Deferred progress callbacks run on a worker thread. These controls do not cancel model loading or inference; asynchronous request cancellation uses the normal Agent Framework API and does not interrupt native preparation already running.

Clients do not close or unload the shared manager. After all clients finish, close their HTTP clients and call client.manager.close() once at application shutdown. If construction fails after creating a new manager, that manager is closed; an existing application's manager is not closed. If failed construction started its web service, that service is stopped; a previously running service is left running.

Bring Your Own Model

Use catalog_type="local" (or CatalogType.LOCAL) for models registered through the SDK's local catalog. Model lookup and HTTP inference both use that catalog: local clients target /catalogs/local/v1, while the default public clients use /v1. There is no fallback between catalogs.

Register compatible ONNX Runtime GenAI model assets through the SDK before constructing the client. The adapter does not import or convert model files. Registration does not transfer ownership of the files. For example:

from agent_framework import Agent
from agent_framework.foundry import FoundryLocalClient
from foundry_local_sdk import Configuration, FoundryLocalManager, ModelInfoBuilder
from foundry_local_sdk.catalog import CatalogType

FoundryLocalManager.initialize(Configuration(app_name="my-local-agent"))
manager = FoundryLocalManager.instance
if manager is None:
    raise RuntimeError("Foundry Local manager was not initialized.")
local_catalog = manager.get_catalog(CatalogType.LOCAL)
with ModelInfoBuilder() as metadata:
    metadata.set_string_property("task", "chat-completion")
    registered_model = local_catalog.register_model(
        "your-model-directory", "my-local-model:1", metadata
    )

client = None
try:
    client = FoundryLocalClient(model=registered_model.id, catalog_type="local")
    agent = Agent(client=client, instructions="You are a helpful assistant.")
    response = await agent.run("What is 2 + 2?")
    print(response.text)
finally:
    if client is not None:
        await client.client.close()
    registered_model.unload()
    local_catalog.unregister_model(registered_model.id)
    manager.close()

The normal preparation, exact variant selection, streaming, middleware, tools, and structured-output APIs also apply to local models, subject to model/runtime capabilities. prepare_model=False also supports lazy preparation of registered local models. Clients do not unregister local models or delete their assets; the application owns registration and manager shutdown.

Chat Options

FoundryLocalChatOptions includes the standard generation settings and native top_k control. The client encodes top_k as the HTTP service's string-valued decoder metadata, preserving zero. The explicit control takes precedence over conflicting values in metadata or extra_body["metadata"]. The HTTP service also accepts random_seed through these dictionaries. do_sample and early_stopping are not supported by the HTTP service and raise ValueError, including when supplied through metadata or extra_body.

Pydantic response models and JSON schemas use the native HTTP service's bare-schema format automatically. JSON-object mode constrains output to an object; it does not guarantee particular fields or values. Supported settings and structured output capabilities depend on the model and runtime. Backend errors remain visible as ChatClientException.

from agent_framework import Message
from agent_framework.foundry import FoundryLocalChatOptions, FoundryLocalClient
from pydantic import BaseModel


class Answer(BaseModel):
    answer: int


client = FoundryLocalClient(model="phi-4-mini")
options: FoundryLocalChatOptions[Answer] = {
    "response_format": Answer,
    "max_tokens": 128,
    "temperature": 0.0,
    "top_k": 1,
}
response = await client.get_response(
    [Message(role="user", contents=["What is 2 + 2?"])],
    options=options,
)
assert isinstance(response.value, Answer)
print(response.value.answer)

Agent sessions, middleware, streaming, and automatic tool execution retain their standard Agent Framework APIs. A per-request model option overrides inference targeting only: that variant must also be loaded in the client's selected catalog. An override does not switch between the public and local catalogs. When a tool is required (including a named tool), the request uses the native tool grammar first, then applies the requested structured format to the final reply after automatic tool execution. With automatic execution disabled, the caller must handle the returned function call before requesting a structured final reply. Structured output with available tools and omitted or "auto" tool choice raises ValueError: the SDK's output schema would override automatic tool selection. Use a required or named tool choice, or perform tool selection/execution first and request the structured final response separately without tools.

Scope

This adapter targets the SDK's public and local model catalogs and HTTP chat service. Local models must be registered through the SDK before use. Native-session-only features, audio transcription, and embeddings are outside this chat integration.

Image content follows the OpenAI transport's serialization, but a successful HTTP request or the presence of vision assets does not establish that a particular model/runtime consumes those images. Verify multimodal behavior for the selected configuration before relying on it.

Migration from SDK 0.5.1

This is a breaking SDK migration: Python 3.10 and SDK 0.5.1/1.x are no longer supported. Update SDK imports from foundry_local to foundry_local_sdk. Use client.manager.catalog.list_models() instead of list_catalog_models(), inspect metadata through model.info, and call download()/load() on catalog model objects instead of download_model()/load_model() on the manager. The SDK now runs in-process; a separately started CLI service does not satisfy bootstrap=False.

Foundry Local Sample

See the Foundry Local provider sample for a runnable example.

Metadata

Release files for agent-framework-foundry-local 1.0.0b261106

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for agent-framework-foundry-local 1.0.0b261106
File Size Uploaded Provenance Actions
agent_framework_foundry_local-1.0.0b261106.tar.gz 15.2 kB Without provenance Details for agent_framework_foundry_local-1.0.0b261106.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for agent-framework-foundry-local 1.0.0b261106
File Interpreter ABI Platform Actions
agent_framework_foundry_local-1.0.0b261106-py3-none-any.whl Python 3 none any Details for agent_framework_foundry_local-1.0.0b261106-py3-none-any.whl

Total release size: 28.9 kB

Release files / agent_framework_foundry_local-1.0.0b261106.tar.gz

Download URL agent_framework_foundry_local-1.0.0b261106.tar.gz
Size 15.2 kB
Tags Source
SHA-256 checksum
How to use checksums
f33e91c5a32a68466d1325e43ed4791baff1f8d35f66ac921e7188638c57ffd9
BLAKE2b-256 checksum
How to use checksums
b9d41c8be1bc31983a40564f0d482d8e02ef5c99bf0a3d6fe1ff7d048c268fd6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via RestSharp/106.13.0.0

Release files / agent_framework_foundry_local-1.0.0b261106-py3-none-any.whl

Download URL agent_framework_foundry_local-1.0.0b261106-py3-none-any.whl
Size 13.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
783fbaa3deb4a2fd4db85cd8b64c749e119d6daab085079faead91f3b50add56
BLAKE2b-256 checksum
How to use checksums
0692ad314158167e6d5ea6ddcf2060b27104130b575fd8dac3c410b949c8f6f6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via RestSharp/106.13.0.0

Security for agent-framework-foundry-local 1.0.0b261106

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release
Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page