Skip to main content

modmex-ai

Lightweight, provider-neutral agents, flows, tools, and voice runtimes for Python.

CI Coverage PyPI Python Versions License

modmex-ai is a small runtime for applications that need an agent to reason, call tools, hand off work, and return a controlled result. It fits equally well in HTTP APIs, CLIs, background workers, WebSocket servers, notebooks, desktop applications, voice systems, and event-driven/serverless workloads.

Its core has no provider SDK dependency. OpenAI and Anthropic (Claude) text adapters ship in the box, both on the built-in zero-dependency HTTP transport; the core contracts remain reusable for another text, speech, or realtime provider.

The basic execution model is intentionally simple:

input -> Flow -> one or more Agents -> output, tools, handoffs, and events

What it provides

  • Agent for instructions, structured output, tools, guardrails, and streaming.
  • Flow for multi-agent routing, handoffs, continuation, and sync/async execution.
  • Sessions, snapshots, optimistic concurrency contracts, and approval checkpoints.
  • Realtime and chained voice contracts that keep STT, reasoning, and TTS composable.
  • Provider-neutral tracing, usage, evals, and host-owned observability.

Install the core package in any Python application:

pip install modmex-ai

Add only the optional transport needed by a deployment:

pip install 'modmex-ai[async]'      # native async HTTP
pip install 'modmex-ai[realtime]'  # WebSocket-based Realtime adapters

Quick start

from modmex import BaseModel
from modmex_ai import Agent, FakeModel, Flow


class TriageOutput(BaseModel):
    intent: str
    confidence: float


triage = Agent(
    name="triage",
    instructions="Classify the incoming message.",
    output_type=TriageOutput,
    handoffs=["support"],
)

flow = Flow(
    name="analyze-message",
    entrypoint=triage,
    agents=[],
    model=FakeModel(['{"intent":"support","confidence":0.91}']),
)

result = flow.run("I need help")
result.output.model_dump()

The same Flow can be called by a REST handler, a queue consumer, a CLI command, or a long-lived WebSocket connection. Event-driven systems are a natural fit, but they are not required.

Multi-agent delegation uses handoff function tools, not a field in the model output. A handoff defaults to transfer_to_agent_name; its typed input is validated and delivered to an on_handoff callback before the next agent continues with the conversation.

Anthropic (Claude)

AnthropicMessagesModel calls POST /v1/messages through the same zero-dependency transport as the OpenAI adapters, so it adds no packages to a Lambda bundle. api_key defaults to the ANTHROPIC_API_KEY environment variable.

from modmex_ai import Agent, AnthropicSettings, ModelSettings, tool
from modmex_ai.providers.anthropic import AnthropicMessagesModel


@tool
def get_weather(city: str) -> dict:
    """Get the current weather for a city."""
    return {"city": city, "temp_c": 21}


agent = Agent(
    name="weather",
    instructions="Answer weather questions using the tool.",
    tools=[get_weather],
    model=AnthropicMessagesModel("claude-opus-5-5"),
    settings=ModelSettings(
        reasoning_effort="high",   # output_config.effort: low|medium|high|xhigh|max
        anthropic=AnthropicSettings(
            cache_ttl="5m",        # automatic prompt caching ("5m" or "1h")
            fallbacks="default",   # server-side fallback if the model declines
        ),
    ),
)
result = agent.run("What's the weather in Paris?")

How the provider-neutral contracts map to the Messages API:

modmex-ai Anthropic
Leading system/developer messages top-level system
Later system/developer messages mid-conversation role: "system" message where the API accepts one, otherwise user text
function_call / function_call_output items tool_use / tool_result blocks (parallel results grouped in one user turn)
output_type / output_schema output_config.format (JSON schema)
ModelSettings.reasoning_effort output_config.effort
ModelSettings.max_tokens max_tokens (defaults: 16000, or 64000 when streaming)
ModelSettings.parallel_tool_calls=False tool_choice: {type: "auto", disable_parallel_tool_use: true}
Usage.input_tokens / cached_input_tokens uncached + cache read + cache write / cache read
stop_reason: "refusal" raises ModelRefusalError (so a FallbackModel moves on)

Notes:

  • Thinking blocks are replayed verbatim. Current Claude models always think, and a tool-use turn must be sent back exactly as it was produced. The adapter stores that turn's content in ToolCall.provider_data; the runner copies it to the function_call SessionItem, and it survives session snapshots and approval checkpoints. Each replayed turn also records a fingerprint of the system prompt and tool set it was produced with; if a tool_selector changes the tools or a handoff moves the history to an agent with other instructions, the now-invalid thinking blocks are dropped automatically (as a leading run, which the API accepts) while text and tool calls stay. With stable tools and instructions nothing is dropped. Don't edit or remove items from the middle of a session's history yourself.
  • temperature and top_p raise ValueError: API version 2026-10-21 rejects sampling parameters on every model.
  • The Messages API is stateless: ProviderState with a remote previous_response_id/conversation_id is rejected.
  • The default transport retries 529 overloaded on top of the usual status codes and uses a 300 s timeout, which is still capped by the remaining execution budget.
  • Streaming (run_stream) is recommended for long or tool-heavy turns.

Multimodal inputs

Messages can combine typed text, document, and image parts without exposing a provider payload in application code. FileInput and ImageInput accept a URL, a provider file id, or inline bytes/Base64 as appropriate. The provider adapter decides which source it supports.

The OpenAI Responses adapter maps these parts to the official input_text, input_file, and input_image request items. For documents stored privately, prefer a short-lived signed URL and create it only for the model request; do not persist it in conversation state or publish it in events.

from modmex_ai import FileInput, InputDetail, Message, TextInput
from modmex_ai.models import ModelRequest
from modmex_ai.providers.openai import OpenAIResponsesModel

request = ModelRequest(
    messages=[
        Message(
            role="user",
            content=[
                TextInput(text="Extract the organization name and expiration date."),
                FileInput(
                    url=presigned_file_url,
                    filename="agreement.pdf",
                    media_type="application/pdf",
                    detail=InputDetail.HIGH,
                ),
            ],
        )
    ]
)

response = OpenAIResponsesModel("gpt-5.6", api_key="...").complete(request)
print(response.output_text)

OpenAIChatModel deliberately rejects FileInput and ImageInput until that adapter implements their distinct wire format. Use OpenAIResponsesModel for multimodal input.

Per-turn context

Context that is true now — the customer's open quotes, the current load status, today's date — should not be part of the stored conversation: stored once per turn, it accumulates, and the model sees several conflicting versions (and may call tools again to find out which is current). Pass it as turn_context:

result = flow.run(
    whatsapp_message,
    session=session,
    turn_context=f"Cliente: {customer}. Cotizaciones abiertas: {open_quotes}.",
)
  • It is sent with this turn only, after the turn's input, and never written to the session: the model always sees exactly one, current copy.
  • The agent's instructions and the stored history stay byte-identical between turns, so prompt caching keeps working (it is not merged into OpenAI instructions; with Claude it is a native mid-conversation system message).
  • A string is sent as a developer message; a list of messages is sent as given.
  • Handoff targets in the same turn receive it too.
  • It is not stored in a suspended (approval) state; pass the current context again to Flow.resume(..., turn_context=...).
  • It requires stateless mode: with server-side conversation state (previous_response_id / conversation_id) the provider would store it, so it raises ValueError.

Only leading system/developer messages go to OpenAI Responses instructions; later ones are sent in place, so mid-conversation instructions (including output-guardrail retries) no longer rewrite the cached prefix.

Durable sessions and approvals

The core provides portable SessionSnapshot and PersistedFlowState models, plus optimistic-concurrency store contracts. A host chooses the database and owns the lifecycle; this keeps the framework usable in a web application, a worker, or serverless infrastructure without importing a storage SDK into the core.

Tools can require an externally verified approval. When a protected tool is requested, a Flow raises FlowSuspended with the exact pending tool call and a serializable checkpoint. Persist it, collect the signed approval in the host, then resume the Flow without asking the model to decide again.

For Lambda-oriented hosts, install the optional connector integration and use the DynamoDB stores from modmex-ai:

pip install 'modmex-ai[lambda]'
from modmex_ai.persistence import DynamoDbDurableSessionStore
from modmex_lambda.connectors.dynamodb import Connector

store = DynamoDbDurableSessionStore(Connector("sessions-table"))

Other applications can implement the same DurableSessionStore and FlowStateStore contracts with Postgres, Redis, or their existing storage.

Realtime voice sessions

VoiceAgentSession is the common contract for voice sessions. There are two implementations with distinct responsibilities:

  • OpenAIRealtimeSession para speech-to-speech por WebSocket/SIP.
  • ChainedVoiceSession para conservar el control explícito de STT → Agent/Flow → TTS.

Ambas exponen eventos normalizados, uso acumulado, herramientas y handoffs. El adaptador de OpenAI conserva el payload específico del proveedor únicamente en su frontera; el core usa tool_call_id para no confundir llamadas de tools con identificadores de llamadas telefónicas.

Install the optional WebSocket transport only in workloads that need it:

pip install 'modmex-ai[realtime]'

The OpenAI adapter reuses Agent tools, handoffs, tool guardrails, context, usage, and traces. It does not depend on the OpenAI Agents SDK. For a SIP call that was already accepted by your server, connect the worker using its realtime_call_id:

import asyncio

from modmex_ai import Agent
from modmex_ai.providers.openai import (
    OpenAIRealtimeClient,
    OpenAIRealtimeSessionConfig,
)


async def run_voice_call(realtime_call_id: str) -> None:
    agent = Agent(
        name="support",
        instructions="Help the caller concisely and use tools when needed.",
    )
    client = OpenAIRealtimeClient()
    session = await client.connect(
        agent=agent,
        realtime_call_id=realtime_call_id,
        config=OpenAIRealtimeSessionConfig(voice="marin"),
    )
    async with session:
        await session.configure()
        await session.create_response(
            instructions="Greet the caller and ask how you can help."
        )
        async for event in session.events():
            if event.type == "response.done":
                print(session.usage.model_dump())


asyncio.run(run_voice_call("rtc_u1_example"))

When the model emits a function call, the session validates and executes the registered Modmex-AI tool, sends its output to the socket, and asks Realtime to continue. A transfer_to_* handoff updates the active agent's instructions and tool surface in the same Realtime session.

For a SIP accept endpoint, build the accept body from the exact same session configuration used after the WebSocket connects. That prevents model, voice, instructions and tool definitions from drifting between both phases:

accept_payload = OpenAIRealtimeSessionConfig(voice="marin").to_accept_payload(agent)

Chained voice pipeline

Use a chained session when the application must choose its transcription and speech providers, or needs text-agent behavior between audio turns. A Flow is preferred when the conversation can hand off to another agent.

session = ChainedVoiceSession(
    flow=conversation_flow,
    speech_to_text=transcriber,
    text_to_speech=synthesizer,
    context={"workspace_id": "workspace-1"},
)

audio_reply = await session.process_audio(caller_audio)
async for event in session.voice_events():
    # transcript_final, tool_finished, handoff_completed,
    # response_completed, session_ended, or error
    handle_voice_event(event)

The STT/TTS providers are small async protocols, so concrete provider SDKs stay outside the core package. Close either session with a VoiceTerminationReason to make lifecycle reporting explicit.

For bounded audio turns, the OpenAI adapters use the Audio API directly through the lightweight HTTP client—no provider SDK is installed:

from modmex_ai.providers.openai import (
    OpenAISpeechProvider,
    OpenAITranscriptionProvider,
)

session = ChainedVoiceSession(
    flow=conversation_flow,
    speech_to_text=OpenAITranscriptionProvider(model="gpt-4o-mini-transcribe"),
    text_to_speech=OpenAISpeechProvider(voice="marin", response_format="wav"),
)

To continue a chained conversation in another process, persist the host's Session implementation and session.continuation. The continuation tracks the active agent and provider state; the host owns durable history storage. For a small self-contained host, SessionSnapshot.from_session(session) can serialize an InMemorySession and later restore it with snapshot.to_memory_session().

ChainedVoiceSession.process_audio_stream() supports providers that implement streaming STT and TTS contracts. It emits transcript deltas and waits for a final transcript before invoking the text flow.

When every agent's model streams natively (astream, as the OpenAI and Anthropic adapters do), the reply is spoken while it is generated: model text is split into sentences and each one is synthesized as soon as it is complete, while the model keeps writing the next. The caller hears the first sentence instead of waiting for the whole reply, which matters most with thinking models. Text a model writes before calling a tool ("Let me check your load.") is spoken while the tool runs. Agents with structured output (output_type) or output guardrails are not spoken early, since their text is not speakable or could still be rejected; their final reply is synthesized when the turn completes, as before. Turn off with VoiceTurnOptions(speak_while_generating=False). VoiceTurnMetrics.first_audio_latency_ms reports time to the first audio chunk. Each sentence is a separate TTS request, so use a raw, concatenable audio format for playback (e.g. OpenAISpeechProvider(response_format="pcm")); container formats such as WAV or MP3 repeat their header per sentence. To use a request-based STT such as OpenAITranscriptionProvider here, wrap it in BufferedStreamingSpeechToText. A turn cancelled mid-reply is not saved to the session. Write tools used in voice turns as async def (or offload blocking I/O to a thread): a synchronous, blocking tool stalls the event loop, so the sentence spoken before it cannot play meanwhile.

Claude has no speech-to-speech, STT or TTS API; use it as the reasoning step of a chained session with any STT/TTS provider. For voice, keep reasoning_effort low and consider the instruction "Latency-sensitive; begin your visible answer immediately." so the model starts speaking sooner.

The callable adapters make it possible to plug in a provider-specific function without making that provider a dependency of modmex-ai.

Native async models

Models can optionally expose async def acomplete(request). Agent.run_async and Flow.run_async use it directly, including async tools; existing sync models retain their compatible worker-thread fallback. The built-in OpenAI clients expose acomplete() as well. Install the optional native transport to make them select httpx automatically:

pip install 'modmex-ai[async]'

Without that extra, the compatible stdlib transport runs in a worker and keeps the event loop responsive.

Realtime transcription

For live microphone or telephony media, use the transcription-only WebSocket adapter instead of a speech-to-speech agent session. It emits Transcription deltas as audio arrives and one final value after manual commit:

from modmex_ai.providers.openai import OpenAIRealtimeTranscriptionProvider

transcriber = OpenAIRealtimeTranscriptionProvider()
async for transcript in transcriber.transcribe_stream(pcm_chunks, context=context):
    render(transcript.text, final=transcript.is_final)

The adapter uses 24 kHz PCM by default and requires modmex-ai[realtime]. For completed audio turns, OpenAITranscriptionProvider remains the simpler request-based alternative.

For a long-lived call, open one reusable session and commit each user turn:

session = await transcriber.connect()
await session.append_audio(pcm_chunk)
await session.commit_turn()
async for transcript in session.events():
    correlate(transcript.item_id, transcript.text, transcript.is_final)

Turn cancellation and provider retries

ChainedVoiceSession accepts VoiceTurnOptions(max_provider_retries=...). Retries apply only to STT/TTS provider I/O, never to the text flow, tools, or handoffs. Call cancel_current_turn() to cooperatively stop a streaming turn before its next provider chunk; hosts can still cancel the owning task when an immediate interruption of a blocked transport is required.

After every completed, cancelled, timed-out, or failed bounded turn, ChainedVoiceSession.last_turn contains a VoiceTurnOutcome. Its metrics separate STT, agent, TTS, total latency, and provider retry count. Pass a VoiceTurnObserver to export those outcomes to the host's own logs or metrics backend without adding an observability dependency to this package.

Model Context Protocol (MCP)

MCPClient is the low-level protocol client. MCPClientConnection adds discovery, tool adaptation, caching and lifecycle management so an Agent can consume one or more remote MCP servers through mcp_clients:

from modmex_ai import Agent, MCPClientConnection, MCPClientManager

async with MCPClientManager([
    MCPClientConnection("https://example.com/mcp"),
]) as mcp_clients:
    agent = Agent(
        name="assistant",
        instructions="Use the available remote tools when appropriate.",
        mcp_clients=mcp_clients,
    )
    result = await agent.run_async("Help me with my request")

Use client.list_tools() directly only when the application needs to manage the remote tool catalog itself. Resources and prompts remain explicit:

connection = mcp_clients[0].client
menu = await connection.read_resource("pizza://menu")
prompt = await connection.get_prompt("assistant")

The client supports MCP 2026-07-28, pagination, structured tool results, modern metadata/headers and buffered SSE responses. It does not create transport sessions. The MCP server transport and Lambda/API Gateway integration are provided by modmex-lambda.

Concurrent MCP discovery and tool calls

Async agent runs and MCPClientManager connect to MCP servers concurrently. Each connection caches its tool catalog. Discovery and cursor pagination within one server stay ordered.

Set Agent(..., parallel_tool_calls=True) to execute the tool calls returned in one model response concurrently in run_async and arun_stream. The default is False. Enable this for independent async tools, including MCP calls; synchronous functions still run on the event loop. Tools share the run context, so avoid parallel calls that depend on each other's results or mutate the same state. Streaming progress and finish events arrive as tools produce them; results sent back to the model retain the original call order. A failed or cancelled batch cancels and awaits its remaining tasks. Synchronous runs remain sequential.

Execution resilience and latency (0.5)

Existing agents retain fail-fast tool handling unless explicitly configured:

from modmex_ai import Agent, ModelSettings, RunTimeoutError, execution_budget

agent = Agent(
    name="driver",
    instructions="Use tools to perform operations; never claim unconfirmed writes succeeded.",
    model=model,
    tools=tools,
    tool_error_policy="return_error",
    execution_timeout=25,
    settings=ModelSettings(reasoning_effort="low", max_tokens=500),
)
try:
    result = await agent.run_async("Report arrival")
except RunTimeoutError:
    # The application persists pending work and sends its own deterministic fallback.
    # A timed-out write may already have completed: verify before retrying.
    ...

return_error converts ToolExecutionError/ToolValidationError into sanitized function outputs with ok: false, an error code, and retryable: false. The model can then respond without exposing internal exception text. There are no automatic tool retries. Approvals, remote MCP requests for human input, guardrail blocks, cancellation, unknown tools and execution deadlines still propagate. With recovery enabled, remote MCP tools use their buffered call path even in agent streaming; model streaming remains available, but MCP progress is not emitted.

Execution deadlines are optional and shared with nested agents and HTTP retries. Wrap a whole Flow in with execution_budget(seconds): to share a single deadline across handoffs. Async execution cancels cooperative operations when the budget expires. Synchronous execution checks deadlines between operations and bounds HTTP timeouts; it cannot forcibly interrupt arbitrary blocking Python functions or undo writes. The same limitation applies to blocking synchronous functions called inside async code and synchronous model fallbacks running in worker threads. Closing a stream early is still the caller's responsibility. Do not hold a stream open while running unrelated work in the same task: its execution budget remains active until it is closed.

Trace events include model response duration, first streamed event latency, message count, UTF-8 context/schema sizes, and tool duration/outcome. Byte counts are not token estimates. New metrics do not include prompt text, arguments or exception messages; existing application tracing policies still apply. Non-streaming model failures produce model_error with exception type and elapsed time. Attach existing Trace observers to export these metrics; supply RetryConfig(on_retry=...) for sanitized retry events.

HTTP retries honor Retry-After (seconds or HTTP date), skip exhausted billing quota, respect the active execution budget and support optional jitter_seconds. An explicitly empty status_codes set disables retries. Streams are never replayed after output starts. Transport/network timeouts retain their existing fail-fast policy. No HTTP connection pooling was replaced: AsyncHttpClient already owns/reuses its transport explicitly.

reasoning_effort is omitted by default and maps to Responses reasoning.effort or Chat Completions reasoning_effort. Model support and accepted effort levels vary; choose compatible settings explicitly. The library validates known effort names but does not maintain a model capability catalog or silently remove sampling parameters. See https://developers.openai.com/api/docs/guides/reasoning.

These mechanisms do not send user messages, escalate business cases, change an agent's permissions, or compact business context automatically. Those remain application policies.

Incremental context compaction

Compaction is opt-in and provider-neutral. It is independent of Agent/Flow execution, so hosts can compact the exact business context they assemble, without changing agents.

from modmex_ai import ContextCompactor, CompactionSnapshot, execution_budget

compactor = ContextCompactor(
    model=summary_model,  # Host chooses the model and credentials.
    threshold=60_000,     # Estimated tokens (default measure: TokenEstimator).
    keep_recent_turns=4,
    max_input_size=80_000,  # Tokens sent to the summarizer per batch.
    max_summary_bytes=16_000,
    preservation_instructions="Preserve load IDs, assignments and unresolved requests.",
)
with execution_budget(25):  # Shared budget for compaction AND the ensuing agent run.
    result = await compactor.compact_async(full_history, previous=saved_snapshot, trace=trace)
    # Persist result.snapshot using your repository and optimistic concurrency control.
    # Supply result.messages as agent input; do not add the previous summary again.
    # If still over your allowed budget, apply an explicit host policy before executing.
    response = await agent.run_async(result.messages)

Measuring in tokens without a tokenizer

Sizes are tokens estimated by a TokenEstimator (the default measure). It needs no tokenizer dependency and makes no extra API calls: it starts at ~4 bytes per token and calibrates itself from the input tokens your provider actually billed for each request in a run's trace.

result = compactor.compact(full_history, previous=saved_snapshot)
response = agent.run(result.messages)
compactor.measure.observe_trace(response.trace)   # learn the real bytes/token ratio
saved_ratio = compactor.measure.bytes_per_token   # persist across cold starts and pass
# ContextCompactor(..., measure=TokenEstimator(bytes_per_token=saved_ratio)) next time

Samples that cannot be a real per-request ratio (for example server-side conversation state, where only new messages are sent but the whole conversation is billed) are ignored. The ratio is fixed during a compact call, so the measure stays deterministic. Pass measure=context_bytes (and byte thresholds) to budget in serialized bytes, or any deterministic, monotonic counter of your own.

Pass the full, append-only history every time, together with the last successful CompactionSnapshot (serializable with model_dump()). The snapshot stores summary, covered item count and a digest of the covered prefix. Changed/truncated history or a snapshot from another history is rejected; the host must reset/rebuild it explicitly. Already covered messages are replaced by the existing summary before checking the threshold, and only new complete turns enter the next incremental summary request. Persist the snapshot only for the matching history revision; concurrent turns require host-side synchronization. The original history is never mutated or deleted.

System/developer messages are preserved verbatim, including those inside the covered prefix. Summaries are injected as user-role historical data, never elevated to privileged instructions. Recent user turns are kept verbatim; pending calls and parallel tool-call blocks are never split. Supply tool history as SessionItem function_call/output (or handoff_call/output) pairs with unique call IDs. Orphan/duplicate outputs are rejected; legacy Message(role="tool") records require explicit conversion by the host.

A single bounded batch is compacted per call. Large turns are not truncated to fit the summarizer. Results report unchanged, compacted, blocked or failed, sizes, duration, usage and a safe reason. A compacted result can still exceed the threshold: the host must check after_size and choose whether to compact again, defer or reject execution. max_summary_bytes is always a byte cap, whatever the measure. Empty, oversized, or non-reducing summaries never replace the old snapshot. Provider failures retain the previous effective context and unsummarized messages; cancellation and RunTimeoutError propagate. Usage from completed summarizer responses is reported even if the summary is rejected. Failed provider calls may incur usage unavailable to the library. Summarization is lossy: critical authorization and transaction state must always be reloaded from authoritative application data.

Provider continuation data (SessionItem.provider_data, e.g. Anthropic's signed thinking blocks) is bound to the original conversation prefix. Once a summary replaces older turns, the compacted view drops it from every completed retained turn (an unfinished tool round keeps it), and it is never sent to the summarizer. A summary that stops at max_tokens is rejected as summary_truncated instead of silently losing history. Thinking models (Claude, OpenAI reasoning models) spend part of max_tokens on reasoning; the default summarizer max_tokens is 8000. For a cheaper summary, add reasoning_effort="low" where the summarizer model supports it.

The compactor does not change provider_state or server-managed conversations. Hosts must rebuild/clear incompatible provider state when replacing raw history with a summary; otherwise old history can be sent twice. Compaction does not enable automatic persistence, age filtering, permissions or TMS-specific retention rules. Existing SessionSnapshot.compact remains a separate legacy item-trimming utility and is not automatically replaced.

Responses state contract and audit corrections

Choose one state strategy, and persist the returned ProviderState with your host's conversation revision (optimistic concurrency is still the host's responsibility):

from modmex_ai import ProviderState, ModelSettings, OpenAISettings

# Explicit manual history; returned response IDs do not switch this run into chaining.
manual = ProviderState(provider="openai", mode="stateless")
# Start a response chain, or resume it with previous_response_id="resp_...".
chain = ProviderState(provider="openai", mode="response_chain")
# Use an existing OpenAI conversation; never add previous_response_id here.
conversation = ProviderState(provider="openai", mode="conversation", conversation_id="conv_...")

Legacy single-ID states infer their mode. States containing both IDs now fail before request execution, including previously persisted invalid states: migrate them by choosing one strategy and clearing the other identifier. Responses containing both a response ID and conversation ID yield a conversation-mode state with only conversation_id. When no explicit mode is given, OpenAI response IDs retain the previous automatic chaining behavior. A chain requires the adapter to return continuation state; missing state is rejected.

With remote state, provide ONLY the new turn as Agent input. The runner tracks pending items per inference; it does not guess newness by filtering function_call_output. Tool outputs from earlier iterations are not resent. Instructions and guardrail corrections are reissued, while rejected assistant responses already in the remote chain are not appended again. Failed transport retries reuse the same pending request; the cursor only advances after receiving a response. This is not an exactly-once provider request guarantee: a timeout can leave remote state ambiguous and requires host-side reconciliation.

Flow may mirror remote conversations in a local Session for auditing, but it never sends that local history alongside the remote state. Passing full history directly to Agent with a remote identifier still duplicates it: the caller owns the first-turn input contract. Remote handoffs send only their pending acknowledgement plus the destination instructions. Local handoff history filters are incompatible with retained server history and are rejected; use stateless execution if a filter must actually remove context. Compaction likewise requires resetting/rebuilding incompatible remote state before using summarized history.

Mixed handoff/tool batches (and multiple handoffs) are rejected before any callback or tool runs. Approval-required calls must be the sole call in their batch. This explicit policy avoids partial side effects and ambiguous parallel approval checkpoints. Flow approvals, including streamed runs, retain the latest remote response ID and can be serialized and resumed with only the pending output. Hosts still must persist approval/execution receipts and prevent double resume; modmex-ai does not guarantee exactly-once external writes.

Tool availability and concurrency

Agent(tool_selector=...) optionally accepts a synchronous callback (run_context, names) returning available tool/handoff names before each inference. Only registered names are accepted, original schema order stays stable, and calls to unavailable tools are rejected before execution. No selector means all registered tools remain available. Availability is also checked before an approved tool is resumed; backend authorization remains mandatory. Use conservative selection, since changing the schema prefix can reduce cache reuse. Tool schemas are built once per execution, not fetched/rebuilt on every inference.

Agent.parallel_tool_calls controls local async concurrency (default False). ModelSettings.parallel_tool_calls independently controls whether the model may propose parallel calls (omitted by default). Neither flag proves tool independence; only enable local parallelism when the host knows those operations can safely overlap.

Cache controls and measurements

settings = ModelSettings(
    parallel_tool_calls=False,
    openai=OpenAISettings(
        prompt_cache_key="workspace-profile-prompt-v1",
        prompt_cache_retention="in_memory",  # or "24h" on compatible models
    ),
)

Cache options map explicitly to Responses and Chat Completions and are omitted by default. They group cache routing; they do not guarantee cache hits, lower billing, or lower latency. Hosts choose stable non-sensitive keys and stable instruction/tool prefixes, with tenant isolation as appropriate. Existing Usage.cached_input_tokens measures actual cached usage. In streamed runs, model_first_fragment measures the first normalized model stream event and model_first_token measures the first nonempty text delta separately. Buffered/streamed provider errors are traced without internal error text. Request sizes describe the input actually sent by the runner, not the full local mirrored history.

Contract references: https://developers.openai.com/api/docs/guides/conversation-state and https://developers.openai.com/api/docs/guides/prompt-caching. The compatibility tests exercise the real OpenAI adapter with simulated HTTP/SSE transports; they do not call live OpenAI or establish production latency improvements.

Native OpenAI server-side compaction (Responses only)

settings = ModelSettings(openai=OpenAISettings(compact_threshold=12_000))
state = ProviderState(provider="openai", mode="response_chain")
agent = Agent(name="carrier", instructions=instructions, tools=tools,
              model=responses_model, settings=settings)
result = await agent.run_async(new_input, provider_state=state)
# responses_model must support native compaction.
# Persist result.provider_state; next time pass only the new input with that state.

compact_threshold is a positive token count, not serialized bytes. It is omitted by default. Responses sends context_management=[{"type":"compaction","compact_threshold":N}]. OpenAI decides when the rendered context crosses the configured threshold. Model support and permitted thresholds are validated by the provider; the library does not assume every model supports the feature. Choose a threshold with room below the model's context limit.

This integration intentionally requires response_chain mode, including for the first call without a previous_response_id. Chat Completions, stateless/manual history and conversation mode are rejected when native compaction is enabled. This restriction prevents silently losing opaque compaction output with the current generic history types. For manual history, use ContextCompactor instead. Do not combine local summary replacement with a retained remote chain, and do not prune the server-managed history yourself.

When a completed response contains compaction output items, ModelResponse.compaction_count reports their count and Agent emits provider_compaction with count, request ID and response usage. It does not log or interpret encrypted_content. The existing raw provider response retains opaque items for low-level callers; application logging of raw responses remains its responsibility. Counts reflect items the provider reports, not an inferred trigger or compaction on every turn. Usage is provider-reported for the response; separate compaction cost/duration is not invented when the provider does not expose it. In-stream compaction is reported when the completed response is processed.

Continuation keeps previous_response_id, tool outputs and subsequent user input; it does not resend the opaque item or the old raw history. Local conversation records remain independent for UI/audit purposes. No separate /responses/compact endpoint is invoked. See https://developers.openai.com/api/docs/guides/compaction.

Metadata

Release files for modmex-ai 0.7.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for modmex-ai 0.7.0
File Size Uploaded Provenance Actions
modmex_ai-0.7.0.tar.gz 197.7 kB With provenance Details for modmex_ai-0.7.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for modmex-ai 0.7.0
File Interpreter ABI Platform Actions
modmex_ai-0.7.0-py3-none-any.whl Python 3 none any Details for modmex_ai-0.7.0-py3-none-any.whl

Total release size: 330.1 kB

Release files / modmex_ai-0.7.0.tar.gz

Download URL modmex_ai-0.7.0.tar.gz
Size 197.7 kB
Tags Source
SHA-256 checksum
How to use checksums
87d5d969717dee695b6aaf21597d1c8feb44002105080b0b601cb34cd1f4fe27
BLAKE2b-256 checksum
How to use checksums
ab137ad6961641c69a21283ff0174ec170c45e030f3658c608eacb7e989ef6d4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 22, 2026.

Transparency log

Release files / modmex_ai-0.7.0-py3-none-any.whl

Download URL modmex_ai-0.7.0-py3-none-any.whl
Size 132.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
58dd50c9d8976c920908aea3a325a1f84ee3b5e2e09321747e9646748b37781d
BLAKE2b-256 checksum
How to use checksums
814f2885a50296f44bea3932de478f90e293d1444d1346fdbd271f71d7ae2507
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 22, 2026.

Transparency log

Security for modmex-ai 0.7.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for modmex_ai-0.7.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 22, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page