mantis-gw 1.0.2

Mantis SDK

Mantis SDK

Python SDK for sending chat completion requests to the Mantis LLM gateway /v1/chat/completions endpoint.

Requirements

• Python 3.12+
• A running Mantis service
• A gateway API token

Install

pip install mantis_gw

Development

Use uv sync only when working on this repository locally. It uses the Python version in .python-version, creates the virtual environment, and installs this SDK from the local source tree with the dependencies pinned in uv.lock.

uv sync

Non-streaming

url should be the gateway base URL and should not include /v1/chat/completions. The SDK appends that endpoint path automatically.

import asyncio

from mantis_gw import gateway


async def main() -> None:
    client = gateway.Gateway(
        url="https://gateway.example.com",
        token="gw_token-id_token-secret",
    )

    response = await client.send(
        {
            "messages": [
                {"role": "user", "content": "Write a short project summary."},
            ],
            "stream": False,
            "temperature": 0.5,
            "max_tokens": 256,
            "system": "Answer clearly and concisely.",
        },
        metadata={"task-type": "summarization"},
    )

    print(response)


asyncio.run(main())

Streaming

When stream is True, send() returns an async iterator of text chunks.

import asyncio

from mantis_gw import gateway


async def main() -> None:
    client = gateway.Gateway(
        url="https://gateway.example.com",
        token="gw_token-id_token-secret"
    )

    chunks = await client.send(
        {
            "messages": [
                {"role": "user", "content": "Write a short project summary."},
            ],
            "stream": True,
            "temperature": 0.5,
            "max_tokens": 256,
            "system": "Answer clearly and concisely.",
        },
        metadata={"task-type": "summarization"},
    )

    async for chunk in chunks:
        print(chunk, end="")


asyncio.run(main())

Request Payload

The SDK sends the request payload to llm-gateway as-is. Gateway-side validation still applies to the endpoint:

• messages is required and must contain at least one message.
• Each message must only contain role and content.
• role must be "user" or "assistant".
• content must be a non-empty string after trimming whitespace.
• Extra top-level request fields are rejected.
• stream defaults to False.
• temperature can be omitted, None, or a number from 0.0 to 2.0.
• max_tokens can be omitted, None, or an integer greater than 0.
• system can be omitted, None, or a non-empty string after trimming whitespace.

Metadata

Pass routing metadata with the metadata keyword argument. Keys and values must be strings. The SDK sends it as the gateway's metadata HTTP header.

await client.send(
    {
        "messages": [{"role": "user", "content": "Generate a Python function."}],
        "stream": False,
    },
    metadata={"task-type": "code_generation"},
)

Errors

Gateway responses with 4xx or 5xx status codes raise httpx.HTTPStatusError.

Integration Tests

The integration tests read gateway credentials from a local .env file in the repo root:

MANTIS_GATEWAY_URL=https://gateway.example.com
MANTIS_GATEWAY_TOKEN=gw_token-id_token-secret

Run them with:

uv run pytest

The .env file is ignored by Git so real credentials stay local.