mankinds-sdk 2.0.1

Python SDK for Mankinds API

Mankinds SDK

Official Python SDK for the public Mankinds AI evaluation lifecycle API.

---

What It Exposes

The SDK exposes the public /api/v1 lifecycle API:

• Systems - create, list, get, update and archive AI systems.
• Connections - create target connections, test them and set defaults.
• Datasets - generate, import, get, update and validate datasets.
• Test plans - generate and read system test plans.
• Evaluations - run, list, get, cancel and explicitly wait for evaluations.
• Jobs - poll asynchronous operations.
• Agents and gates - trigger CI runs and enforce quality gates.
• Evidence, findings and remediation - retrieve evidence packs, findings and remediation items.

It does not expose Mankinds administration routes, webapp internals, OAuth flows, Copilot, ontology internals, users, roles or organization settings.

Installation

pip install "mankinds-sdk>=2.0.0"

Quickstart

import os
from mankinds_sdk import MankindsClient

client = MankindsClient(api_key=os.environ["MANKINDS_API_KEY"])

system = client.systems.create({
    "name": "Support Assistant",
    "description": "Customer support assistant for billing and account questions.",
})

connection = client.connections.create(system["id"], {
    "connector_type": "custom_api",
    "url": "https://api.example.com/chat",
    "method": "POST",
    "body": {"message": "{{input}}"},
    "input_path": "message",
    "output_path": "reply",
})

dataset_job = client.datasets.generate(system["id"], {"scenario_count": 20})
client.jobs.wait(dataset_job["id"])
client.datasets.validate(system["id"])

run = client.evaluations.run({
    "system_id": system["id"],
    "profile": "required",
    "connection_id": connection["id"],
})

evaluation = client.evaluations.wait(run["evaluation_id"])
print(evaluation.get("summary"))

Client

client = MankindsClient(api_key, base_url=None, timeout=120)

Parameter	Type	Required	Default	Description
api_key	str	Yes	-	API key sent as X-API-Key.
base_url	str	No	https://app.mankinds.io	Custom API base URL.
timeout	int	No	120	Request timeout in seconds.

Use client.whoami() to inspect the authenticated organization, actor and scopes.

Endpoint Configuration

Create AI target connections with client.connections.create(system_id, config).

client.connections.create(system["id"], {
    "connector_type": "custom_api",
    "url": "https://api.example.com/chat",
    "method": "POST",
    "headers": {"Authorization": "Bearer ..."},
    "body": {"message": "{{input}}"},
    "input_path": "message",
    "output_path": "reply",
})

Common fields:

Field	Type	Description
connector_type	str	Target connector type, for example custom_api.
url	str	Target endpoint URL.
method	str	HTTP method. Defaults to POST server-side.
headers	dict	Headers sent to the target endpoint.
body	dict	Request body template.
input_path	str	Path where Mankinds injects the test input.
output_path	str	Path where Mankinds reads the AI response.
streaming	dict	SSE streaming configuration.
multiturn	dict	Multi-turn conversation configuration.
session	dict	Dynamic auth/session token configuration.

Datasets

job = client.datasets.generate(system["id"], {"scenario_count": 20})
dataset = client.jobs.wait(job["id"])

imported = client.datasets.import_(system["id"], [
    {
        "input": "Where is my order?",
        "expected_outputs": ["I can help you track your order."],
    },
])

client.datasets.validate(system["id"])

Evaluations

Evaluation start is asynchronous. Waiting is explicit.

run = client.evaluations.run({
    "system_id": system["id"],
    "connection_id": connection["id"],
    "profile": "required",
    "mode": "offline",
})

evaluation = client.evaluations.wait(
    run["evaluation_id"],
    timeout_ms=30 * 60 * 1000,
    interval_ms=5000,
)

CI Quality Gates

Use agents for CI checks. Configure a gate once, trigger a CI run, then wait for the gate result. If the gate fails, the SDK raises GateFailedError.

import os
from mankinds_sdk import GateFailedError

agents = client.agents.list({"system_id": system["id"]})
agent = agents[0]

client.gates.update(agent["id"], {
    "overall_score_min": 0.85,
    "max_failed_criteria": 0,
})

run = client.agents.run(agent["id"], {
    "trigger_source": "ci",
    "metadata": {
        "commit_sha": os.environ.get("GITHUB_SHA"),
        "pr_number": os.environ.get("PR_NUMBER"),
    },
})

try:
    client.agents.wait_for_gate(agent["id"], run["id"])
except GateFailedError as exc:
    print(exc.gate)
    raise

Namespaces

Namespace	Methods
client.systems	create, list, get, update, archive
client.connections	create, list, test, set_default
client.datasets	generate, import_, get, update, validate
client.test_plans	generate, list, get
client.evaluations	run, get, list, cancel, wait
client.jobs	get, wait
client.agents	list, get, run, get_run, wait_for_gate
client.gates	get, update
client.evidence	get_pack
client.findings	list
client.remediation	list

Errors

The SDK maps API failures to typed errors where possible:

• CredentialsError
• AuthenticationError
• ScopeDeniedError
• ValidationError
• NotFoundError
• RateLimitError
• ServerError
• JobFailedError
• PollTimeoutError
• GateFailedError

Scopes

API keys are scoped. The organization is derived from the key; do not send organization_id.

Common scopes:

• systems:read, systems:write
• connections:read, connections:write
• datasets:read, datasets:write
• test_plans:read, test_plans:write
• evaluations:read, evaluations:run, evaluations:cancel
• agents:read, agents:run
• gates:read, gates:write
• evidence:read
• findings:read, remediation:read

Documentation

See the full SDK and REST API documentation at docs.mankinds.io.

License

MIT