Skip to main content

ApiWells

ApiWells is the Python package and CLI for ApiWells developer tooling. Its current public capability, Endpoint Doctor, is a Local-First CLI for diagnosing OpenAI-compatible model API endpoints from your machine. Endpoint Doctor checks connectivity and protocol capabilities; it does not score model quality. Python 3.10+ is required.

Runtime dependencies include jsonschema for Structured Output validation.

Install

Install the latest release from PyPI:

python -m pip install --upgrade apiwells
apiwells --version

apiwells --version reports the installed package version.

Install this local checkout with:

python -m pip install .

Basic Doctor

Set APIWELLS_API_KEY using your shell's secret-input mechanism. Do not put a real key in a command, screenshot, issue or repository. Use --api-key-env NAME to select another environment variable.

apiwells doctor --base-url https://YOUR-API-HOST/v1
apiwells doctor --base-url https://YOUR-API-HOST/v1 --v2-json

Basic is the default mode: URL → DNS → TLS → HTTP → Authentication → /models. It sends no inference request. HTTP, authentication and model-list checks share one GET to BASE/models. An empty model list can pass the response-shape check; a model list alone does not prove inference works.

Supply the exact API base: /v1 is not added automatically. For a gateway with /openai/v1, include that prefix. Do not supply the full /models or /chat/completions URL.

For a local unauthenticated development server:

apiwells doctor --base-url http://127.0.0.1:3000/v1 --anonymous

Deep Doctor

apiwells doctor --base-url https://YOUR-API-HOST/v1 --deep --model YOUR-MODEL-ID
apiwells doctor --base-url https://YOUR-API-HOST/v1 --deep --model YOUR-MODEL-ID --v2-json

Deep requires an explicit, nonempty model ID and follows: Basic → Chat → Streaming/SSE → Usage → Tool Calling → Structured Output. Usage is extracted from Chat and Streaming responses, not a separate request. Capability probes run only after Basic passes and the requested model is found.

Deep makes multiple potentially billable inference requests:

  • Chat validates a nonempty assistant text response.
  • Streaming validates SSE framing, output and terminal [DONE]; it reports TTFT and total latency. TTFT is measured from request start to the first valid model-output event. HTTP headers, keepalive, role-only, empty and metadata-only events do not count.
  • Usage contains provider-reported token counts only. Missing values remain unavailable or null; ApiWells never estimates them.
  • Tool Calling must complete a full round trip: tool call, validated arguments, local diagnostic tool execution, tool result, second request and a final response using that result. Merely accepting tools is insufficient.
  • Structured Output must return JSON that passes the requested JSON Schema. An explicit unsupported rejection or a sufficiently specific feature-related json_schema/response_format rejection can trigger one additional, potentially billable json_object sub-test. That sub-test does not by itself classify strict schema support as unsupported; support may remain unknown, and JSON-object-only success does not certify strict JSON Schema support.

There are no hidden billable retries or fallback models. The Tool Calling second request and conditional Structured Output sub-test are diagnostic steps, with request counts included in their results. --max-tokens controls applicable generation budgets, including both Tool Calling requests in Deep mode, and is not a guaranteed total cost ceiling; diagnostic sub-tests may use bounded request parameters. Provider support varies; not every endpoint supports every capability.

Legacy chat compatibility

apiwells doctor --base-url https://YOUR-API-HOST/v1 --chat --model YOUR-MODEL-ID --json

Legacy --chat remains a single, opt-in, potentially billable non-streaming request to BASE/chat/completions, with Reply OK. and default max_tokens=8. A pass requires nonempty assistant text, not exactly OK. Use --max-tokens 32 to change the budget; some models need more output tokens. It performs no automatic retry or fallback model selection. --chat and --deep are mutually exclusive.

Results and timing

Probe execution status (PASS, FAIL, PARTIAL) is separate from capability support (SUPPORTED, UNSUPPORTED, UNKNOWN, NOT_APPLICABLE). A failed or partial probe with unknown support does not establish that a capability is unsupported. Core failures make the overall result FAIL; non-passing Streaming, Tool Calling or Structured Output probes make it PARTIAL when core checks pass.

The exit-code contract remains:

Code Meaning
0 Selected check / overall diagnostics passed
1 Endpoint diagnostics did not pass, including an overall PARTIAL
2 Local usage or configuration error

For Basic/Deep, --v2-json writes a versioned JSON report with schema_version: "1", apiwells_version, overall_status and probes. Each probe includes status, support, summary, metrics, evidence and error code. The report schema version is independent of the package version. --json retains the compatibility report with ok; legacy --chat --json retains its single-check report. Use --json, not --v2-json, for legacy chat. --chat --v2-json is rejected before any diagnostic request, including when --json is also supplied: stdout is empty, stderr explains the supported combination, and the exit code is 2. With a supported JSON option, completed checks produce one JSON object on stdout; usage/configuration errors go to stderr and do not produce a JSON report.

Error codes distinguish configuration, DNS/TLS/connectivity/timeout, HTTP/auth, JSON/schema, SSE/interruption and capability validation failures. They are diagnostic evidence, not definitive root-cause conclusions.

--timeout 15 is a socket-operation timeout, not an overall deadline. Client TTFT and total latency include transport effects and are not server-only inference timings. Legacy elapsed_ms is request/response elapsed time, not streaming TTFT.

Security and boundaries

  • Local-First execution; no telemetry or hidden upload of prompts, responses or usage. Requested diagnostic traffic goes to the endpoint you select.
  • API keys and Authorization header values must not be printed. Centralized redaction protects console/JSON reports and error paths; raw provider bodies are not dumped into reports. Review reports before sharing.
  • HTTPS certificate verification stays enabled. Remote HTTP requires --allow-http; literal loopback addresses and localhost are allowed for development.
  • All redirects, including same-host redirects, are blocked so credentials are not forwarded to redirect destinations.
  • Environment proxies are ignored unless --use-env-proxy is supplied.
  • Test only endpoints you are authorized to test. Private/local destinations are intentionally allowed; this CLI is not an unrestricted server-side URL fetcher.

See Security for handling and reporting guidance.

Compatibility and development

Compatibility indexes three point-in-time ED-031 live certification records. These establish observed endpoint behavior, not universal provider support or model quality.

The release-facing specification summarizes the frozen v0.2 scope. Benchmark, scoring/ranking, recommendations, cost optimization, Router, Gateway, long-term Monitoring, Web Dashboard, SaaS accounts, cloud telemetry and complete Responses API coverage are outside that scope.

For test coverage and the observed QA harness dependencies, see the test plan. With the test harness installed, run:

python -m pytest tests -q

See the release checklist for release evidence, and the changelog for version history and changes. License: MIT.

Metadata

Release files for apiwells 0.2.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for apiwells 0.2.3
File Size Uploaded Provenance Actions
apiwells-0.2.3.tar.gz 93.1 kB Without provenance Details for apiwells-0.2.3.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for apiwells 0.2.3
File Interpreter ABI Platform Actions
apiwells-0.2.3-py3-none-any.whl Python 3 none any Details for apiwells-0.2.3-py3-none-any.whl

Total release size: 135.1 kB

Release files / apiwells-0.2.3.tar.gz

Download URL apiwells-0.2.3.tar.gz
Size 93.1 kB
Tags Source
SHA-256 checksum
How to use checksums
3f60aca2c03840b9c1f4b0ee8be6a744da20267eb8e2fd373148658a19417aaa
BLAKE2b-256 checksum
How to use checksums
260056f54247c9d3fa622b1c82fea8c6f1ed30aa6dbeac42ae3a1ada9d30260b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.13

Release files / apiwells-0.2.3-py3-none-any.whl

Download URL apiwells-0.2.3-py3-none-any.whl
Size 42.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
68a36471a4429378653f621841d4f38eec27b2f66f860fab503d5f3804ed7324
BLAKE2b-256 checksum
How to use checksums
162c19f9e67f064a2a4b6586d4df86d0b527ce8783c052f936902b61051ff02c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.13

Security for apiwells 0.2.3

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page