WonderSearch for Python

Upload files and retrieve relevant passages with matching synchronous and asynchronous clients. Install the SDK from PyPI with pip install wondersearch.

pip install wondersearch

Configure WONDERSEARCH_API_KEY with a workspace API key from Console. The SDK connects to https://api.wondersearch.ai unless you set WONDERSEARCH_BASE_URL or pass base_url. POLYGRES_API_KEY and POLYGRES_BASE_URL remain supported and take precedence when both names are set. A read_write key can upload and change the workspace default; a search key retains its existing read/search permissions.

from wondersearch import WonderSearch

with WonderSearch() as client:
    batch = client.upload("handbook.pdf")
    results = client.search("What is our refund policy?")
    for passage in results.results:
        print(passage.text, passage.document_id, passage.document_revision)

from wondersearch import AsyncWonderSearch


async def search_handbook():
    async with AsyncWonderSearch() as client:
        await client.upload(["handbook.pdf", "policies.docx"])
        return await client.search("What is our refund policy?")

search() returns ranked passages and citations, not a generated answer. Empty results is a successful search. Every effort returns at most ten results, or fewer when fewer eligible matches exist. It defaults to the medium effort. Advanced options include effort, folder_id, group_by_document, timeout_ms (server search budget), and timeout (whole SDK operation budget). The server enforces query length, quotas and enabled efforts.

ask() is a deprecated alias of search() with the same arguments and response. It emits one SDKDeprecationWarning per process and will be removed in the next major version. Only ask() accepts the legacy limit argument, which is ignored and not sent to the API.

The SDK has no model-selection argument, and the API ignores a model request field. The response's model field names the WonderSearch version that served the search. group_by_document=False permits several matching passages from the same document.

Compare effort="small" and effort="medium" on representative questions from your collection.

Drives

New workspaces have a selected Default drive. Change it on the Storage page or through the SDK:

client.drives.list()
client.drives.get_default()
client.drives.set_default("DRIVE_UUID")
client.upload("handbook.pdf", drive_id="DRIVE_UUID")
client.search("Refund policy?", drive_id="DRIVE_UUID")

Create a drive with either client:

drive = client.drives.create("Knowledge Base")
print(drive.id, drive.status)
# Async: drive = await client.drives.create("Knowledge Base")

create(name, *, timeout=30, idempotency_key=None) returns a typed Drive. Names are trimmed and must contain 1-120 characters. Creation requires content write permission. The index may still be provisioning when this call returns; creation does not change the shared default. Use drives.set_default(drive.id) to select it explicitly. Reuse an idempotency_key and the same name to retry a creation whose response was lost.

Selection order is the method's drive_id, the client's drive_id, then the shared workspace default. The default is resolved once per operation and remains pinned through retries and polling. Explicit invalid drives never fall back. Changing the shared default affects subsequent calls by all workspace users. You must choose another default before deleting the selected default drive.

Uploads and recovery

upload() accepts a path or a list of paths for PDF, DOCX, TXT and Markdown, each from 1 through 1,000,000,000 bytes. It hashes and streams original bytes, follows proxy or direct transfer instructions, then optionally waits for indexing. Transfer concurrency defaults to three and is configurable with concurrency=1..50. Processing polling starts after the transfer phase, so indexing never occupies a transfer slot. Directories are not traversed. wait=False returns after queuing processing; the transfer phase has a timeout budget of 600 seconds by default. The subsequent processing phase has a separate processing_timeout budget of 600 seconds. Each budget covers its whole batch, including waiting for a slot.

Uploads return an UploadBatch, including for one file. Each receipt includes upload, drive and workspace IDs. Searchable receipts also contain document, operation and snapshot IDs. A fresh document identity is generated by default. Use a stable external_id on single-file uploads to deliberately replace that document with a server-assigned revision.

from wondersearch import UploadBatchError

try:
    batch = client.upload("handbook.pdf", idempotency_key="handbook-update-42")
except UploadBatchError as exc:
    print(exc.batch.files)  # Successful files are preserved.
    print(exc.idempotency_key, exc.drive_id)  # Reuse these for a retry.
    for filename, failure in exc.failures.items():
        print(filename, failure)

Reuse the same paths in the same order, content, metadata, drive and idempotency_key to retry an interrupted upload. Do not change the workspace default between those retries without passing the original explicit drive. Unfinished transfers expire after 24 hours. Durably queued processing and its storage reservation survive that deadline. Creation idempotency still expires after 24 hours; retain receipts and use uploads.resume() for queued work instead of recreating an old batch. SDK recovery is not a persistent upload queue.

For an upload already queued for extraction/indexing:

receipt = client.uploads.resume("UPLOAD_UUID", drive_id="DRIVE_UUID")

If bytes have not been sealed, resume() raises upload_transfer_incomplete and instructs you to repeat upload() with the original operation key. It does not pretend that polling can finish transferring bytes. Cancellation stops local waiting; already accepted server work may continue. A synchronous network request already in flight may finish cleanup after its caller's deadline; outstanding transport work is bounded.

Bulk imports

client.imports.run() loads a large collection of TXT and Markdown files into one drive. It requires bulk imports to be enabled for your organization; otherwise it raises BulkImportError with code not_found.

from wondersearch import WonderSearch

with WonderSearch(drive_id="DRIVE_UUID") as client:
    status = client.imports.run(
        "library/",                      # a directory, a file, or a list of paths and ImportFile
        journal="library-import.jsonl",  # run again with the same journal to resume
        on_progress=lambda p: print(p.phase, p.acknowledged_bytes, p.searchable_documents),
    )

The SDK plans chunks from file sizes (client.imports.plan() shows the plan without uploading), checks the declared totals against limits(), creates the import, uploads and acknowledges chunks with bounded concurrency (default 8, at most 32), seals it and waits until it is searchable. A chunk is a 4-byte big-endian manifest length, a wondersearch-import-chunk.v1 JSON manifest and the original file bytes; each file keeps its own manifest entry and external_id (its path relative to the directory, or as given, or set with ImportFile). Files must be non-empty UTF-8 and fit in one chunk (chunk_bytes, at most 32,000,000 bytes); use upload() for others.

• Combining. combine_bytes (0 to 1,000,000, off by default) joins neighbouring small files without metadata into one document, separated by a blank line. The combined document's external_id is <first external_id>+<count>, and its source_files metadata is a JSON list of [external_id, start_byte, end_byte], so a passage can be traced to its file.
• Retries. Backpressure (request_in_progress) waits for the service's Retry-After; outages, failed transfers and missing objects retry with a fresh upload URL. The service stays authoritative for drive limits (drive_limit_exceeded).
• Resume. The journal is append-only JSON lines with IDs, sizes and checksums, never credentials, URLs or text. A rerun with the same sources, options and journal skips acknowledged chunks; a sealed import only waits. Without a journal, pass the same import_id. Changed sources raise source_changed or journal_mismatch.
• Errors. BulkImportError carries import_id, drive_id, journal and, when known, import_status. Exceptions raised by on_progress propagate unchanged.
• Other actions. create, get, seal, wait, close, recover, failed_operations, retry_operation and retry_failed expose each step. After import_failed, call retry_failed() then wait(). import_revision_conflict means a document with that external_id already exists or was deleted: SDK imports only create documents, so retrying cannot help.

Memory use peaks at about concurrency times chunk_bytes. UTF-8 is checked when each chunk is prepared, so an invalid file stops the import at that point (invalid_import_file); other file and metadata problems are rejected before anything is uploaded.

While part of an import is applied, other writes to the same drive return temporarily_unavailable. upload() defaults to route="auto": a batch of at least 1,000 files or 1,000,000,000 bytes, all TXT or Markdown that fit in a chunk, with no folder_id or external_id, goes through an import when imports are enabled, and returns an UploadBatch with import_status and no receipts. Other batches upload as before. route="imports" always imports and route="uploads" never does. upload() rejects directories on the import route; pass them to imports.run(). On the import route, failures raise BulkImportError, the upload timeouts apply, idempotency_key or journal makes a rerun resume, a failed open import without either is closed, and each document's external_id is <import_id>/<path> (or <import_id>/sha256:<hex> when that exceeds 1,024 bytes), so uploading the same files again creates new documents.

Limits

Each drive holds at most 1,000,000 documents and 100 GB of files; a single file can be up to 1 GB. Read the values the service currently enforces with client.limits() (or await client.limits()); each client fetches them at most once an hour. Treat them as guidance only: the service enforces its current limits and raises a PolygresError with code drive_limit_exceeded (HTTP 409) when a drive would grow past them. Call client.limits(refresh=True) after such an error; after a limit changes, cached values can be up to about two hours old.

limits = client.limits()
print(limits.max_drive_documents, limits.max_drive_source_bytes, limits.max_file_bytes)

Errors and citations

All API errors derive from PolygresError and expose status, code, request_id, details and retry_after. Unknown server codes remain inspectable. Upload failures retain receipts when a session exists. UploadBatchError retains successes and per-file errors.

Search and mutation retries use the same payload and idempotency key, with at most two retries and bounded backoff. When a retried search finds the first attempt still running (HTTP 409 search_request_in_progress), the SDK keeps polling with the same key until the call's timeout, without using up those retries. Polls wait at least the service's Retry-After, backing off from 0.5 s to 4 s with jitter; the last wait is shortened so a final poll lands about 0.5 s before the deadline. Transport failures while polling still count against the two retries. If the search is still running at the deadline, the SDK raises SearchInProgressTimeoutError (an SDKTimeoutError, code search_in_progress_timeout) with the idempotency_key, request_id and status of the last reply. Call search() again with the same question, options and idempotency_key to fetch the result without running the search again. Upload admission receiving upload_limit instead honors Retry-After, adds jitter and waits until the transfer deadline. A search whose replay has expired is never silently re-executed. Supplying your own idempotency_key lets you identify a logical search across invocations.

Search results include passage text and citation identifiers directly. New search responses include the selected model; historical replay responses may omit it. Snapshot IDs, cache counts and timings remain internal. passages.get() requires a separately known snapshot ID, such as one retained from an upload receipt. A current drive snapshot may differ from the snapshot used by a concurrent search, so it is not a substitute for that exact snapshot.

An expired snapshot raises the server error instead of returning newer content. SearchResponse.raw preserves additive response fields, and usage amounts retain the server's exact decimal representation.

Versions

Package versions use semantic versioning and PEP 440 release candidates. 0.3.0 targets API /v1 and is prepared for release. During the 0.x series, minor releases may introduce breaking changes; patch releases contain compatible fixes. From stable v1 onward, removals receive at least 90 days' notice and a migration guide.

Earlier 1.0.0rc* versions were TestPyPI previews. Version 0.1.0 begins the production release series and sorts below those previews. When migrating, after publication explicitly install wondersearch==0.3.0 from PyPI and change imports from polygres_search to wondersearch. Version 0.1.1 corrects the client names to WonderSearch and AsyncWonderSearch. Update imports and constructors from PolygresSearch and AsyncPolygresSearch; the old names are not exported. This initial branding correction is an exception to the patch compatibility policy.

Version 0.2.0 stops sending a model selector; the service chooses the search model. Search returns at most ten results, and the legacy limit argument is ignored. The server enforces the active model's query limit. Retry an uncertain search with the same SDK version, request options, and idempotency key.

Version 0.1.2 makes an additional exception to the patch compatibility policy: upload(timeout=...) now covers transfers rather than the complete upload-and-processing operation. Set processing_timeout explicitly and allocate both budgets when enforcing an overall deadline.

The SDK sends its version on API requests (X-Polygres-SDK-Version and X-WonderSearch-SDK-Version) and processes compatibility metadata from the API. Unsupported API majors or explicitly unsupported SDK versions raise SDKCompatibilityError. Supported older versions receive SDKUpgradeWarning; deprecations use SDKDeprecationWarning.

From 0.3.0 the SDK also reads notices from every API response:

• Deprecation, Sunset and Link: <...>; rel="deprecation" headers, and entries in a response's top-level deprecations list (also on SearchResponse.deprecations), emit SDKDeprecationWarning naming the replacement, retirement date and migration guide.
• WonderSearch-Client-Notice: upgrade; latest=<v> emits SDKUpgradeWarning; deprecation; minimum=<v> emits SDKDeprecationWarning. Both include the exact pip install --upgrade command.
• HTTP 426 client_upgrade_required raises ClientUpgradeRequiredError (an SDKCompatibilityError) with minimum_version, latest_version and upgrade_command. HTTP 410 api_version_retired raises APIVersionRetiredError with replacement, docs_url and sunset_at.

Warnings occur at most once per notice per process and can be controlled with Python warning filters or version_warnings=False. Structured notices remain available at client.notices. A library must not reinstall itself inside an application, so the SDK only notifies: there is no automatic upgrade or import-time PyPI request.

Development and release

Use a project virtual environment. Run pytest, ruff check, and mypy src, then python -m build --no-isolation. Build tooling includes Hatchling, build, pytest, Ruff and mypy. The wheel needs only HTTPX and packaging plus their transitive dependencies.

Do not publish until package ownership, public metadata and package contents are approved. The release manifest must leave latest_stable_sdk_version null until a corresponding stable release actually exists. Deployment and package publication are separate actions. Python 3.10+ is the target; report the actual tested platforms and interpreter versions for each release.

The supplied tox.ini exercises installed wheels on Python 3.10 through 3.14. Run this matrix on Linux, macOS and Windows before stable publication; missing interpreters are failures rather than silently skipped. Tox is optional release tooling.

License

Licensed under Apache-2.0, matching the Polygres CLI and SDK.

Bulk uploads in 0.1.2

Multi-file uploads automatically use bulk requests when the API advertises support. Reservation and processing-status requests contain at most 50 files; byte transfers still obey concurrency (default 3, maximum 50). Capacity is enforced per drive by the server. Extraction and indexing do not consume transfer slots. wait=False returns only after source bytes have been verified and durably queued.

Use a stable idempotency_key and the same ordered file list when retrying an interrupted call. Reordering the list changes the derived per-file identities. Keep returned receipts and UploadBatchError failures for recovery. This SDK does not persist a local recovery manifest. Explicit single-file replacements use the existing endpoint and replacement quota rules; bulk admission reserves each new file’s full source size.