Skip to main content

Railengine Retrieval SDK

Python SDK for retrieving and searching data from Railtown AI Railengine - handles embeddings, storage documents, and indexed content.

Overview

The rail-engine package provides a Pythonic interface for retrieving and searching data from Railengine. It supports async/await patterns, client-side filtering, explicit storage list paging via StorageListPage, and Pydantic model deserialization.

Installation

uv pip install rail-engine

Quick Start

import asyncio
from railtown.engine import Railengine

async def main():
    # Initialize client (reads from ENGINE_PAT and ENGINE_ID env vars)
    async with Railengine() as client:
        page = await client.list_storage_documents(
            page_number=1,
            page_size=25,
        )
        print(f"total_count={page.total_count} total_pages={page.total_pages}")
        for doc in page.items:
            print(doc)

asyncio.run(main())

Configuration

Environment Variables

  • ENGINE_PAT (required) - Personal Access Token
  • ENGINE_ID (required) - Engine ID (can also be passed to constructor)
  • RAILTOWN_API_URL (optional) - Base API URL (defaults to https://cndr.railtown.ai/api)

Constructor Parameters

  • pat (optional) - PAT token (if not provided, reads from ENGINE_PAT env)
  • engine_id (optional) - Engine ID (if not provided, reads from ENGINE_ID env, required if not in env)
  • api_url (optional) - Base API URL (if not provided, reads from RAILTOWN_API_URL env or defaults to production)
  • model (optional) - Pydantic model type for deserializing retrieved data
  • timeout (optional) - Request timeout in seconds or httpx.Timeout (default 30). Applied to the SDK-created client and to each request, including vector search.
  • verify (optional) - TLS verification. Defaults to True. Pass a CA bundle path or ssl.SSLContext for custom trust. False disables verification and must be set explicitly.
  • http_client (optional) - Caller-owned httpx.AsyncClient for proxies, custom transports, or tests. The SDK does not close this client.
  • retrieval_error_mode (optional) - "raise" (default) or "fallback"; see the error contract below.
  • logging_policy (optional) - LoggingPolicy for SDK logs. Default logs operation metadata only. Search text, JSONPath, and response bodies are not logged unless log_content=True.

Earlier SDK versions disabled TLS verification. 0.4.0 verifies certificates by default. Use verify= with your CA bundle if the API host uses a private certificate.

Features

  • Multiple retrieval methods:
    • search_vector_store() - Semantic search in vector stores
    • search_vector_store_filtered() - Semantic search restricted to the documents an index query matches (search_query takes the same IndexingSearchQuery/dict as search_index; requires Indexing and Embeddings enabled on the engine)
    • get_storage_document_by_event_id() - Get document by EventId
    • get_storage_document_by_customer_key() - Same paging as list_storage_documents (page_number, page_size) but scoped to a CustomerKey; returns StorageListPage with items, total_pages, and total_count
    • query_storage_by_jsonpath() - Query using JSONPath
    • list_storage_documents() - Use page_number and page_size to paginate; returns StorageListPage with items, total_pages, and total_count
    • search_index() - Index search (default raw=True: each hit is the API dict; use raw=False to parse each hit's body with model or the client default); returns IndexingSearchResult with items and optional total_count
    • delete_event() - Delete an event from storage, embeddings, and indexing (raises on API errors)
  • Client-side filtering on storage reads - Optional filter_fn on storage/list/query helpers (not on search_vector_store or search_vector_store_filtered; filter vector hits in your loop)
  • Storage list pagination - One HTTP GET per page; use page_number / page_size on the request and total_pages / total_count on StorageListPage when looping
  • Model deserialization - Optional Pydantic model support with per-call override
  • Actionable error handling - Retrieval failures raise by default; explicit fallback mode supports legacy integrations
  • Async/await support - Built for modern async Python applications

Error Handling

The SDK provides custom exception classes:

  • RailtownError - Base exception
  • RailtownBadRequestError - 400 Bad Request
  • RailtownUnauthorizedError - 401 Unauthorized
  • RailtownNotFoundError - 404 Not Found
  • RailtownConflictError - 409 Conflict
  • RailtownServerError - 5xx Server errors

Error contract and migration to 0.5.0

Version 0.5.0 changes retrieval failures to raise by default. Successful empty queries keep their existing result types and pagination metadata. To retain the previous retrieval fallback behavior, explicitly construct Railengine(retrieval_error_mode="fallback"). Fallback logs the failure and returns an empty list, empty result/page, or None for an event lookup. It does not suppress ingestion or deletion failures, invalid arguments, cancellation, or application callback errors. User-model validation and per-record deserialization fallback are unchanged.

A 404 raises RailtownNotFoundError by default, including event lookups. The current API contract does not establish a reliable discriminator between a missing document and another missing resource; the SDK does not infer which one is absent or perform another request to guess.

Catch the common base class for all request/response errors:

from railtown.engine import Railengine, RailtownError

async def search():
    async with Railengine() as client:
        try:
            return await client.search_index(query={"search": "example"})
        except RailtownError as error:
            print(error.operation, error.status_code, error.request_id)
            raise

HTTP 400, 401, 403, 404, 409, and 429 map to RailtownBadRequestError, RailtownUnauthorizedError, RailtownForbiddenError, RailtownNotFoundError, RailtownConflictError, and RailtownRateLimitError. Other 4xx responses use RailtownError; 5xx responses use RailtownServerError. Transport failures use RailtownTransportError, timeouts its subclass RailtownTimeoutError, and invalid response JSON/envelopes use RailtownResponseError. For retrieval (rail-engine), import these exceptions from railtown.engine. For ingestion (rail-engine-ingest), import them from railtown.engine.ingest. The railtown.engine.ingest namespace requires rail-engine-ingest; it is not included in the retrieval wheel.

Request/response errors expose status_code when available, operation, the effective engine_id, request_id when available, and retryable. Retryable is true for timeouts, connection failures, and HTTP 408, 429, 502, 503, and 504. It describes a potentially transient failure, not whether replaying a write is safe. The SDK does not retry automatically.

Diagnostic compatibility change: SDK-generated errors now leave response_text as None. The constructor still accepts an explicitly supplied value. Messages exclude query strings and payloads, but the preserved original exception cause can retain sensitive data. Full traceback logging is not guaranteed to be sanitized. Explicit content logging remains governed by LoggingPolicy and does not populate error attributes.

Requirements

  • uv
  • Python 3.11+
  • httpx >= 0.24.0
  • pydantic >= 1.10.0

For ingesting data into Railengine, install and use rail-engine-ingest.

License

MIT

Metadata

Release files for rail-engine 0.5.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for rail-engine 0.5.1
File Size Uploaded Provenance Actions
rail_engine-0.5.1.tar.gz 27.6 kB Without provenance Details for rail_engine-0.5.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for rail-engine 0.5.1
File Interpreter ABI Platform Actions
rail_engine-0.5.1-py3-none-any.whl Python 3 none any Details for rail_engine-0.5.1-py3-none-any.whl

Total release size: 58.1 kB

Release files / rail_engine-0.5.1.tar.gz

Download URL rail_engine-0.5.1.tar.gz
Size 27.6 kB
Tags Source
SHA-256 checksum
How to use checksums
c97318a66fda6068c1bf59611ba625ade076fb9e2bc0a1a5dbef9edbf7e6d292
BLAKE2b-256 checksum
How to use checksums
165a4a3eb5f589401f7fc6c47e6252d291c417ba10f20852962d3d294c863336
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.5 {"installer":{"name":"uv","version":"0.13.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / rail_engine-0.5.1-py3-none-any.whl

Download URL rail_engine-0.5.1-py3-none-any.whl
Size 30.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
91ee9d02b7b08708b5adc82330ad8e53b92591b677a106dc9422a21d4d3427aa
BLAKE2b-256 checksum
How to use checksums
957f9c3379e4c66ffd19d7c8bbe2a7830bf9c255519777580c5bf3b84abe48b8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.5 {"installer":{"name":"uv","version":"0.13.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for rail-engine 0.5.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page