Skip to main content

Railengine Ingestion SDK

Python SDK for ingesting data into Railtown AI Railengine - handles embeddings, storage, indexing, and webhook publishing.

Overview

The rail-engine-ingest package provides a Pythonic interface for publishing data to Railengine. It supports async/await patterns, Pydantic model validation, and familiar configuration patterns.

Installation

uv pip install rail-engine-ingest

Quick Start

# Copy your ENGINE_TOKEN from https://cndr.railtown.ai Project Settings

import asyncio
import os
from railtown.engine.ingest import RailengineIngest
import base64
import json

engine_token = os.getenv("ENGINE_TOKEN")

async def main():
    # Initialize client
    async with RailengineIngest(engine_token=engine_token) as client:
        # Ingest data
        data = {
            "name": "Chicken Shawarma",
            "meal": "lunch",
            "calories": 95
        }
        response = await client.upsert(data)
        print(f"Status: {response.status_code}")

asyncio.run(main())

Configuration

Environment Variable

  • ENGINE_TOKEN - Base64-encoded JSON string containing ingestion credentials

Constructor Parameters

  • engine_token (optional) - ENGINE_TOKEN string (if not provided, reads from env)
  • model (optional) - Pydantic model type for validating ingested data
  • timeout (optional) - Request timeout in seconds or httpx.Timeout (default 30)
  • verify (optional) - TLS verification. Defaults to True. Pass a CA bundle path or ssl.SSLContext for custom trust. False disables verification and must be set explicitly.
  • http_client (optional) - Caller-owned httpx.AsyncClient for proxies, custom transports, or tests. The SDK does not close this client.
  • logging_policy (optional) - LoggingPolicy for SDK logs. Default INFO logs include engine id, status, duration, and payload size. The request body is not logged unless log_content=True.

Earlier SDK versions disabled TLS verification. 0.4.0 verifies certificates by default. Use verify= with your CA bundle if the ingestion host uses a private certificate.

SDK loggers are named (railtown.engine.*). Default INFO logging does not write the upsert body. Pass logging_policy=LoggingPolicy(log_content=True) only when you explicitly want redacted, truncated payload logs.

Features

  • Single upsert() method - Unified interface for ingesting data
  • Multiple data formats - Accepts Pydantic models, dictionaries, or JSON strings
  • Model validation - Optional Pydantic model validation
  • Direct JSON payload - Sends data directly as JSON to the ingestion endpoint
  • Webhook handler - get_webhook_handler() returns a payload parser when the client is constructed with model=
  • Async/await support - Built for modern async Python applications

Error Handling

The SDK provides custom exception classes:

  • RailtownError - Base exception
  • RailtownBadRequestError - 400 Bad Request
  • RailtownUnauthorizedError - 401 Unauthorized
  • RailtownNotFoundError - 404 Not Found
  • RailtownConflictError - 409 Conflict
  • RailtownServerError - 5xx Server errors

Exceptions are raised on errors. In 0.5.0, ingestion shares the retrieval error classification: HTTP 403 and 429 use RailtownForbiddenError and RailtownRateLimitError, transport failures use RailtownTransportError, and timeouts use its subclass RailtownTimeoutError. HTTP 404 and 409 use RailtownNotFoundError and RailtownConflictError. These types are exported from railtown.engine.ingest; all inherit from RailtownError.

Request errors expose status_code when available, operation, engine_id, request_id when available, and retryable. Retryable is true for timeouts, connection failures, and HTTP 408, 429, 502, 503, and 504. It does not mean replaying an ingestion request is safe, and no automatic retries are performed.

Migration to 0.5.0: SDK-generated errors leave response_text as None; existing exception constructors still accept explicitly supplied values. SDK-generated messages exclude request/response payloads, but the preserved original cause can retain sensitive data. Full traceback logging is not guaranteed to be sanitized. Ingestion always raises on request failures, independent of the retrieval compatibility mode. User-model validation behavior is unchanged.

Requirements

  • uv
  • Python 3.11+
  • httpx >= 0.24.0
  • pydantic >= 1.10.0

For retrieving and searching data from Railengine, install and use rail-engine.

License

MIT

Metadata

Release files for rail-engine-ingest 0.5.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for rail-engine-ingest 0.5.1
File Size Uploaded Provenance Actions
rail_engine_ingest-0.5.1.tar.gz 32.9 kB Without provenance Details for rail_engine_ingest-0.5.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for rail-engine-ingest 0.5.1
File Interpreter ABI Platform Actions
rail_engine_ingest-0.5.1-py3-none-any.whl Python 3 none any Details for rail_engine_ingest-0.5.1-py3-none-any.whl

Total release size: 68.6 kB

Release files / rail_engine_ingest-0.5.1.tar.gz

Download URL rail_engine_ingest-0.5.1.tar.gz
Size 32.9 kB
Tags Source
SHA-256 checksum
How to use checksums
e8ef266f62ce8fc5f9ee4e984b256e90032026c5c2fcc7f7c60154c11d66b1e9
BLAKE2b-256 checksum
How to use checksums
2fa0d8bbd99a1e9c2051bf7363bbb70b46f649b6e40534f81272b735fe3e1ec6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.5 {"installer":{"name":"uv","version":"0.13.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / rail_engine_ingest-0.5.1-py3-none-any.whl

Download URL rail_engine_ingest-0.5.1-py3-none-any.whl
Size 35.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a98a373befe39f31152edba831c49ffe819a6fd82efdf65d05fa8709726059ec
BLAKE2b-256 checksum
How to use checksums
d6692c8ef5c7ced810c9a59a2a9741d935658292a0bba9328abe51fc2e4f7f05
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.5 {"installer":{"name":"uv","version":"0.13.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for rail-engine-ingest 0.5.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page