xgen-sdk

A general-purpose Python toolkit of powerful, reusable building blocks — the foundation the entire XGen platform is built on.

xgen-sdk is a curated library of strong, general-purpose Python methods and modules — the kind of logic every serious backend, agent runtime, or RAG pipeline ends up reinventing. Database pools, config management, object storage, ABAC authorization, quota evaluation, structured logging, LLM model discovery, in-app notifications — all consolidated into one canonical implementation that any Python project can pick up and use.

The XGen platform itself (xgen-core, xgen-workflow, xgen-documents, …) is essentially a thin layer of business logic built on top of this SDK. The SDK is where the heavy lifting lives.

One install. No optional extras. Everything you need, ready on pip install xgen-sdk.

Philosophy

xgen-sdk follows three rules:

Generalize relentlessly. If a piece of logic is useful in more than one place, it belongs in the SDK — not duplicated in each service.
Be opinionated where it matters, flexible where it doesn't. Sensible defaults from environment variables, escape hatches for everything.
Stay batteries-included. No [extras] to remember, no optional dependency dance. pip install xgen-sdk and you have the full toolkit.

The platform services are intentionally small. xgen-core is admin and auth business logic. xgen-workflow is workflow orchestration business logic. xgen-documents is document-processing business logic. All of the heavy infrastructure code lives here.

What's in the box

Module	Purpose
`xgen_sdk.db`	psycopg3 connection pool, model-based + table-name CRUD, raw SQL, retries
`xgen_sdk.config`	Redis-backed config with local-file fallback, typed `BaseConfig` registration
`xgen_sdk.storage`	MinIO / S3-compatible client — upload, download, listing, copy, presigned URLs
`xgen_sdk.auth`	Gateway header parsing, ABAC permissions with wildcards, FastAPI `Depends()` guards
`xgen_sdk.redis`	General-purpose Redis client for sessions, caching, and pub/sub
`xgen_sdk.logging`	`BackendLogger` for structured, DB-persisted application logs
`xgen_sdk.quota`	Pure-Python quota policy specs and evaluation (no DB / HTTP coupling)
`xgen_sdk.notification`	Generic per-user persistent in-app notifications with read tracking
`xgen_sdk.llm_catalog`	Dynamic model list for OpenAI / Anthropic / Gemini with TTL cache and fallback
`xgen_sdk.comfyui`	ComfyUI API-workflow rules (parameter mapping, validation, tool schema) and an async client
`xgen_sdk.control_policy`	PII and forbidden-word policies: scope switches, masking rules, file byte masking, policy event log
`xgen_sdk.XgenApp`	One-call bootstrap that wires DB + Config + Storage together

More general-purpose utilities are added with every release — tracing, retry strategies, schema migration, agent and RAG primitives, prompt templating, async task patterns. If it's reusable, it belongs here.

Installation

pip install xgen-sdk

Requires Python 3.11+. That's it — every module is installed and ready to import. No extras, no flags, no surprises.

Quick start

The fastest way to use xgen-sdk inside a FastAPI service:

from contextlib import asynccontextmanager
from fastapi import FastAPI
from xgen_sdk import XgenApp

xgen = XgenApp()

@asynccontextmanager
async def lifespan(app: FastAPI):
    xgen.boot()           # Connect DB, Config, Storage
    app.state.xgen = xgen
    yield
    xgen.shutdown()       # Cleanly release all resources

app = FastAPI(lifespan=lifespan)

@app.get("/users/{user_id}")
async def get_user(user_id: int):
    return xgen.db.find_by_id_table("users", user_id)

XgenApp.boot() reads its configuration from environment variables (see Environment below) and initializes:

PostgreSQL — psycopg3 connection pool via xgen.db
Config — RedisConfigManager (or local-file fallback) via xgen.config
Storage — MinIO client via xgen.minio_client

Each subsystem can be disabled independently:

xgen = XgenApp(enable_db=True, enable_config=True, enable_storage=False)

You're not required to use XgenApp — every module is fully usable on its own. XgenApp is the convenient default for "I want a FastAPI service with the standard infrastructure stack."

Modules

Database — `xgen_sdk.db`

A psycopg3-based connection pool with both model-class and table-name APIs.

from xgen_sdk.db import XgenDB, database_config

db = XgenDB(database_config)
db.initialize_connection()

# Model-based CRUD
record_id = db.insert(user_model)
records = db.find_by_condition(UserModel, {"status": "active"}, limit=100)

# Table-name CRUD (no model class required)
db.insert_record("users", {"name": "Ada", "email": "ada@example.com"})
rows = db.find_records_by_condition("users", {"is_active": True})

# Raw SQL with parameter binding
rows = db.execute_raw_query(
    "SELECT id, name FROM users WHERE created_at > %s", (cutoff,)
)

Configuration — `xgen_sdk.config`

A two-tier configuration system: Redis is the source of truth, with local-file fallback when Redis is unavailable. Typed config is declared with BaseConfig and assembled by ConfigComposer.

from xgen_sdk.config import create_config_manager, get_config_value

config = create_config_manager(db_manager=db)
api_key = get_config_value("OPENAI_API_KEY")

Storage — `xgen_sdk.storage`

A MinIO client that covers the full surface area used across XGen services: upload, download, existence checks, recursive listing, copying, and presigned URL generation.

from xgen_sdk.storage import (
    upload_file, download_file, get_presigned_url, list_files_in_path,
)

upload_file("documents", "reports/q4.pdf", "/tmp/q4.pdf")
url = get_presigned_url("documents", "reports/q4.pdf", expires=3600)

Auth — `xgen_sdk.auth`

ABAC-style permission system with FastAPI integration. Permissions support wildcards (admin.*:*, workflow:*) and superusers bypass all checks. The decorator pattern doubles as a self-registering permission catalog — the act of declaring an endpoint is the act of declaring its permission.

from fastapi import Depends, Request
from xgen_sdk.auth import require_perm, require_any_perm, require_superuser

@router.get("/roles")
async def list_roles(
    request: Request,
    session=Depends(require_perm("admin.role:read", description="List roles")),
):
    ...

@router.delete("/system/reset")
async def reset(_=Depends(require_superuser())):
    ...

At startup, call validate_and_sync(db) to push all decorator-declared permissions into the database — the frontend then reads them back to build its role-assignment UI. No more drift between code and DB.

Logging — `xgen_sdk.logging`

A BackendLogger that writes structured logs to the application database, tied to the current user and request.

from xgen_sdk.logging import create_logger

log = create_logger(db, user_id=session["user_id"], request=request)
log.info("Workflow started", metadata={"workflow_id": wf_id})
log.success("Workflow finished")
log.error("Step failed", exception=exc)

Quota — `xgen_sdk.quota`

Pure-Python policy evaluation for token quotas. No DB, no HTTP — just dataclasses and functions, so it can be imported safely from any service.

from xgen_sdk.quota import (
    resolve_effective_policies, evaluate_quota, period_bounds_kst,
)

effective = resolve_effective_policies(user_id, role_ids, policies)
decision = evaluate_quota(effective, usage_by_period)
if not decision.allowed:
    raise HTTPException(429, detail=decision.reason)

Notifications — `xgen_sdk.notification`

Generic per-user, persistent, in-app notifications. Failures in the notification path never break the caller's main flow.

from xgen_sdk.notification import (
    NotificationPayload, NotificationCategory, NotificationSeverity, publish,
)

publish(db, NotificationPayload(
    user_id=123,
    category=NotificationCategory.QUOTA,
    severity=NotificationSeverity.ERROR,
    title="Token quota exceeded",
    body="You have exceeded your monthly limit of 1,000,000 tokens.",
    link="/mypage?tab=quota",
    metadata={"policy_id": 7, "used": 1234567, "limit": 1000000},
))

LLM Catalog — `xgen_sdk.llm_catalog`

Fetches the live list of available chat / vision models from OpenAI, Anthropic, and Gemini, with an in-process TTL cache and a hardcoded fallback so UI dropdowns never go empty.

from xgen_sdk.llm_catalog import get_models, invalidate

models = get_models(provider="openai", capability="chat")
# [{"id": "gpt-4o", "label": "GPT-4o", ...}, ...]

invalidate("openai")   # Call after rotating the API key

ComfyUI — `xgen_sdk.comfyui`

One source of truth for turning a ComfyUI API-format workflow into an agent tool: which inputs are exposed as parameters, how they are validated, the tool's JSON Schema, and applying arguments. The async client (httpx, trust_env=False, no redirects) submits, polls history, downloads outputs, and on cancellation or timeout removes the job from the queue and interrupts it.

from xgen_sdk.comfyui import (
    parse_workflow, validate_entry, tool_input_schema, apply_arguments,
    ComfyUIClient, run_workflow,
)

graph = parse_workflow(api_export_json)            # rejects UI-format JSON
issues = validate_entry(graph, mapping, object_info)  # [] means valid
schema = tool_input_schema(mapping)
prompt = apply_arguments(graph, mapping, {"prompt": "a red fox"})

async with ComfyUIClient("http://comfy:8188") as client:
    result = await run_workflow(client, prompt, output_node_ids=mapping["outputs"], timeout_s=300)
    # {"prompt_id": ..., "duration_ms": ..., "files": [{"node_id", "filename", "content_type", "data"}]}

Control policy: `xgen_sdk.control_policy`

One definition of the admin PII and forbidden-word policies for every service. Each policy kind (PII, forbidden words) has a detection switch and four scope switches: block chat input, mask what goes to the model, strip chat uploads, strip file storage uploads. A scope is active only when both switches are on.

from xgen_sdk import control_policy as cp

rules = cp.load_scope_rules(app_db, cp.SCOPE_CHAT_UPLOAD, workflow_id=wf_id)  # per-agent exceptions removed
found = cp.apply_rules(text, rules, mask=False)                               # detect only
masked = cp.apply_rules(text, rules).text                                     # full or partial masks
result = cp.strip_file(tmp_path, "report.docx", rules)                        # txt, html, docx/xlsx/pptx, pdf
if result.changed:
    size, sha256 = result.size, result.sha256
cp.log_result(app_db, result.found, source=cp.SOURCE_CHAT_UPLOAD, workflow_id=wf_id)

PDF needs xgen-pdf at runtime (and chardet helps guess legacy text encodings). Both are imported lazily, so services that never mask files do not need them. pdfium is not thread safe, so PDF work runs in small worker processes that import only this SDK, never the service's own main module.

Variable	Default	Meaning
`XGEN_FILE_MASKING_MAX_BYTES`	32 MiB	Largest file read into memory; larger files are left as they are (`unmaskable`)
`XGEN_FILE_MASKING_MAX_UNZIPPED_BYTES`	256 MiB	Largest total unzipped size of an Office file
`XGEN_FILE_MASKING_PDF_WORKERS`	2	PDF worker processes; `0` runs PDF work in process (tests only)
`XGEN_FILE_MASKING_PDF_TIMEOUT_S`	300	Time limit for one PDF; the worker is killed and the file is `unmaskable`

Environment variables

XgenApp.boot() reads these from the environment. All have sensible defaults for local development.

PostgreSQL

Variable	Default	Description
`POSTGRES_HOST`	`localhost`	DB host
`POSTGRES_PORT`	`5432`	DB port
`POSTGRES_DB`	`xgen`	Database name
`POSTGRES_USER`	`postgres`	DB user
`POSTGRES_PASSWORD`	`postgres`	DB password

Redis (optional — falls back to local file)

Variable	Default	Description
`REDIS_HOST`	`localhost`	Redis host
`REDIS_PORT`	`6379`	Redis port
`REDIS_PASSWORD`	(empty)	Redis password

MinIO

Variable	Default	Description
`MINIO_ENDPOINT`	`http://minio:9000`	MinIO endpoint URL
`MINIO_ROOT_USER`	`minioadmin`	Access key
`MINIO_ROOT_PASSWORD`	`minioadmin`	Secret key

Where it fits

        ┌────────────────────────────────────────────────────┐
        │                  Your application                  │
        │   (xgen-core / xgen-workflow / xgen-documents /    │
        │    your own agent / RAG / API service)             │
        │                                                    │
        │              ──── business logic only ────         │
        └─────────────────────────┬──────────────────────────┘
                                  │
                                  ▼
        ┌────────────────────────────────────────────────────┐
        │                    xgen-sdk                        │
        │                                                    │
        │   db │ config │ storage │ auth │ redis │ logging   │
        │   quota │ notification │ llm_catalog               │
        │                    │ XgenApp │                     │
        │              (+ more in every release)             │
        └─────────────────────────┬──────────────────────────┘
                                  │
                                  ▼
        ┌────────────────────────────────────────────────────┐
        │   PostgreSQL │ Redis │ MinIO │ Gateway │ LLM APIs  │
        └────────────────────────────────────────────────────┘

Every XGen service speaks directly to the underlying infrastructure through the same SDK code path — no internal HTTP proxies, no duplicated client code. The SDK is the only place those concerns live.

Compatibility

Python: 3.11, 3.12
PostgreSQL: 13+
Redis: 5+
MinIO: any S3-API-compatible release

Runtime dependencies

psycopg[binary], psycopg-pool, redis, minio, pycryptodome, httpx, pydantic, fastapi. All installed automatically with pip install xgen-sdk.

Versioning

xgen-sdk follows Semantic Versioning. Pin services with a major-version range to receive bug fixes and additive features automatically:

dependencies = ["xgen-sdk>=2.0,<3.0"]

License

Proprietary — maintained by PlateerLab as the foundation of the XGen platform.

Metadata

Release files for xgen-sdk 2.9.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for xgen-sdk 2.9.0
File	Size	Uploaded
xgen_sdk-2.9.0.tar.gz	647.5 kB	Oct 13, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for xgen-sdk 2.9.0
File	Interpreter	ABI	Platform	Reset
xgen_sdk-2.9.0-py3-none-any.whl 570.2 kB Oct 13, 2026	Python 3	none	any	Details

Total release size: 1.2 MB

Release files / xgen_sdk-2.9.0.tar.gz

Download URL	xgen_sdk-2.9.0.tar.gz
Size	647.5 kB
Tags	Source
SHA-256 checksum How to use checksums	`98c002bba95b53e7bada3d9f7c772e4b62526d856aa518f1be878e0c3140c6bf`
BLAKE2b-256 checksum How to use checksums	`a66751666a16aba84f3a0575cbe8d306aa424fb1213822fd73149df1b2606e2c`
Upload date	Oct 13, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/PlateerLab/xgen-sdk
Publishing commit: github.com/PlateerLab/xgen-sdk/tree/0ad678c6318a6b20c60eed97c832d5d2227a8751

Workflow

Publishing configuration: github.com/PlateerLab/xgen-sdk/blob/0ad678c6318a6b20c60eed97c832d5d2227a8751/.github/workflows/publish-pypi.yml
Publishing logs: github.com/PlateerLab/xgen-sdk/actions/runs/38336469522/attempts/1

Artifact

Subject: xgen_sdk-2.9.0.tar.gz
SHA-256 checksum: 98c002bba95b53e7bada3d9f7c772e4b62526d856aa518f1be878e0c3140c6bf Verifying attestations

Release files / xgen_sdk-2.9.0-py3-none-any.whl

Download URL	xgen_sdk-2.9.0-py3-none-any.whl
Size	570.2 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`7cf703445dc2822144ae6e54e4b427e50b298c3740ce481815c474fe7c5b32a3`
BLAKE2b-256 checksum How to use checksums	`41be8853c4e67db81e9912e2e413ad55d675916f229a646ab7d1e8454616afd2`
Upload date	Oct 13, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/PlateerLab/xgen-sdk
Publishing commit: github.com/PlateerLab/xgen-sdk/tree/0ad678c6318a6b20c60eed97c832d5d2227a8751

Workflow

Publishing configuration: github.com/PlateerLab/xgen-sdk/blob/0ad678c6318a6b20c60eed97c832d5d2227a8751/.github/workflows/publish-pypi.yml
Publishing logs: github.com/PlateerLab/xgen-sdk/actions/runs/38336469522/attempts/1

Artifact

Subject: xgen_sdk-2.9.0-py3-none-any.whl
SHA-256 checksum: 7cf703445dc2822144ae6e54e4b427e50b298c3740ce481815c474fe7c5b32a3 Verifying attestations