sxd-sdk 0.3.0

SDK for building and deploying data pipelines on SentientX Data Platform

SXD SDK

Python library for building data pipelines on SentientX Data Platform.

Installation

pip install sxd-sdk

Quick Start

1. Create Pipeline

Create a new directory with this structure:

my-pipeline/
├── my_pipeline/
│   ├── __init__.py
│   └── activities.py
├── sxd.yaml
└── pyproject.toml

2. Define Tasks

# my_pipeline/activities.py
from sxd_sdk import sxd_task, TaskContext, ProcessingResult

@sxd_task(name="process_video")
async def process_video(input_data: dict, ctx: TaskContext) -> dict:
    video_path = input_data["source_url"]
    await ctx.heartbeat(f"Processing {video_path}")
    result = ProcessingResult.ok(output_path="/output/result.mp4")
    return result.model_dump()

3. Configure (sxd.yaml)

base_image: sxd-base    # or: sxd-pytorch, sxd-opencv, sxd-cuda
timeout: 3600           # optional, default 3600s
gpu: false              # optional, default false

4. Publish

sxd publish .

5. Submit Tasks

sxd submit process_video --input '{"source_url": "s3://bucket/video.mp4"}'

API Reference

Schemas

from sxd_sdk import (
    PipelineInput,    # Base input with source_url, customer_id, options
    PipelineOutput,   # Base output with status, output_path, metrics, errors
    ProcessingResult, # Task result with success, output_path, error, metadata
)

# Convenience constructors
result = ProcessingResult.ok("/output/path", count=10)
result = ProcessingResult.fail("Something went wrong")

Testing

from sxd_sdk.testing import MockTaskContext, MockControlPlane

# Test tasks
async def test_my_task():
    ctx = MockTaskContext()
    result = await process_video({"source_url": "s3://video.mp4"}, ctx)
    assert result["success"]
    assert ctx._heartbeats  # verify heartbeat was called

Client

from sxd_sdk import SXDClient

client = SXDClient(api_url="https://api.example.com", api_key="...")
task = await client.submit("process_video", {"source_url": "s3://video.mp4"})
status = await client.status(task["id"])

License

Apache 2.0