synthetic-dataset-sdk 0.1.0

A lightweight Python client for the Synthetic Dataset Generation SaaS API

Synthetic Dataset SDK

A lightweight Python client for the Synthetic Dataset Generation API. This SDK allows you to easily ingest document pages and retrieve generated synthetic datasets.

Installation

pip install synthetic-dataset-sdk

Quick Start

from synthetic_dataset_sdk import SyntheticDatasetClient

# 1. Initialize the client
sdk = SyntheticDatasetClient(
    api_key="your-api-key",
    base_url="http://your-backend-url:8000"
)

# 2. Upload document pages
# pages should be a list of dicts: [{"page_no": 1, "doc_id": "DOC-A", "text": "..."}]
pages = [
    {
        "page_no": 1, 
        "doc_id": "DOC-A", 
        "text": "The quick brown fox jumps over the lazy dog."
    }
]
result = sdk.upload(pages)
print(f"Ingest status: {result['job_status']}")

# 3. Check batch formation status
status = sdk.get_batch_status()
print(f"Total batches: {status['total_batches']}")

# 4. Fetch the generated dataset
# This will return validated QA pairs once the pipeline is done
dataset = sdk.get_dataset(include_faulty=False)
for entry in dataset["entries"]:
    print(f"Q: {entry['question']}")
    print(f"A: {entry['answer']}")

Features

• Easy Ingestion: Upload raw text pages directly from your pipeline.
• Batch Tracking: Monitor the progress of batch formation and QA generation.
• Faulty Filtering: Toggle between high-accuracy pairs and the full generated set.
• Lightweight: Minimal dependencies (just requests).

Requirements

• Python 3.8+
• requests