datalier 0.1.0

Datalier by Concave AI - data infrastructure for AI model training

Datalier — by Concave AI

Data infrastructure for AI model training. One SDK — raw data in, quality-assured training data out.

Installation

pip install datalier

Quick Start

from datalier import DatalierClient

client = DatalierClient(
    api_key="sk_concave_...",
    base_url="https://api.theconcaveai.com",  # or http://localhost:8000 for local
)

# Upload a dataset
dataset = client.upload("training_pairs.jsonl", dataset_type="rlhf")
print(f"Uploaded: {dataset.id} — {dataset.row_count} rows")

# Transform (validate, deduplicate, detect PII)
result = client.transform(dataset.id, steps=["validate", "dedup", "pii_scan"])
print(f"PII found: {result['pii_detection']['total_pii_found']}")

# Label with RLAIF (AI handles 80-90%, humans review edge cases)
result = client.label(dataset.id, dataset_type="rlhf", min_kappa=0.70)
print(f"Auto-labeled: {result['ai_labeled']}/{result['total_tasks']}")

# Check quality metrics
quality = client.get_quality(dataset.id)
print(f"Kappa: {quality.kappa} | Gold Accuracy: {quality.gold_accuracy}")

# Approve and version
client.approve(dataset.id)
version = client.snapshot(dataset.id)
print(f"Snapshot: {version['version']} — hash: {version['snapshot_hash'][:8]}...")

# Export as DPO format for training
download_url = client.export(dataset.id, fmt="dpo", version="v1.0")

# Monitor model performance (Layer 5)
model = client.register_model("my_model_v1", trained_on_dataset_id=dataset.id, trained_on_version="v1.0")
client.submit_metrics(model["model_id"], accuracy=0.89, f1=0.85)
drift = client.get_drift(model["model_id"])

Platform Layers

Layer	Function	SDK Methods
1 - Ingest	Upload from any source	upload(), list_datasets(), get_dataset()
2 - Prepare	Validate, dedup, PII scan	transform(), get_profile(), get_pii_report(), redact()
3 - Label	RLAIF + human review	label(), get_quality(), approve()
4 - Version	Snapshots + lineage	snapshot(), list_versions(), get_lineage(), rollback(), export()
5 - Observe	Monitor + re-label loop	register_model(), submit_metrics(), get_drift(), trigger_relabel()

Using with Local Dev Server

client = DatalierClient(
    api_key="your-jwt-token",
    base_url="http://localhost:8000",
)

Links

• Website: theconcaveai.com
• Documentation: theconcaveai.com/docs

License

MIT — Concave AI 2026