decompressed-sdk 0.4.0

Decompressed public SDK (Python)

decompressed-sdk

Python SDK for Decompressed — chunk, embed, and version your vector data using strategies validated in the RAG Lab.

Install

pip install decompressed-sdk

Quick start

from decompressed_sdk import DecompressedClient

dc = DecompressedClient(api_key="dck_your_key_here")

---

RAG Lab — client.lab

Embed text and build production pipelines using strategies you tested in the RAG Lab. Every call applies the same chunking method, chunk size, overlap, and embedding model you validated, so what you benchmarked is exactly what runs in production.

Embed pre-chunked texts

Use when you handle chunking yourself and just need embeddings.

result = dc.lab.embed(
    texts=["Document 1", "Document 2"],
    preset_id="balanced",       # or strategy="My Strategy" / strategy_id="uuid"
)

print(result.embeddings[0][:5])     # first 5 floats
print(result.model)                  # text-embedding-3-large
print(result.dimensions)             # 3072
print(result.usage["token_count"])

Built-in presets

ID	Name	Model	Dims	Search
ghost	Economy	text-embedding-3-small	256	vector
balanced	Balanced	text-embedding-3-large	3072	vector
scholar	High Accuracy	text-embedding-3-large	3072	hybrid + rerank
hybrid	Hybrid Search	gte-large	1024	hybrid + rerank

Supported embedding models (use any via chunk_and_embed or custom strategies in the Lab)

Model	Provider	Dims	Cost/1M tokens	Best for
text-embedding-3-small	OpenAI	1536	$0.02	Cost-effective, general use
text-embedding-3-large	OpenAI	3072	$0.13	Higher accuracy, complex queries
pplx-embed-v1-4b	Perplexity	2560	$0.03	Strong multilingual and reasoning
pplx-embed-v1-0.6b	Perplexity	1024	$0.004	Ultra-low cost, high throughput
gte-large	Thenlper	1024	$0.01	Open-source, solid retrieval quality
e5-large-v2	Intfloat	1024	$0.01	Instruction-tuned retrieval

---

Chunk and embed a single document

Decompressed handles chunking for you using the strategy's config. Returns every chunk with its text, embedding, and metadata.

with open("handbook.pdf", "r") as f:
    text = f.read()

result = dc.lab.chunk_and_embed(
    text=text,
    source="handbook.pdf",   # optional — attached to every chunk's metadata
    preset_id="balanced",
)

print(f"{len(result.chunks)} chunks · {result.dimensions}d · {result.chunking_method}")
print(f"Cost: ${result.usage['billable_cost_usd']:.6f}")

# Upsert to your vector DB
index.upsert(vectors=result.to_pinecone(id_prefix="handbook"))           # Pinecone
qdrant.upsert("docs", [PointStruct(**p) for p in result.to_qdrant()])   # Qdrant
collection.insert(list(result.to_milvus().values()))                     # Milvus
records = result.to_records()                                            # plain dicts

ChunkAndEmbedResponse

Field	Type	Description
chunks	list[Chunk]	Every chunk with text, embedding, and metadata
model	str	Embedding model used
dimensions	int	Vector dimensionality
strategy_name	str	Resolved strategy or preset name
chunking_method	str	Chunking method applied
usage['chunk_count']	int	Number of chunks produced
usage['token_count']	int	Tokens consumed
usage['billable_cost_usd']	float	Model cost + 20% platform fee
usage['remaining_tokens']	int	Monthly quota remaining

Chunk fields

Field	Type	Description
chunk_index	int	Position within the document
text	str	Chunk text
embedding	list[float]	Embedding vector
metadata['source']	str	Source label you passed in
metadata['start_char']	int	Character offset start
metadata['end_char']	int	Character offset end
metadata['model']	str	Model used
metadata['recommended_retrieval']	dict	Suggested top_k, search_type, rerank for your vector DB

---

Chunk and embed many documents

Process a full corpus. Same strategy config applied to every document. All chunks collected in one flat response — upsert your entire corpus in a single call.

import os

docs = [
    {"text": open(f).read(), "source": f}
    for f in os.listdir("./corpus")
    if f.endswith(".txt")
]

result = dc.lab.chunk_and_embed_many(
    documents=docs,
    preset_id="balanced",    # or strategy="My Strategy" / strategy_id="uuid"
)

print(f"{result.document_count} docs → {len(result.chunks)} chunks")
print(f"Total cost: ${result.usage['billable_cost_usd']:.4f}")
print(f"Tokens remaining: {result.usage['remaining_tokens']}")

# Upsert entire corpus in one call
index.upsert(vectors=result.to_pinecone(id_prefix="corpus"))

# Each chunk carries source + position metadata
for chunk in result.chunks:
    print(chunk.metadata["source"], chunk.metadata["doc_chunk_index"], chunk.text[:60])

documents input format

Each dict in the list:

Key	Required	Description
text	Yes	Raw document text
source	No	Label for this document (filename, URL, ID)

ChunkAndEmbedManyResponse — additional fields

Field	Type	Description
document_count	int	Number of documents processed
chunks	list[Chunk]	All chunks, flat. chunk_index is globally unique across the corpus.
usage['chunk_count']	int	Total chunks across all documents
usage['token_count']	int	Total tokens across all documents
usage['billable_cost_usd']	float	Total cost across all documents
usage['remaining_tokens']	int	Quota after all documents processed
metadata['doc_chunk_index']	int	Per-document chunk position (on each Chunk)

Same output helpers: to_pinecone() · to_qdrant() · to_milvus() · to_records()

---

List strategies

available = dc.lab.list_strategies()

for preset in available["presets"]:
    print(f"{preset['id']}: {preset['name']} ({preset['model']})")

for s in available["saved_strategies"]:
    print(f"{s['name']} — used {s['usage_count']} times")

---

Strategy reference priority

For all lab methods: preset_id → strategy_id → strategy (name).

---

Datasets — client.datasets

datasets = client.datasets.list()