krira-embed 0.1.0

Modal-powered embedding pipeline for Krira chunks with Pinecone upsert.

krira-embed

krira-embed provides a Modal-powered embedding pipeline for chunked text and upserts vectors to Pinecone.

Features

• Batch chunk ingestion from JSONL (text, optional metadata, optional id)
• Distributed embedding jobs with Modal
• Pinecone upsert with deterministic ID fallback
• Simple Python client API: KriraEmbedding

Requirements

• Python >=3.10,<3.13
• Modal account and token (MODAL_TOKEN_ID, MODAL_TOKEN_SECRET)
• Pinecone API key (PINECONE_API_KEY)
• Chunked JSONL file (for example, chunks.jsonl)

Installation

pip install krira-embed

Quickstart

from krira_embed import KriraEmbedding

client = KriraEmbedding(
    chunk_file_path="chunks.jsonl",
    pinecone_api_key="YOUR_PINECONE_API_KEY",
    pinecone_index_name="YOUR_INDEX_NAME",
    namespace="default",
)

result = client.embed(
    worker_batch_size=12000,
    parallel_jobs=6,
    model_batch_size=768,
    upsert_batch_size=200,
)

print(result)

Credentials model

• End users provide Pinecone credentials explicitly in code (pinecone_api_key, pinecone_index_name).
• The package does not read local .env for Pinecone credentials.
• Modal credentials can still be supplied via environment variables (MODAL_TOKEN_ID, MODAL_TOKEN_SECRET) or existing Modal auth setup.

Modal CLI usage

When using the Modal entrypoint directly, pass both values explicitly:

modal run main.py --index-name YOUR_INDEX_NAME --pinecone-api-key YOUR_PINECONE_API_KEY

Use .env only for your local Modal tokens if needed (see .env.example).

Local validation (maintainers)

python -m pip install --upgrade build twine
python -m build
python -m twine check dist/*

License

MIT License. See LICENSE.