bulk-chain 0.25.1

A lightweight, no-strings-attached Chain-of-Thought framework for your LLM, ensuring reliable results for bulk input requests.

bulk-chain 0.25.1

Third-party providers hosting↗️

A no-strings-attached framework for your LLM that allows applying Chain-of-Thought-alike prompt schema towards a massive textual collections using custom third-party providers ↗️.

Main Features

• ✅ No-strings: you're free to LLM dependencies and flexible venv customization.
• ✅ Support schemas descriptions for Chain-of-Thought concept.
• ✅ Provides iterator over infinite amount of input contexts served in CSV/JSONL.

Extra Features

• ✅ Progress caching [for remote LLMs]: withstanding exception during LLM calls by using sqlite3 engine for caching LLM answers;

Installation

From PyPI:

pip install --no-deps bulk-chain

or latest version from here:

pip install git+https://github.com/nicolay-r/bulk-chain@master

Chain-of-Thought Schema

To declare Chain-of-Though (CoT) schema, this project exploits JSON format. This format adopts name field for declaring a name and schema is a list of CoT instructions for the Large Language Model.

Each step represents a dictionary with prompt and out keys that corresponds to the input prompt and output variable name respectively. All the variable names are expected to be mentioned in {}.

Below, is an example on how to declare your own schema:

{
"name": "schema-name",
"schema": [
    {"prompt": "Given the question '{text}', let's think step-by-step.", 
     "out": "steps"},
    {"prompt": "For the question '{text}' the reasoining steps are '{steps}'. what would be an answer?", 
     "out":  "answer"},
]
}

Usage

Preliminary steps:

1. Define your schema (Example for Sentiment Analysis))
2. Wrap or pick LLM model from the Third-party providers hosting↗️.

API

Please take a look at the related Wiki page

Shell

NOTE: You have to install source-iter and tqdm packages that actual dependencies of this project

1. ⬇️ Download replicate provider for bulk-chain:

wget https://raw.githubusercontent.com/nicolay-r/nlp-thirdgate/refs/heads/master/llm/replicate_104.py

2. 📜 Setup your reasoning schema.json according to the following example ↗️
3. 🚀 Launch inference using DeepSeek-R1:

python3 -m bulk_chain.infer \
    --src "<PATH-TO-YOUR-CSV-or-JSONL>" \
    --schema "test/schema/default.json" \
    --adapter "replicate_104.py:Replicate" \
    %%m \
    --model_name "deepseek-ai/deepseek-r1" \
    --api_token "<REPLICATE-API-TOKEN>"

Or, you can launch demo mode to interact with LLM via command line:

NOTE: Demo supports streaming!

python3 -m bulk_chain.demo \
    --schema "test/schema/thor_cot_schema.json" \
    --adapter "dynamic:replicate_104.py:Replicate" \
    %%m \
    --model_name "meta/meta-llama-3-70b-instruct" \
    --api_token "<REPLICATE-API-TOKEN>" \
    --stream

Embed your LLM

All you have to do is to implement BaseLM class, that includes:

• __init__ -- for setting up batching mode support and (optional) model name;
• ask(prompt) -- infer your model with the given prompt.

See examples with models at nlp-thirdgate 🌌.