CaBSALLM

Efficient, context-aware LLM annotation for conversational and tabular text data. CaBSALLM lets researchers configure a dataset, text and group columns, annotation prompt, provider/model, batch controller, and optimization strategy without editing pipeline code.

It supports resumable, structured annotation for CSV, TSV, Excel, JSON, and JSONL inputs, with provider presets for OpenAI, Claude, Gemini, Qwen, Kimi, and OpenAI-compatible endpoints.

Install

pip install cabsallm

Install only the provider extras you need:

pip install "cabsallm[anthropic]"
pip install "cabsallm[gemini]"
pip install "cabsallm[all-providers]"

The command-line tools are cabsallm and cuebatch. The Python import is codeedit_cues.

Quick start

1. Inspect a dataset and create a configuration template:

   cabsallm inspect messages.csv
   cabsallm init cabsallm.yaml
   

2. Put the provider credential in a local .env file beside the configuration. Never commit this file.

   OPENAI_API_KEY=<set-this-locally>
   

3. Edit cabsallm.yaml to identify the data, columns, prompt, model, and output path. Run a no-cost validation first, then a small pilot:

   cabsallm run cabsallm.yaml --dry-run
   cabsallm run cabsallm.yaml --max-rows 50
   

4. Run the full job after checking the pilot output:

   cabsallm run cabsallm.yaml
   

CaBSALLM writes annotations, a resumable progress state, run events, and a run summary to the configured output directory.

Configuration

This minimal configuration annotates a text column while preserving each row's identifier and conversation/group context:

input_path: data/messages.csv
output_dir: runs/my-study

columns:
  id: message_id
  text: message_text
  group: conversation_id

output:
  label_column: annotation
  rationale_column: annotation_rationale

prompt:
  system: prompts/system.txt
  user_template: prompts/user.txt

llm:
  provider: openai
  model: gpt-5.4
  api_key_env: OPENAI_API_KEY
  reasoning_effort: medium
  temperature: 0.0

batch:
  strategy: hybrid
  initial_size: 8
  min_size: 1
  max_size: 32

optimization:
  enabled: false

Prompts should request structured JSON and explicitly require the original row identifiers. CaBSALLM rejects malformed responses and records recoverable failures rather than silently assigning labels to the wrong rows.

Providers and credentials

Credentials may be supplied through an environment variable, a local .env file, or application code. Keep keys out of YAML, prompts, notebooks, outputs, and version control.

Provider preset	Example model	Default key variable	Notes
openai	gpt-5.4	OPENAI_API_KEY	Supports reasoning_effort.
anthropic	claude-sonnet-4-5	ANTHROPIC_API_KEY	Install cabsallm[anthropic].
gemini	gemini-2.5-pro	GEMINI_API_KEY	Install cabsallm[gemini]; supports thinking_budget.
qwen	qwen-plus	DASHSCOPE_API_KEY	Use its OpenAI-compatible endpoint when required.
kimi	moonshot-v1-8k	MOONSHOT_API_KEY	Use its OpenAI-compatible endpoint when required.
openai_compatible	provider-specific	your choice	Set base_url and api_key_env.

Example for a Qwen-compatible endpoint:

llm:
  provider: qwen
  model: qwen-plus
  api_key_env: DASHSCOPE_API_KEY
  base_url: https://dashscope.aliyuncs.com/compatible-mode/v1

Use reasoning_effort: low, medium, or high when supported by the selected model. For Gemini, use thinking_budget when that model supports a controllable thinking budget. Provider-specific controls are forwarded only when compatible with the selected preset.

Batch control and comfortable progress reporting

Choose the controller that fits the reliability and cost profile of the task:

Strategy	Use it when
fixed	You need a constant, known batch size.
aimd	You want conservative additive growth and quick backoff after failures.
ewma	You want batch size to follow a smoothed latency/error signal.
hybrid	You want adaptive control with safety limits; this is the usual default.

Make long runs easier to supervise:

reporting:
  verbosity: detailed       # quiet, normal, or detailed
  show_eta: true
  show_batch_metrics: true
  show_token_estimates: true
  show_controller_updates: true
  show_error_details: true
  write_events: true
  write_partials: true
  update_every_batches: 1

Set verbosity: quiet for unattended runs, normal for concise checkpoints, or detailed to include batch sizes, controller decisions, latency, errors, estimates, and ETA.

Tune the batch controller

BOHB is the default tuning method. Tuning always requires an explicit list of hyperparameters, so the search space is visible and reproducible. Other available methods are successive_halving, random, grid, and greedy.

optimization:
  enabled: true
  method: bohb
  objective: cost_adjusted_throughput
  budget: 24
  hyperparameters:
    - name: batch.initial_size
      type: int
      min: 2
      max: 16
    - name: batch.max_size
      type: int
      min: 16
      max: 64
    - name: batch.strategy
      type: categorical
      values: [aimd, ewma, hybrid]

Run tuning with:

cabsallm tune cabsallm.yaml

The selected configuration, trial history, metrics, and recommendation are saved beneath the tuning output directory. Review the recommendation before using it for a full production run.

Python API and command examples

Run an annotation project from Python:

from codeedit_cues import AnnotationRunner, load_config

config = load_config("cabsallm.yaml")
runner = AnnotationRunner(config)
summary = runner.run(max_rows=50)
print(summary)

Run tuning from Python:

from codeedit_cues import load_config
from codeedit_cues.tuning import Tuner

config = load_config("cabsallm.yaml")
result = Tuner(config).tune()
print(result.best_config)

Runnable scripts are included in the source distribution under examples/scripts/:

python examples/scripts/create_project.py
python examples/scripts/inspect_data.py data/messages.csv
python examples/scripts/dry_run.py cabsallm.yaml
python examples/scripts/run_annotation.py cabsallm.yaml --max-rows 50
python examples/scripts/tune.py cabsallm.yaml
python examples/scripts/cite_paper.py --style bibtex

Use cabsallm --help or cabsallm <command> --help for every command and option.

Citation

CaBSALLM is built on:

Abolhasani Mohammadsadegh, Reza Mousavi, and Paul Jen-Hwa Hu. 2026. CaBSALLM: Efficient Context-Aware Batch Annotation of Conversational Streams with Large Language Models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 615–636. Association for Computational Linguistics. https://doi.org/10.18653/v1/2026.acl-short.51

Print a ready-to-use citation from the command line:

cabsallm cite --style acl
cabsallm cite --style bibtex
cabsallm cite --style markdown
cabsallm cite --style doi

Or from Python:

from codeedit_cues import citation

print(citation("bibtex"))

Read the paper at the ACL Anthology.

Responsible use

Pilot prompts and models before a full run. Inspect samples for systematic errors, use an appropriate human-review process for consequential labels, and follow the data-use, privacy, and provider requirements that apply to your study.