lightassay 0.3.1

File-based orchestrator for structured evaluation of applied LLM workflows: humans declare intent, LLMs reason about quality, code runs and records raw facts

lightassay

lightassay is a simple first way to test an LLM workflow.

• You describe what worries you in plain language.
• Your agent, using the LLM access you already have, helps turn that into directions, test cases, and analysis.
• You do not need to build a formal eval system first.
• The code runs the workflow and records raw facts.
• The results are analyzed in terms that make sense to you.

Install

Inside an activated virtual environment:

python -m venv .venv && source .venv/bin/activate
pip install lightassay

Or install the CLI with pipx:

pipx install lightassay

First-time setup (required once)

Run:

lightassay init

This is the mandatory first-run step after installation.

Quick start

Once init is done:

lightassay quickstart \
  --message "Check myapp.pipeline.run. I care about obvious mistakes, over-correction, and preserving names and numbers." \
  --target "myapp.pipeline.run"

This creates the workbook, runs the first pass, and writes the analysis artifact.

Follow-up continue

lightassay continue --compare-previous

Use this after editing the workbook or adding a follow-up --message.

Manual workbook path

If you want an empty workbook first:

lightassay workbook

For the explicit stage-by-stage flow, see docs/quickstart.md. For a runnable example, see examples/quickstart/.

---

Documentation

• quickstart.md — explicit stage-by-stage flow
• workbook_spec.md — workbook structure
• workflow_config_spec.md — workflow execution config
• semantic_adapter_spec.md — analysis and compare config
• code_architecture.md — code structure

---

License

MIT — see LICENSE.