stress-csp-library 0.1.5

Extensible JSONL task editing library for code and formula inputs.

stress_csp_library

stress_csp_library processes JSON or JSONL task records, applies controlled line edits, runs edited variants through registered tools, evaluates tool outputs, and writes successful variants back as JSONL.

Features

• Dataclass-based record models with extra_options for forward-compatible schema changes.
• single_edit mode for independent edits from the original input.
• cumulative_edit mode for greedy accepted edits applied on top of prior edits.
• Tool registry with in-memory tools and Python-file tool adapters.
• Configurable label comparison strategies for label-preserving and label-variation workflows.
• Minimal examples with a mock tool and mock editor.

Install

cd stress_csp_library
python3 -m pip install -e .[dev]

Run

python3 -m stress_csp_library.cli \
  --input /path/to/tasks.jsonl \
  --output /path/to/successes.jsonl \
  --edit-mode single_edit \
  --number-of-edits 3 \
  --label-variation \
  --single-edit-ids 0,2,4

number_of_edits accepts either a positive integer or "all".

• In single_edit mode, the library evaluates independent edits from the original input and saves up to the requested maximum number of successful premise-level edits. Use --single-edit-ids or extra_options.selected_edit_ids to restrict which editable units are considered.
• In cumulative_edit mode, "all" means the runner walks the editable statements from first to last, tries the available operator edits for each statement against the label_variation condition, applies the first acceptable edit, and then moves to the next statement.
• --no-label-variation is the default behavior: only save records whose tool output matches the original label.
• --label-variation inverts that rule: only save records whose tool output differs from the original label.

For csp_solver.py, provide the tool name in each record or via CLI defaults and point extra_options.tool_search_paths at the folder containing the solver. The Python-file adapter can call plain functions such as run(input_text) and also supports the CSP_Program class shape found in the source files.

Example Record

{
  "id": "logical_deduction_0",
  "tool": "csp_solver.py",
  "input_type": "code",
  "edit_mode": "single_edit",
  "number_of_edits": 2,
  "label_variation": false,
  "expected_answer": "D",
  "logic_program": "Domain:\n...\nConstraints:\nThe blue book is to the right of the yellow book. ::: blue_book > yellow_book\n...\n\nQuery:\n..."
}

Test

python3 -m pytest