CIT Course Tools

cit-course-tools gives students one small command-line interface for preparing a course workspace, exploring deterministic model scenarios, testing cumulative Python harnesses, and producing consistent lab evidence. It is course-agnostic: the installed package provides the machinery, while the instructor-provided YAML file defines what a particular lab teaches and tests.

The goal is a simple student experience with rigorous behavior underneath. The package manages ports, simulator sessions, graph traversal, evidence, fractional scoring, and current-result reports. It does not download course scenario files, AI models, llama.cpp, or other external software.

Install

Python 3.10 or newer is required.

python -m pip install cit-course-tools

To update later in the semester:

python -m pip install --upgrade cit-course-tools

Start a course workspace

Navigate to the directory you want to use and initialize it once:

cit-course init .

The command creates the submitted orientation lab labs/lab00 and the regular labs/lab01 through labs/lab12. Every lab has logs/ for non-submitted debugging material and submission/ for the three files submitted together. It preserves student work and refreshes only the package-managed Lab 00 guide when needed.

Lab 00 contains interaction-guide.yaml. Download later labs' YAML files from the course site and place each one in its matching lab directory. The package does not generate or download course-specific lab suites.

The scaffold also creates a preserved project/project_harness.py starter. Labs that require a harness select it automatically. The file must export run_harness(context).

After initialization, start with:

cit-course specs

Review the resulting specs/hardware_specs.txt before sharing it.

Learn the simulator

Start the Model Simulator with labs/lab00/interaction-guide.yaml in one terminal. The bare command selects it automatically:

cit-course serve

Leave it running, then start the User Simulator in a second terminal:

cit-course run

The guide introduces the package and its simulator workflow without requiring a download. Explore its paths because the submitted Lab 00 evaluation asks questions answered by the guide.

When finished, run the Lab 00 suite and generate its submission files:

cit-course test

Explore a lab scenario

To explore an instructor-supplied lab instead, start its YAML file explicitly:

cit-course serve labs/lab01/lab01.yaml

Then run the User Simulator in a second terminal anywhere inside the same initialized workspace:

cit-course run

The package discovers the active lab automatically. The serve terminal shows model-service activity; the run terminal stays focused on predefined user prompts and Model Simulator responses. The Model Simulator uses port 8081 and the User Simulator uses port 8082 by default.

Run a lab test suite

Bare cit-course test selects Lab 00. For a later early lab that does not require a student harness:

cit-course test labs/lab01/lab01.yaml

For a cumulative lab, the default project harness is selected automatically:

cit-course test labs/lab03/lab03.yaml

The Python file must export one function:

def run_harness(context):
    """Run one complete simulator-test or local-model interaction session."""

An alternate Python filename can be selected explicitly with --harness. The full suite runs by default. During implementation, rerun only failed, errored, and incomplete steps with:

cit-course test labs/lab03/lab03.yaml \
  --continue

For an alternate harness file:

cit-course test labs/lab03/lab03.yaml --harness project/my_harness.py

Each selectable prompt-response step is worth no more than one point. If three of five required checks pass, that step earns 0.60 / 1.00. Infrastructure errors are reported clearly and excluded from grading.

Every run replaces the current generated report and evidence without touching the evaluation or another lab:

labs/lab03/
├── logs/
└── submission/
    ├── evaluation.md
    ├── evidence.jsonl
    └── test-summary.md

evaluation.md is created during initialization and never overwritten. Students complete it in their own words. The evidence trace and test summary are regenerated by the package. When ready, the student compresses and submits that one submission/ directory.

Run the project harness with a real local model

The same cumulative run_harness(context) implementation can run as an ordinary terminal application against llama-server or another OpenAI-compatible local service. Start the real model server first, normally at http://127.0.0.1:8080, and then run:

python project/project_harness.py

The standalone adapter discovers the served model through /v1/models, reads user prompts from the terminal, and supplies empty simulator-specific headers. Enter q at the User Prompt to stop. An alternate server root or advertised model ID can be selected explicitly:

python project/project_harness.py --model-url http://127.0.0.1:9000
python project/project_harness.py --model my-model-alias

Run python project/project_harness.py --help for all standalone options. The student-authored core remains the same: tests import run_harness(context) and supply deterministic User Simulator prompts, while direct execution supplies terminal prompts and the real local endpoint.

Hardware report

Create or refresh the optional, privacy-conscious hardware report from anywhere inside the workspace:

cit-course specs

It is written to specs/hardware_specs.txt. Review it before sharing it.

Commands

cit-course init     Create or repair the course workspace
cit-course specs    Generate the optional hardware report
cit-course serve    Start the Model Simulator with the guide or a lab YAML file
cit-course run      Start the User Simulator for the active lab
cit-course test     Run Lab 00 or an explicit lab suite and replace its report

Run cit-course --help or cit-course COMMAND --help for options.

Instructor and developer references

• TECHNICAL_SUMMARY.md provides a concise comprehensive architecture and workflow summary.
• SCENARIO_AUTHORING.md documents the YAML authoring and scoring contract.
• TECHNICAL_SPECIFICATION.md documents components, isolation, evidence, and runtime behavior.
• PUBLISHING.md contains the release checklist.

Run the complete regression suite with:

python -m unittest discover -s tests -v