wex-addon-process

Version: 3.0.1

wexample-wex-addon-process runs the treatments a board declares. A process says what to run and on which selection of files; a run is one execution of it, asked for by writing a record and picked up by a worker.

Nothing here talks to a database. A run names a process, the process names a selection and a type, the selection names files, and all four are records under the app's own .wex/data/ — which is what lets a process run on a repository that has no board at all, and what makes a demand survive a broker that falls over.

Running one by hand

wex process::run/execute --run 8020e9fb-05b8-4da7-857d-d3947449b6f7

The run record goes from pending to running to complete, gaining its dates, its counters and, at the end, whatever the type wrote in data. A run in any state but pending is left alone: a queue redelivers, and a run is done once.

Running as a worker

wex process::worker/start

Blocks, consuming the process_run queue, and rings back on process_run_event every time a run moves — on each state change, and at most four times a second while it advances. This is the process a worker container runs, and there may be as many as the parallelism asked for.

The types on offer

A type is named by the family that answers for it. The whole of the filestate family is read off filestate's own options — every option that can produce an operation is a treatment a process may name:

filestate:report                    says what the selection takes, changes nothing
filestate:mode                      permissions, and ownership with them
filestate:should_exist              create when missing, delete when forbidden
filestate:name                      rename to a required form
filestate:content  filestate:text   write, trim, sort, keep unique lines
filestate:yaml     filestate:structured_keys
filestate:should_contain_lines      and its `should_not_` twin
filestate:managed_blocks            filestate:class      filestate:on_bad_format

Nothing is written per treatment: an option added to filestate is a type available here the same day, under the very name filestate gives it.

The process's options object is the configuration handed to each file, and has to carry the option the type is named after:

// the options of a process of type filestate:mode
{"mode": "600", "action": "dry_run"}

action is read by the family and handed to nobody. It picks which of filestate's three verbs is asked, the same three the command line offers on an app's declared state:

• check, the default, says what is wrong with each selected file and changes nothing. Its data counts the files checked and failing, and lists under files only those at fault, each with its code, message, parameters and remedy, the name of the operation that would fix it or null when nothing can.
• dry_run lists the operations one pass would apply.
• fix applies them, then lists what is still wrong, the way check does.

A rule that only observes, such as max_lines, is a type like the others: it has no remedy, so a fix changes nothing and reports it again.

Writing a type

A type of your own is a name and a run():

class ReviewProcessType(AbstractProcessType):
    name = "agent:review"
    label = "Review"

    def run(self, context: ProcessContext) -> dict[str, Any]:
        for path in context.each():
            ...

        return {"files": len(context.files)}

context.each() hands the files over one at a time and keeps the run up to date: the file being worked on is the run's current, and it is counted once the loop comes back for the next one. A file whose treatment raises is never counted and stays current, so a failed run says where it failed.

What run() returns goes into the run's data. Raising is how a type says it could not go on — the failure is written down for it, so nothing in a type has to know what a failed run looks like.

A type that touches files should not be written this way: it belongs to the filestate family, where the operation, its description and its undo already exist.

Table of Contents

• Running one by hand
• Running as a worker
• The types on offer
• Writing a type
• Installation
• What a run may ask for itself
• Runs that last a night
• An item that fails, a run that is cancelled
• Tests
• Architecture
• Integration in the Suite
• Dependencies
• Versioning & Compatibility Policy
• License
• About us
• Known Limitations & Roadmap
• Status & Compatibility
• Useful Links
• Migration Notes

Installation

pip install wexample-wex-addon-process

Requires Python >=3.10.

A process and a selection are records, so a treatment can be declared without a board. Records are JSON, written and read by programs on both sides:

// .wex/data/selection/<uuid>.json
{
  "title": "Markdowns",
  "patterns": "*.md\n!.wex/**"
}

// .wex/data/process/<uuid>.json
{
  "title": "What the Markdowns take",
  "type": "filestate:report",
  "selection_id": "<the selection's uuid>",
  "options": {}
}

Asking for a run is writing a third record:

// .wex/data/process_run/<uuid>.json
{
  "process_id": "<the process's uuid>",
  "state": "pending"
}

Then:

wex process::run/execute --run <the run's uuid>

The same file is read back afterwards, holding what happened:

{
  "process_id": "...",
  "state": "complete",
  "items_total": 3,
  "items_done": 3,
  "data": {
    "files": 3,
    "bytes": 16,
    "by_extension": {".md": 3}
  }
}

What a run may ask for itself

A run record may carry three keys of its own, written when it is created:

• action — check, dry_run or fix, in place of the process's own, for this run.
• items — paths relative to the app, taken in place of the selection. A path that is gone is listed under items_missing rather than failing the run; a path outside the app fails it.
• commit: true — once the run completes, commit what it wrote and nothing else. The result replaces the key: {"sha": ..., "files": [...]}, {"sha": null, "files": []} when nothing changed, or {"error": ..., "files": [...]} when git refused — the changes stay either way.

{"process_id": "...", "state": "pending", "action": "fix", "items": ["docs/a.md"], "commit": true}

What a run wrote is what its type declared through context.touch(path): the operations of a filestate fix, the items an agent changed and their engrams.

Runs that last a night

A worker takes one message at a time from the queue and acknowledges it at once: the run's record, not the broker, says what became of it. The run is worked on in a thread of its own while the worker keeps the broker connection alive, so a run of several hours needs no RabbitMQ setting — neither consumer_timeout nor the heartbeat can cut it. A connection lost meanwhile is opened again once the run ends; the board is told of the run's moves on a best-effort basis and reads the record again anyway.

A running run writes date_alive into its record every minute. One still running whose last sign of life is too old was interrupted — worker killed, machine stopped — and is said failed, never run again: by the next run of the same app, and by any worker that served that app, within a minute of finding its queue empty.

Variable	Default	What it sets
PROCESS_RUN_STALE_AFTER	600	Seconds without a sign of life after which a running run is taken for dead
RABBITMQ_HEARTBEAT	the broker's	Seconds between two heartbeats on the worker's connection

There is no limit on how long a run may take.

An item that fails, a run that is cancelled

A type that cannot deal with one item says so with context.fail(path, reason) and moves on: the run still completes, and lists what it could not do under its data.

"data": {"errors": [{"path": "docs/a.pdf", "reason": "docling timed out"}]}

To stop a run, write "cancel": true into its record. The runner checks it before every item (context.each(), context.advance(), or context.check_cancel() in a type's own loop) and ends the run cancelled, with items_done where it got to and the errors and artifacts collected so far. A run cancelled before it was taken ends cancelled without starting.

Tests

This project uses pytest for testing and pytest-cov for code coverage analysis.

Installation

First, install the required testing dependencies:

.venv/bin/python -m pip install pytest pytest-cov

Basic Usage

Run all tests with coverage:

.venv/bin/python -m pytest --cov --cov-report=html

Common Commands

# Run tests with coverage for a specific module
.venv/bin/python -m pytest --cov=your_module

# Show which lines are not covered
.venv/bin/python -m pytest --cov=your_module --cov-report=term-missing

# Generate an HTML coverage report
.venv/bin/python -m pytest --cov=your_module --cov-report=html

# Combine terminal and HTML reports
.venv/bin/python -m pytest --cov=your_module --cov-report=term-missing --cov-report=html

# Run specific test file with coverage
.venv/bin/python -m pytest tests/test_file.py --cov=your_module --cov-report=term-missing

Viewing HTML Reports

After generating an HTML report, open htmlcov/index.html in your browser to view detailed line-by-line coverage information.

Coverage Threshold

To enforce a minimum coverage percentage:

.venv/bin/python -m pytest --cov=your_module --cov-fail-under=80

This will cause the test suite to fail if coverage drops below 80%.

Architecture

The addon is one runner, a few types, and two ways to reach it.

The runner

src/wexample_wex_addon_process/runner/process_runner.py is where a run is taken from pending to complete or to failed. It reads the run, the process it names, the selection that process names, resolves the type, walks the files, and writes back at every state change.

Three decisions are worth reading. A run in any state but pending is left where it is, because a queue redelivers and a run is done once. Progress — the files done and the file being worked on, current — is kept in memory and written down at most four times a second, each write followed by a ring: the other end reads the record when rung, and ten thousand files must not be ten thousand rewrites saying the same thing. A state change is never held back, and a failed run keeps current on the file it failed on. And the type never sees the run: it is handed a src/wexample_wex_addon_process/process_type/process_context.py holding the files, the options and a way to say how far it has got, so what a run is in is decided in one place.

Reading a selection

src/wexample_wex_addon_process/process_type/selection.py holds no list of files: a selection is a rule, and what it takes is read from disk each time it is asked for. The rule is PathMatcher from wexample-file, a line-for-line port of the PHP one the board uses; the walking is filestate's, the matcher being handed to a ChildrenFilterOption as its filter.

Going through filestate to enumerate and not only to change is deliberate: a type that reads the files and a type that rewrites them are then handed the very same tree, and there is one answer to what a selection takes rather than two.

The two implementations are kept honest against each other: three hundred random pattern sets, sixty paths and fifty directories were run through both and compared, with no difference.

Types, and the families that declare them

src/wexample_wex_addon_process/process_type/abstract_process_type.py is deliberately an ordinary class — no kernel, no attrs, no state. A type contributed from a bundle should be readable by whoever wrote the bundle rather than by whoever wrote wex.

A type's name carries its family: filestate:mode, agent:review. The prefix is not a branch taken at run time — src/wexample_wex_addon_process/process_type/process_type_registry.py asks each family for its names once, and what a record names is found in that map or is an error. A family declares names; it decides nothing.

src/wexample_wex_addon_process/process_type/family/file_state_process_type_family.py declares none of its types by hand. It walks filestate's DefaultOptionsProvider and keeps the options that override create_required_operation — the precise line between a treatment and a piece of structure, asked of the option itself rather than listed here. mode is a treatment, children is not. Thirteen types come out of it, and an option added to filestate is a fourteenth the same day.

Each of those is a src/wexample_wex_addon_process/process_type/filestate/file_state_option_process_type.py, which holds no logic either: the process's options become the configuration asked of every selected file, and filestate turns the difference with the disk into operations that can be described, applied and undone. action in the options picks the verb: check returns verdicts per file, dry_run the operations a pass would apply, fix applies them.

src/wexample_wex_addon_process/process_type/filestate/file_state_report_process_type.py is in that family and yet emits no operation, on filestate's own rule: a scope names something observable on disk, and a report changes nothing observable. An operation with no scope would be an operation nothing could ever run.

The two entry points

src/wexample_wex_addon_process/commands/run/execute.py runs one run, here and now, with no queue in sight. src/wexample_wex_addon_process/service/process_run_service.py is the same runner behind a wexample-queue service: it consumes process_run, and rings back on process_run_event with the same shape of message it received — kind, id, workdir. The runner is rebuilt for each message so that the workdir a ring names is the one the message came with: one worker serves every app.

The message carries no work at all, only a pointer. A worker starting after the message was published, or reading a run somebody has since edited, sees what is true now rather than what was true when the button was pressed.

What a record is called

src/wexample_wex_addon_process/const/process.py spells every key once. The board writes these very names into the same files, so one changed here has to be changed there — this file and ProcessRunHydrator on the PHP side are two halves of one contract.

Integration in the Suite

This package is part of the Wexample Suite — a collection of high-quality, modular tools designed to work seamlessly together across multiple languages and environments.

Related Packages

The suite includes packages for configuration management, file handling, prompts, and more. Each package can be used independently or as part of the integrated suite.

Visit the Wexample Suite documentation for the complete package ecosystem.

Dependencies

• wexample-file: >=9.2.0
• wexample-queue: >=2.0.0
• wexample-wex-addon-app: >=32.1.0

Versioning & Compatibility Policy

Wexample packages follow Semantic Versioning (SemVer):

• MAJOR: Breaking changes
• MINOR: New features, backward compatible
• PATCH: Bug fixes, backward compatible

We maintain backward compatibility within major versions and provide clear migration guides for breaking changes.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Free to use in both personal and commercial projects.

About us

Wexample stands as a cornerstone of the digital ecosystem — a collective of seasoned engineers, researchers, and creators driven by a relentless pursuit of technological excellence. More than a media platform, it has grown into a vibrant community where innovation meets craftsmanship, and where every line of code reflects a commitment to clarity, durability, and shared intelligence.

This packages suite embodies this spirit. Trusted by professionals and enthusiasts alike, it delivers a consistent, high-quality foundation for modern development — open, elegant, and battle-tested. Its reputation is built on years of collaboration, refinement, and rigorous attention to detail, making it a natural choice for those who demand both robustness and beauty in their tools.

Wexample cultivates a culture of mastery. Each package, each contribution carries the mark of a community that values precision, ethics, and innovation — a community proud to shape the future of digital craftsmanship.

Known Limitations & Roadmap

Current limitations and planned features are tracked in the GitHub issues.

See the project roadmap for upcoming features and improvements.

Status & Compatibility

Maturity: Production-ready

Python Support: >=3.10

OS Support: Linux, macOS, Windows

Status: Actively maintained

Useful Links

• Homepage: https://github.com/wexample/python-wex-addon-process
• Documentation: docs.wexample.com
• Issue Tracker: https://github.com/wexample/python-wex-addon-process/issues
• Discussions: https://github.com/wexample/python-wex-addon-process/discussions
• PyPI: pypi.org/project/wexample-wex-addon-process

Migration Notes

When upgrading between major versions, refer to the migration guides in the documentation.

Breaking changes are clearly documented with upgrade paths and examples.