commentdraft 0.1.0

Triage social-media comments from a CSV and draft grounded replies for a human to review.

commentdraft

Triage social-media comments from a CSV and draft grounded replies for a person to send.

The one thing to know before anything else. commentdraft never publishes. It reads a CSV, writes a page of drafts, and exits. Sending a reply is a person's job, done by hand, after reading it. There is no code path that posts, and you can check that with grep in about four seconds.

What it is

• A triage step. Every comment is classified reply, skip or escalate, each with a one-line reason you can disagree with.
• A drafting step tied to one document you supply. That document is the only thing a draft may state as fact. A question it does not answer becomes escalate, not a guess.
• A review artefact. One table, one row per comment, carrying the decision, the reason and the draft alongside the platform, the author and the model that produced it. Cost appears once per file, as a total the run actually billed, not per row: the per-row token counts and cost live in the CSV underneath the page, not in front of a reviewer. You read it, edit elsewhere, and send by hand.

Requirements

Python 3.11 or newer, and an API key for any OpenAI-compatible chat endpoint.

Install

uv tool install commentdraft
uv tool install "commentdraft[pdf]"

The second adds the PDF knowledge source, which is an optional extra because it pulls in a rendering library that a text knowledge file does not need. pip install works the same way if you would rather not use uv.

To run an unreleased change, install from the repository instead:

uv tool install git+https://github.com/Ab-Romia/commentdraft.git

The whole thing, end to end

The repository ships a worked example: a fictional foraging field guide, with its knowledge file, its voice file, its worked examples, and a small comment file.

git clone https://github.com/Ab-Romia/commentdraft.git
cd commentdraft/examples/field-guide-book

export OPENROUTER_API_KEY="your key"      # or whatever [model].api_key_env names
commentdraft run    --config config.toml --comments comments.csv --out out
commentdraft review out/review.csv --out out

run writes out/review.csv and prints a run report: how many replies, skips and escalates, what the run cost, and which drafts repeat each other. review turns the CSV into out/review.html, which is the page below.

One table, one row per comment, carrying the decision, the reason, the draft reply and the model that wrote it, plus one cost total for the whole file rather than a column of per-row figures in front of a reviewer; the per-row token counts and costs stay in the CSV underneath. Green rows were answered, pink rows go to a person, and the rest were left alone. The three decisions are the product: what it declines to answer matters as much as what it drafts.

Nothing on that page is mocked up. Every row is real output from the run written up in docs/bakeoff.md, which put thirty comments through three models. Eleven of those rows are shown here, chosen so that all three decisions and the awkward cases are visible at once rather than the first seven rows, which all happen to be replies. Your own run produces the same page from out/review.csv with whatever it measured.

That page was rendered with --approved. Without it, every page carries a red banner saying the drafts were approved by nobody, and it stays until you pass the flag, which you pass after reading every row and not before:

commentdraft review out/review.csv --out out --approved

Two more things you will want on day one:

commentdraft chat --config config.toml    # one comment at a time, with the full trace
commentdraft ui   --config config.toml    # the same, in a page on 127.0.0.1 only

Both send one comment through the exact code a batch run uses and show the decision, the draft, the token counts and the cost, without writing a CSV. docs/writing-a-voice.md has a short section on using them to check an edit to your voice file before spending on a full run.

And two you will want once:

commentdraft ingest  --config config.toml --pdf book.pdf --out knowledge.md
commentdraft bakeoff --config config.toml --comments comments.csv --out out

Three concepts, and no fourth

Concept	Where it lives	What it holds
Configuration	config.toml	what you sell, how it behaves, which model, what it costs
Knowledge	a text file you supply	the only thing a draft may state as fact
Voice	prompts/voice.md, prompts/examples.md	your rules and your worked examples, in your language

The engine holds no copy in your language and none about your product. Everything a reader of your replies perceives comes from your files, and you never open a .py file to change it. No string literal under src/ contains a non-ASCII character, and a test walks the AST of every module to keep it that way.

The engine does hold English of its own, and it is machine scaffolding rather than voice: the JSON output contract and the two instructions sent to a model, the review page's fixed notices and column headings, the local test page's own button labels and headings, the four lines of the run report, and the one reason string a locally decided empty comment carries. That list is complete, it is written out in docs/limits.md with who reads each item, and none of it enters a reply as your voice.

tests/fixtures/nazzef-kit-ar is a third product, config, voice, worked examples and comments, written entirely in Arabic, and it passes the same acceptance tests as the two English examples above. docs/writing-a-voice.md is the chapter to read next.

What leaves your machine

Every comment and the whole knowledge document travel to whichever endpoint [model].base_url names, as part of the request that drafts a reply. That is the one place your data leaves the machine running commentdraft.

load_config refuses to start a run unless every model entry, the default one and every [[bakeoff.models]] entry alike, sets params.provider.data_collection = "deny" in its own table. That is an instruction sent with the request, not something this tool can enforce once the request has left it: whether the endpoint honors it is between you and whoever operates it. Nothing else about your product, your audience or their comments leaves this machine: no telemetry, no analytics call, and no second server this tool talks to.

What it deliberately does not do

• It does not post, queue, schedule, or hold a credential for any platform. There is no client for one in the package:

  grep -rniE 'youtube|tiktok|instagram|facebook|googleapis|oauth' src/
  

  returns nothing, and tests/test_generality.py::test_the_package_contains_no_client_for_any_platform runs an equivalent check on every push: that one walks the AST for exact import names and exact hostnames in string constants, where the grep above is a case-insensitive substring sweep over every byte of src/, comments included. Both pass. Neither is the other.

• It does not retrieve. The whole knowledge document goes into the cached prompt prefix. That is the right answer while the document fits the context window and the wrong one afterwards. docs/architecture.md says where the line is.

• It does not evaluate models in general. commentdraft bakeoff runs your own comments through several models and builds a blind judging page. That is the whole feature. If you want an evaluation harness with datasets, assertions and CI gates, use promptfoo. It is the right tool for that job and this is not trying to be it.

• It does not check whether your facts are true. load_config validates shape and never content. A wrong price travels straight through to a human reviewer, which is the correct place to catch it.

• plug_cap is an alarm threshold. It reports the share of replies matching one of your configured plug_markers, a case-insensitive substring test, and flags a run that goes over. It does not know what a link or a price looks like on its own; it only recognizes the exact markers you listed. Nothing stops mid-run. What holds the rate down is what you wrote in your voice file. docs/limits.md covers the false positives and false negatives that come with a substring test.

Documentation

File	What it covers
docs/architecture.md	the cached prefix, why there is no retrieval, and where that stops working
docs/configuration.md	every key config.toml accepts, which are required, and what each does
docs/comments-csv.md	the input CSV: every column, which are required, and what happens to the rest
docs/writing-a-voice.md	writing your rules and examples, in your language, placeholder by placeholder
docs/sources.md	adding a knowledge source handler
docs/bakeoff.md	comparing models blind, one measured run against the example, and one parameter that lies
docs/platform-policy.md	which clause each safety property exists for
docs/limits.md	what this cannot do, stated before you find out

What this does, and what it does not do

commentdraft drafts. It does not send, schedule, queue, or hold a credential for anywhere a draft could be sent. Every draft leaves this tool as a row in a CSV and a card on an HTML page, and the only way one reaches an audience is a person reading it and choosing to send it.

That person is responsible for what they send. The tool has no view on whether a draft is accurate, appropriate, or permitted where they are about to publish it, and it is not capable of forming one. load_config validates the shape of your configuration and never its truth: a wrong price passes straight through, on purpose, to the reviewer who can recognize it.

This is a design decision, not a missing feature. docs/platform-policy.md maps this behavior, clause by clause, to the platform and legal provisions it was built to satisfy; that mapping is not repeated here.

This project is not affiliated with, authorized by, maintained by, or endorsed by Google, YouTube, Meta, Instagram, or TikTok. Platform names appear in this repository only to cite a published policy clause or as an example configuration value. Anything you publish with this tool's help is your responsibility, under the terms of whichever platform you publish on.

What you supply

You supply the source document. Everything a draft states as fact comes from it, which makes it the single most important input and the one this project has the least ability to check.

By pointing commentdraft at a document you confirm that you have the right to use it this way: that you own it, licensed it, or are otherwise permitted to send its contents to the model provider you have configured. The tool does not check ownership, licensing, copyright status, or whether the document contains personal data, and it cannot. commentdraft includes, distributes, and licenses no book, course, or other work of its own; the example products in this repository are fictional, written for this project, and not a real product for you to reuse.

The same applies to the comments you feed it. They were written by people. Exporting them, sending them to a provider, and storing the output CSV are all processing decisions you are making, under whatever law applies to you.

Two defaults exist to keep those inputs where you put them:

• Source text, PDFs and CSVs are gitignored by class rather than by filename, so an accidental commit takes deliberate effort. knowledge/ and any *.pdf file are covered by that rule, and they should stay that way in any fork or deployment of this project.
• Every model entry is required to send provider.data_collection = "deny", and a test asserts it on every entry. Whether the provider honors it is between you and the provider. The request is the part this project controls, and it makes that part unambiguous.

Transparency

Drafts are written by software. Pretending otherwise is dishonest and, in a growing number of jurisdictions, unlawful.

bot_disclosure_text is a required, non-empty setting: the exact sentence used when someone asks whether a person wrote a reply. It is validated non-empty and there is no evasive mode; an earlier version of this design had one and it was removed rather than defaulted off, because a setting that exists is a setting somebody eventually turns on.

A human reads every draft, edits it, and takes editorial responsibility for it before anything is sent. Whether the disclosure sentence alone satisfies your obligations depends on where you are and what you publish, and this README is not legal advice. Any deployment obligation that follows from publishing a reply belongs to you, the operator, not to this repository. docs/platform-policy.md maps each property here to the specific clause it was built against, so you can check the reasoning rather than trust this summary.

About any cost figures

Any rate that appears in a config file or a document in this repository is a point-in-time observation, dated where it appears, and never a quote. Providers change pricing without notice, and a number that was true on the date it was written can be wrong by the time you read it. Verify against your provider's current rates before you plan around any figure here.

Cost per row is computed from the token counts the API actually returned, never from an estimate, and it is left blank rather than set to zero when pricing is incomplete. Blank is the truth. Zero is a claim.

The one set of measured figures this repository publishes is in docs/bakeoff.md: one bake-off run against the shipped example on 2026-08-01, three models, with the command that reproduces it and the same dating rule applied to every number in it.

Where this came from

This started as a private commission. The person on the other end had a real product, a real price and a real audience, and a public reply that quoted the wrong price or invented a claim was a cost that landed on them, not on the tool. That is the entire explanation for the parts of this design that look over-careful: the review page, the disclosure line that is required and has no evasive setting, the refusal to answer outside the source document, and the missing posting path.

The client-specific parts are gone. The product, the language, the source document and the platform credentials all stayed behind, and nothing here was copied forward from them. What survived is the shape those constraints forced, which turned out to be the useful part.

Contributing

Read CONTRIBUTING.md. In short: tests first, no non-ASCII string literals under src/, no posting path, and no dash typography in prose. make check runs what CI runs.

License

Apache-2.0. See LICENSE and NOTICE.