newswatcher

English | 한국어

newswatcher watches RSS feeds and robots-permitted listing pages, matches new articles against topics you define, summarizes the matches with an LLM, and sends one topic-grouped digest by email, chat, or both. Delivery is optional — the core collects, summarizes, and archives on its own; email and chat are opt-in add-ons (newswatcher[email] / newswatcher[chat]). Several outlets covering the same story collapse into a single entry. The topics are yours to define, so the same tool tracks a stock ticker, a technology, a policy beat, or any subject a feed covers.

1. Install

newswatcher requires Python 3.11 or newer. The core — collect, summarize, archive — installs on its own; delivery is opt-in, so add the channel you want:

pip install newswatcher            # core: collect, summarize, archive (read with `articles`)
pip install "newswatcher[email]"   # + email digests, via mailmail
pip install "newswatcher[chat]"    # + chat digests, via pushpush
pip install "newswatcher[all]"     # + both

The Quickstart below needs only the core newswatcher — email and chat come later (Delivery).

2. Quickstart

These are terminal commands (a shell — Terminal, PowerShell, or Command Prompt — not the Python prompt). First get a free Gemini API key from Google AI Studio (sign in with Google, click Create API key, copy it), then store it — you are prompted, and the key is not echoed:

newswatcher set-key gemini

Define a topic, register an RSS source, run one pass, and read the summaries — no email setup needed to get a first result:

newswatcher add-topic markets --include stocks Fed "interest rate" earnings --exclude sports
newswatcher add-source korea-herald "https://www.koreaherald.com/rss/newsAll" --kind rss --topic markets
newswatcher poll --no-mail
newswatcher articles

If articles shows nothing, the poll just matched no new articles yet — that is normal, not an error; broaden the --include keywords or run poll again later.

A topic matches on the feed's own language, so pair the keywords with the feed: English keywords for an English feed, Korean keywords for a Korean feed. newswatcher topics and newswatcher sources show what you registered.

To have the digest emailed or sent to chat instead of read with articles, install the delivery extra and configure the channel once — see Delivery.

3. Commands

Run newswatcher --help or newswatcher <command> --help for every option; newswatcher --version prints the version.

4. Delivery

The digest is sent by email, to a chat channel, or both — set one or both destinations. Each channel is an opt-in extra (newswatcher[email] / newswatcher[chat]); its companion package keeps its own credentials, so newswatcher never stores your email password or bot token. Without either extra, newswatcher still collects, summarizes, and archives — read the archive with newswatcher articles.

• Email goes through the mailmail package (newswatcher[email]). Set up an account (or an address-book alias) once with mailmail's own CLI (mailmail --help); then --to ADDRESS, or the NEWSWATCHER_DIGEST_TO setting, names that alias or a plain address.
• Chat goes through the pushpush package (newswatcher[chat]). Configure a route (a bot plus its destination — Telegram, Slack, or Discord) once with pushpush's own CLI (pushpush --help); then --push ROUTE, or the NEWSWATCHER_DIGEST_PUSH setting, names that route. The digest is sent as one markdown message.

So newswatcher holds no secret of its own: the LLM key lives with thinchat, the email password with mailmail, the chat token with pushpush — each in its own store, exactly as when the tool is used on its own. Configure them all in one guided pass with newswatcher setup, and see the full map — what is set and where it lives — with newswatcher doctor.

5. News feeds

Any valid RSS/Atom feed works as a source. A representative set of verified Korean feeds is below; the full list, grouped by section and marked with which were live at verification, is in docs/korean-news-rss.md. A site with no feed can still be followed with a --kind crawl source.

A representative set of verified international (English) feeds is below; the full list, grouped by category and marked with which outlets paywall the article body, is in docs/world-news-rss.md. Pair these with English topic keywords.

6. Configuration files

newswatcher stores hand-edited configuration under $XDG_CONFIG_HOME/newswatcher, or ~/.config/newswatcher when XDG_CONFIG_HOME is unset. The CLI writes the same files, so CLI and manual configuration can be mixed: a CLI write appends or edits in place, leaving any comments and formatting you added by hand intact.

topics.toml contains topic filters. An article matches when its title or feed summary contains any include keyword and no exclude keyword. An empty includes list matches every article.

[[topic]]
name = "markets"
includes = ["stocks", "Fed", "interest rate", "earnings"]
excludes = ["sports"]

[[topic]]
name = "semiconductors"
includes = ["chip", "foundry", "HBM", "TSMC", "Nvidia"]

categories.toml (optional) contains classification labels. When it is present, the summarizer -- in the same LLM call that writes the summary, so at no extra cost -- also classifies each article into one category by name (empty unless the reply names a defined category on its own first line), stored on the article. Unlike a topic (a keyword filter deciding what is kept), a category is a single label for display grouping, chosen by the model from context (so "the regulator held a charity event" is not miscategorised as regulation the way a keyword would). Each entry is a name and an optional hint that guides the model. When the file is absent, classification is off and the category is empty.

[[category]]
name = "M&A"
hint = "인수합병, 매각, 지분·경영권 거래"

[[category]]
name = "기타"
hint = "위에 안 맞는 나눔·인사·행사 등 일반"

sources.toml contains RSS or crawl sources. topics selects the topic filters applied to a source. Set keep_all = true for a source whose every article should be retained without keyword filtering.

[[source]]
name = "korea-herald"
kind = "rss"
url = "https://www.koreaherald.com/rss/newsAll"
topics = ["markets", "semiconductors"]

[[source]]
name = "exchange-notices"
kind = "crawl"
url = "https://example.com/markets/notices"
topics = ["markets"]
item = "article.news-item"
title = "h2"
link = "a@href"
date = "time"
body_selector = "main article"

The item, title, and link selectors are required for crawl sources; date and body_selector are optional. The link selector uses the css@attribute form when the URL is stored in an attribute.

Non-secret settings can also be placed in config.toml. Environment variables take precedence over corresponding settings there. For example, NEWSWATCHER_DIGEST_TO maps to digest_to, and NEWSWATCHER_DIGEST_PUSH to digest_push. NEWSWATCHER_DEDUP_THRESHOLD (dedup_threshold, 0.0–1.0, default 0.5) sets how alike two headlines must be to collapse as one story — raise it to merge less, lower it to merge more. The article archive and run state use the XDG data and state directories; NEWSWATCHER_DATA_DIR and NEWSWATCHER_STATE_DIR can override them. The archive deletes nothing by default; to prune old records, set NEWSWATCHER_ARCHIVE_KEEP_DAYS (archive_keep_days, a positive integer) and each poll removes archived articles older than that after the digest is sent. Leaving it unset keeps everything (this deletion is irreversible, so enable it deliberately). NEWSWATCHER_STORE_BODY (store_body, a boolean) turns on body capture without the --store-body flag: each fetched article body is then kept in a separate bodies directory under the data dir (alongside archive), for re-summarizing or keeping the original text. It is off by default, and captured bodies are local only — never delivered. Captured bodies are never auto-pruned (unlike the article archive), so removing them is your choice; with --no-store the kept bodies have no matching archived article.

7. Provider keys and model

A provider is the LLM service that writes the summaries. newswatcher supports four; pass the name in the left column to --provider or set-key:

The provider key is a secret, and it lives with thinchat — the library newswatcher summarizes through — not with newswatcher. Store it once with setup (which configures email and chat in the same pass) or with set-key for the key alone; both prompt without echoing and write to thinchat's own credential store:

newswatcher setup            # the LLM key, plus email and chat, in one pass
newswatcher set-key gemini   # just the LLM key

Each key is also read from its environment variable (the table above), which takes precedence, so a one-off run can supply a key without storing anything.

newswatcher summarizes with Gemini's free tier by default. Choose another provider, and optionally a specific model, with --provider / --model, or persistently with the NEWSWATCHER_LLM_PROVIDER / NEWSWATCHER_LLM_MODEL settings (llm_provider and llm_model in config.toml):

newswatcher poll --provider claude --model claude-sonnet-5
export NEWSWATCHER_LLM_PROVIDER=openai

8. Responsible collection

Every feed, listing-page, and article request is checked against the site's robots policy before it is sent, and newswatcher identifies itself with its user agent. A disallowed URL is not fetched. The durable archive and outbound digest contain the LLM-written summary, source link, and metadata only. Raw article bodies are transient summary input, never sent and not in the article archive; they are kept only in the separate bodies store, and only when you opt in with --store-body (or NEWSWATCHER_STORE_BODY).

9. Scheduling

Install a recurring poll every 30 minutes:

newswatcher schedule install

Choose another interval with minutes, Nm, or Nh, and inspect or remove the job as needed:

newswatcher schedule install --every 2h
newswatcher schedule status
newswatcher schedule remove

Scheduling uses crontab on Linux and macOS and schtasks on Windows. On Windows any interval under a day works (--every 45, --every 5h); on Linux and macOS cron only fires intervals that divide evenly (15/20/30 min, 1/2/4/8/12 h, or daily) and rejects the rest rather than mis-scheduling them. The scheduled process uses the same configuration as an interactive poll, so make sure the LLM key is reachable (from thinchat's store or its environment variable) along with any settings not stored in config.toml. newswatcher doctor confirms it before you schedule.

On Windows the task is registered under the installing user and runs in their interactive session, so it does not fire while nobody is signed in — a locked screen is fine, a machine sitting at the sign-in screen is not. It also inherits the Task Scheduler default of not starting on battery power. Check it with schtasks /Query /TN newswatcher-poll. On Linux and macOS the cron job has neither restriction.

A poll takes a single-instance lock, so a scheduled poll and a manual one never run at once — whichever starts second logs that a poll is already running and exits. The lock uses flock on Linux and macOS and msvcrt on Windows.

10. Use it from an AI coding agent

This repo ships a poll skill: ask in plain words ("run my newswatcher poll", "check the news") and it runs one poll and relays what it found.

Claude Code

In the Claude Code chat, add the marketplace and install:

/plugin marketplace add seokhoonj/newswatcher
/plugin install newswatcher@newswatcher

Then invoke it with /newswatcher:poll, or just ask in plain language. The skill calls the newswatcher command, so install the package too (pip install newswatcher). See plugins/newswatcher/skills/poll/SKILL.md.

Codex

In your terminal, add the marketplace and install:

codex plugin marketplace add seokhoonj/newswatcher
codex plugin add newswatcher@newswatcher

The poll skill responds automatically to matching requests.

By hand (symlink)

Symlink the skill into your skills directory and call it as /poll:

ln -s "$PWD/plugins/newswatcher/skills/poll" ~/.claude/skills/poll   # Claude Code -> /poll
ln -s "$PWD/plugins/newswatcher/skills/poll" ~/.codex/skills/poll    # Codex -> $newswatcher:poll

Claude Code picks it up immediately; Codex needs a restart to load it.

11. License

MIT