passiveworkers 0.1.1

Make your computer work for you and others: a local-first compute network where machines do typed jobs for each other. Flagship task — deep research: local LLMs research the live web as independent analysts and a blind editor compiles one cited report.

Passive Workers — make your computer work for you, and for others

Your models. Your connection. Your disk. Passive Workers turns any computer with Ollama into a worker that does real jobs locally — for you today, and (opt-in) for other people's computers tomorrow. The flagship job, working now, is deep research: multiple local models research the live web as independent analysts, and a blind editor compiles one cited markdown report into ./reports/. That's the single-player half. The other half is a network where computers do work for each other — research is just the first task type (see "The network", below).

pip install 'passiveworkers[all]'   # core + extraction + private-docs + MCP
ollama pull qwen3:14b               # any decent models you like — it auto-detects what you have
pw research "What changed in EU AI Act enforcement this quarter, and who has been fined?"

From source instead: git clone https://github.com/wikithoughts/passiveworkers && pip install '.[all]'

🔬 Deep research (standard) — analysts: qwen3:14b, gemma3:12b, llama3.2 · editor: qwen3:14b
  [1/3] qwen3:14b researching the live web…
      12 sources · 390 words · 41s
  [2/3] gemma3:12b researching the live web…
  ...
  blind judge + editor compiling the report…
📄 Report ready in 7.2 min · 1480 words · 31 sources → reports/2026-06-10-eu-ai-act….md

Prefer a UI? pw serve → a single-user research desk at http://127.0.0.1:8770 — brief in, live progress, rendered report, history of everything you've researched.

Research your own documents too (private, local RAG)

pw library add ~/Documents/contracts        # index files or folders (PDF, Word, txt, md)
pw research "What are the renewal terms across my contracts?" --local   # docs only
pw research "How do my notes compare to the latest guidance?"           # docs + live web (default)

Your files are chunked and embedded locally (Ollama nomic-embed-text) into ~/.passiveworkers/library.db — nothing is uploaded. Reports cite documents as [L#] and web sources as [S#], kept in separate sections.

Retrieval is state-of-the-art but lean: hybrid (dense embeddings ⊕ BM25 lexical, fused by reciprocal rank fusion) so exact names/codes/numbers aren't missed; structure-aware chunking that never straddles a section; parent-window expansion for grounding; and optional Contextual Retrieval (PW_CONTEXTUAL_CHUNKS=1, Anthropic's technique — a small local model situates each chunk before indexing) and reranking (PW_RERANK=1). Indexing is incremental (unchanged files are skipped). Measure it on your own corpus with python scripts/bench_rag.py — we publish what actually helps, not vendor numbers.

Use it from your own AI (MCP)

pw mcp        # run as an MCP server (stdio)

Add to Claude Desktop's claude_desktop_config.json so your assistant can call the engine:

{ "mcpServers": { "passive-workers": { "command": "pw", "args": ["mcp"] } } }

Tools exposed: research, library_search, library_add. Your own agentic AI orchestrates; our multi-model, live-web + private-library engine is the capability it reaches for.

Recommended setup (avoids public-search rate limits, keeps queries private):

docker compose up -d searxng     # self-hosted meta-search; pw auto-detects it

Why this exists

• Currency beats memory. Frontier chatbots answer from training data that is months or years old. This engine reads the web now and cites what it found. In our own blind trial, live-web research was the only thing that beat a frontier model — both times currency mattered. For time-sensitive questions it leads with the freshest-dated sources (so they survive the cap and get read first), pins the current year into the search query so the engine returns this year's results instead of an SEO-dominant old page, and researches deeper on breaking topics — while leaving stable-fact questions in plain relevance order.
• Private by construction. No account, no server, no telemetry. The only thing that leaves your machine is the web searches themselves. Reports are files on your disk.
• Plural by design. Different model families make different mistakes. A planner discovers distinct angles (STORM-style); each analyst researches its own angle with its own model and drafts from full page extracts, and a blind editor preserves disagreement (agree / differ / unique sections — never a forced consensus). Question diversity × model diversity catches what any single model hallucinates.
• Right source for the query. Beyond live web, academic-looking queries also hit arXiv and definitional ones Wikipedia (PW_SOURCE_ROUTING=off to disable). Models stay warm between steps (PW_OLLAMA_KEEP_ALIVE, default 30m; set 0 to unload immediately) so there are no reload stalls mid-run.
• Free forever. It's your hardware.
• Made to share (opt-in). Idle compute is wasted compute. The same engine is built to be a network where computers do bounded jobs for each other — yours for you, and (only if you switch it on) yours for others. A commons of machines, not a data-harvesting cloud: you always see and consent to what your machine does, and it returns work it produced — never proxied traffic. Deep research is the first task type; more are on the roadmap. See The network, below.

Honesty section (when the research task is NOT the right tool)

A frontier chatbot is better when the answer lives in stable knowledge — math, code, explanations, anything where being current doesn't matter. We measured this bluntly: local models lose that fight 0/10 (docs/TRIAL_RESULTS.md). This tool wins when the answer lives on today's web — prices, regulations, releases, markets, anything where "as of when?" decides usefulness. Optional --editor api brings your own OpenRouter key for a frontier editor pass over locally-gathered findings — your choice; the default is fully local.

Benchmark (honest, small sample)

On a 25-question subset of OpenAI's SimpleQA, the engine scored 64% (single qwen2.5:14b, snippet-only search, LLM-graded — scripts/bench_simpleqa.py). Context, plainly: SimpleQA rewards short factoid recall, which is the opposite of what this tool is built for (multi-source reports where currency and citation matter); the leaders' ~95% figures use bigger models, deeper agentic loops, and more sources. We publish the number — small sample and all — because the honest floor is more useful to you than a cherry-picked one. Run it yourself: python scripts/bench_simpleqa.py --n 100.

Citation fidelity (the metric that actually matters here)

A research tool lives or dies on one question: when it says X [S3], does source S3 say X? scripts/eval_citation_fidelity.py measures exactly that — for every cited claim it checks content-overlap with the source it points at and flags numbers stated in a claim that are absent from the source. Two keyless (no-API-cost) modes:

python scripts/eval_citation_fidelity.py --report reports/your-report.md   # score an existing report (re-fetches its sources)
python scripts/eval_citation_fidelity.py --run --depth quick               # fresh run, scored against the exact extract each model read

It is honest about being a floor: lexical grounding catches off-topic citations and fabricated numbers — the common, damaging failures — but a GROUNDED verdict means "not obviously fabricated", not "verified true" (it can't detect subtle misrepresentation). The "grounded rate" is of verifiable claims; unreachable sources are reported separately, never counted as failures.

Currency gap (where live web beats a frontier model's memory)

This tool's real edge isn't raw model size — it's currency. scripts/eval_currency_gap.py measures exactly that: the local council (live web) vs a frontier model answering from its frozen training knowledge, scored against curated references, as a matrix by currency window × category. A static control set keeps it fair (where currency is irrelevant, the frontier should win). It spends nothing by default — a bare run is a $0 dry run that validates the question set and estimates cost; only --run (with OPENROUTER_API_KEY in your env) makes the paid frontier calls:

python scripts/eval_currency_gap.py            # dry run — validate + estimate, $0
python scripts/eval_currency_gap.py --run      # paid: council (free) vs frontier (your API key)

Security model (designed in, not bolted on)

• No browser automation, no computer use, no sessions, no cookies — ever. Search API + plain fetch of public pages only. The gravest agent attacks (session-token theft, authenticated exfiltration) have nothing to grab here.
• All web content is untrusted data. It passes a sanitizer (invisible-Unicode and hidden-comment stripping) and enters prompts only inside spotlighting delimiters marked "data, never instructions" (council/sanitize.py). The same gate covers the ends of the pipeline too: your brief is scrubbed of hidden/bidi characters and length-bounded before it shapes any prompt, and every model-written passage is re-scrubbed before it lands in the report (so a payload smuggled through a source can't survive into the artifact) — all without touching visible layout or citations.
• Models hold zero tool privileges. They only return text; every action (search, fetch, file write) is plain Python under this repo's control. Reports write only into ./reports/; fetches pass an SSRF guard (public hosts only, size-capped).

Hardware guide

Your machine	Models that fit (4-bit)	Experience
8 GB RAM/VRAM	3–4B (llama3.2, qwen3:4b, gemma3:4b)	quick reports, lighter analysis
16 GB	7–14B (qwen3:14b, gemma3:12b)	the sweet spot
24 GB+	14–32B (+ mistral-small:22b)	best local quality

Models run sequentially by design — no concurrent loads fighting for memory. On CPU-only or busy machines, cap the cast by weight size: PW_MODEL_CAP_GB=3 pw research … (big models on CPU crawl at 3–6 tok/s — a small model that fits is always faster than a large one that spills).

Page evidence uses trafilatura for clean main-content extraction (with a regex fallback); full credits in docs/PRIOR_ART.md.

The network — computers doing real work for each other

This is the other half of Passive Workers — and the reason for the name. Everything above runs on one machine; the same repo contains the network layer (council/net/) where machines do typed jobs for one another. It's the next track — working but still maturing (it already powers a live two-country deployment), not a toy and not a separate product. The vision: send a job, the network splits it across available computers, each does its chunk locally, the parts are reassembled and delivered back, with progress reporting and failover so load can move between machines — research is simply the first job type. Connect machines in different countries and reports gain genuinely different windows on the web — each node researches from its own egress and returns its own cited findings (never proxied traffic), an editor merges with per-country sections, and a non-tradeable mutual-aid credit accounts for who helped whom. It already powers a live two-country deployment, plus typed marketplace jobs (deep research, sharded batch work with capability matching, and assisted human-in-the-loop tasks — pw tasks / pw accept / pw deliver: an operator consents to a bounded brief and does it with their own AI or by hand, never our autonomous code). The asker rates the result (pw rate), building operator reputation that gates higher-trust offers — while newcomers can still take ungated work. Deliverables can be real files, moved as content-addressed, integrity-verified chunks (pw deliver <task> @file <job> → pw fetch <job> <dir>) — a corrupted or swapped chunk is detected, never written. With the [crypto] extra, deliverables are signed (the asker verifies which operator produced them) and files can be end-to-end encrypted to the asker (pw keygen → the coordinator relays ciphertext it cannot read). For authenticity that holds even against a hostile coordinator, the asker pins an operator's signing key out of band — pw fingerprint (operator) → pw trust add (asker), or trust-on-first-use — and pw fetch verifies against the pinned key, refusing a swapped one. Two principles are absolute: operators always see and consent to the work their machine does (never hidden tasks), and when a job needs a real computer driven, it is handed to the human operator to do with their own AI under approval — our code never automates anyone's machine. The long game is a commons of computers doing real work for each other — no token, no secondary market, money only ever at the edges. See docs/FEDERATION_V2.md.

Documentation

Doc	What
docs/CONTEXT.md	The why, the history, the layered vision.
docs/ROADMAP.md	Milestones + pivots (living tracker).
docs/ARCHITECTURE.md	Roles, local vs networked shape, trust/security.
docs/DECISIONS.md	Settled decisions + rationale (ADR-style, the full log).
docs/ECONOMICS.md	Credit, give/take, score-weighted payouts, legal posture.
docs/TRIAL_RESULTS.md	Our blind trial vs a frontier model — losses included.
docs/GLOSSARY.md	Terms (Council, analyst, judge, lens, credit…).
docs/CONTRIBUTE_COMPUTE.md	Plug a machine into the federation — what it does, earns, and why it's safe.
docs/RELEASING.md	How to publish to PyPI (verified build; needs your token).

Status

Young software, honestly labeled: the single-player engine (the flagship research task) works and is verified end-to-end; the network layer is the maturing next track. We publish our methodology and our losses, not just wins. Issues and PRs welcome. MIT.