proseforge-agent 0.2.0

Agentic orchestration layer for long-form novel production on top of the ProseForge engine.

🌐 English · 简体中文

ProseForge Agent

An agentic orchestration layer for long-form novel production. It wraps the canonical ProseForge engine ($PROSEFORGE_ROOT) with planning, retrieval, drafting, review, revision, deep memory, multi-provider model routing, daily workbooks, reports, extensions, an agent runtime, chat, an autonomous goal-directed loop, MCP integration, execution environments, skills, hosted cron, observability, middleware, and a real publish/binary/installer chain.

ProseForge Agent does not reimplement the writing engine. The engine at $PROSEFORGE_ROOT remains the source of truth for project slots, pipeline actions, guards, reports, and exports. This package owns orchestration: model calls, provider routing, schedules, evidence packs, agent memory, workflow state, conversational agent loop, and background event processing.

Status

v0.2.0 — first public release. 1301 tests passing on Python 3.10/3.11/3.12 across Windows, macOS, and Linux. See CHANGELOG.md for release notes, and "What is production-ready and what is contract-only" below for exactly which surfaces are live.

Installation

From PyPI (recommended):

pip install proseforge-agent
pf-agent --help

One-line installer (Linux / macOS / WSL2 — installs uv or uses pipx and puts pf-agent on your PATH):

curl -fsSL https://raw.githubusercontent.com/remacheybn408-boop/ProseForge-Agent/main/scripts/install.sh | sh

Windows PowerShell:

iex (irm https://raw.githubusercontent.com/remacheybn408-boop/ProseForge-Agent/main/scripts/install.ps1)

Docker (runs the HTTP service on port 8765; a bearer token is required for the non-loopback bind):

docker pull ghcr.io/remacheybn408-boop/proseforge-agent:latest
PF_AGENT_SERVICE_TOKEN=<your-token> docker compose up

From source:

git clone https://github.com/remacheybn408-boop/ProseForge-Agent.git
cd ProseForge-Agent
pip install -e ".[dev]"

PyPI and GHCR artifacts are published by the tag-triggered release workflow; if a brand-new version has not been tagged yet, install from source or pass --git main to install.sh.

Quickstart

pf-agent setup                 # guided wizard: providers, keys, workspace
pf-agent                       # bare command opens the interactive chat REPL

The bare pf-agent command works like other coding agents' CLIs: the first run bootstraps a workspace, and after pf-agent setup it talks to your configured default model (the REPL banner shows which provider and model are active). Without setup it runs against the offline fake provider so you can tour the CLI with no keys.

More entry points:

pf-agent doctor                # verify the installation and wiring
pf-agent chat --message "draft an opening"        # one-shot chat turn
pf-agent run --goal "draft a one-line opening" --max-iterations 5
pf-agent service serve         # local HTTP API on 127.0.0.1:8765

Set provider API keys in your environment (or .env, see .env.example): ANTHROPIC_API_KEY, OPENAI_API_KEY, and friends. Any command accepts --provider fake for a no-key offline tour.

Feature overview

The full stack: core orchestration, agent runtime, chat, native install, hardening, autonomous runtime, agent tooling, guided setup, novel operations, canon and story intelligence, editorial systems, MCP integration, provider/tool resilience, RAG, messaging gateway, execution environments, skills, hosted cron, observability, middleware, and the release chain.

Core stack

Portable config, workspace layout, ProseForge engine adapter, provider contract + fake provider, OpenAI-compatible transport, native/local provider profiles, memory schema/store with ingestion + compaction, retrieval router and evidence packs, phase plan generator, daily workbook engine, workflow state and recovery, chapter lifecycle (draft → review → rewrite → accept), CLI commands and reports, extensions, and an end-to-end demo with release checks.

Provider registry

Ten native provider profiles — OpenAI, Anthropic, Gemini, xAI/Grok, DeepSeek, Qwen, GLM, Mimo, MiniMax, Doubao — plus capability probing, provider fallback router, and a provider certification workflow. All providers go through a shared HttpTransport (urllib-backed in production, FakeHttpTransport in tests), so no test hits the network.

Agent runtime & chat

AgentKernel with dependency-injected provider, tools, session store, retrieval, and intent router; conversation modes; permission policy; tool registry with capability-based access; chat session store, prompt protocol, retrieval-cited evidence, chat-to-workflow handoff, and an agent event bus for background jobs with progress tracking.

Native install & distribution

Cross-platform app directories, installation doctor, first-run onboarding, native secret storage, provider setup wizard, pip/pipx/source and standalone-binary packaging manifest, Windows/macOS/Linux native support checks, shell completions, upgrade/migration/backup, uninstall, offline local model setup, a local agent service API, agent profiles, and a cross-platform native QA matrix.

Hardening & release gates

Provider usage metering + budgets, an agent safety / prompt-injection guard, streaming responses, a cross-platform CI pipeline, concurrency + advisory file locking, capability flags with safe-mode boot, and a contract/golden regression tier built on canonical fakes.

Autonomous runtime & agent tooling

A bounded autonomous loop (plan → act → verify → reflect → repeat), task planner with dependency-aware TODOs, self-verification with bounded reflection (pluggable domain verifiers such as the ProseForge review gate), a general tool framework (fs.read/write/edit, web.fetch/search), tool execution sandbox, sub-agent delegation, interruptibility + steering, and an agent evaluation harness.

Guided setup & novel operations

Guided pf-agent setup wizard, multiple setup modes, config generator, setup recovery, first-run bootstrap, novel project manifest, artifact graph, bulk import, scene-level workflow, chapter reorganization, export/compilation pipeline, and publishing metadata.

Canon & story intelligence

Canon bible manager, continuity conflict resolver, timeline engine, plot-thread manager, foreshadowing tracker, character-arc tracker, and relationship graph.

Writing quality & editorial

Writing domain tools registry, explicit writing rules, tone/style profile compiler, quality gates, literary regression suite, rewrite strategy library (nine selectable strategies), reader-experience review, whole-manuscript search, project health doctor, draft versioning with diff/branch/approval-gated rollback, a staged editorial pipeline (outline → … → final), human approval queue, writing analytics with completion prediction, and checksum-verified backups with dry-run restore.

Agent protocol, prompt, context, audit

Structured function-calling adapter, structured-output repair, context window management with compaction, system-prompt management, prompt template registry, multimodal / attachment ingestion (PDF, DOCX, CSV, Markdown, TXT, image metadata + vision describer), and an audit trail with deterministic replay.

MCP integration

MCP client foundation (stdio and HTTP/SSE transports with StaticMCPTransport for tests), server registry, security boundary, tool approval gate, tool schema validation, and credentials boundary. Real process/network transports are pluggable at call time; test suites use the deterministic in-process transport.

Provider & tool resilience

Tool timeout / rate limit / circuit breaker, provider fallback, graceful degradation, offline mode, and a request cache.

Sessions & context

Conversation lifecycle, cross-session search, session export/import, session branching, session merge, and multi-context switch.

Retrieval & RAG

Embeddings abstraction with fake provider, pluggable vector stores, hybrid retrieval (BM25 + vector), RAG ingestion pipeline, evidence-pack retrieval, and a RAG evaluation harness.

Notifications & jobs

Notification core, desktop notifications, webhook notifications, and a job status center.

Plugins

Plugin manifest spec, discovery, install/update/remove CLI, permission model, sandbox, dependency management, hooks, and a plugin test harness.

TUI & terminal

Full-screen terminal UI over the agent kernel (pf-agent tui), stdlib only (ANSI alternate screen — Linux, macOS, and Windows 10+): header with live provider/model, scrolling transcript with streamed responses, a status bar showing context-window usage (token meter per provider limit), a line editor with history/arrows/tab completion, double-Ctrl-C quit, and live /provider switching. Non-TTY streams and --plain fall back to the line-based UI — plus the slash-command registry, streaming tool output, and terminal session operations.

Messaging gateway

Messaging gateway core, platform adapter contract, and adapters for Telegram, Slack, Discord, Signal, mobile e-mail, Email, and a relay auth + pairing flow. The Telegram adapter has a real Bot API transport (long-polling over HTTPS; pf-agent gateway telegram start); the other adapters ship contract-shaped fake transports with real wiring injected per-adapter at call time.

Gateway delivery reliability (retry/backoff/dead-lettering) and media + voice ingestion round out the gateway track.

Execution environments

Execution environment abstraction, local / Docker / SSH / Singularity / Modal / Daytona backends, remote file sync and checkpoints, and a process registry + terminal lifecycle. The local backend really executes commands (pf-agent environments exec, permission + approval gated); remote backends emit deterministic dry-run plans with real invocation injectable at call time.

Managed tool gateway

Managed tool gateway foundation, web search and URL safety tools, cloud browser adapter, media generation and transcription tools, tool result artifacts + output limits, and the skill specification and registry.

Skills

Skill hub install and sync, autonomous skill creation, skill self-improvement + provenance tracking, and skill usage analytics + a safety audit surface.

Long-lived agents

User model + memory nudges, and hosted cron + scale-to-zero lifecycle support (authenticated + idempotent cron fires, deterministic wake plan, local fallback).

Observability & research

Observer hooks (read-only) with 7 event families (session / turn / provider_request / tool_call / approval / subagent / job), correlation-id propagation, fail-open error handling, and a pf-agent telemetry export --format jsonl --redact command. Middleware hooks (behavior-changing, opt-in, ordered, fail-open) with rewritten requests re-checked by downstream policy, plus research-ready pf-agent trajectories export --redact --format jsonl.

Real release chain

PyPIPublisher — real python -m build + twine upload for TestPyPI / PyPI, gated by the Task 47 PackageChecker and duplicate-version refusal, with injectable runner + dry_run for offline tests; tokens never appear in the plan, report, or logs.

BinaryBuilder — real PyInstaller invocation derived from the Task 48 BinaryManifest, followed by the pf-agent --version smoke command as the gate; portable-path report, dry_run for offline tests.

InstallerBuilder — per-OS installers (.msi via signtool, .dmg via codesign, install.sh via gpg), install paths from AppDirs, credential-gated signing that warn-skips when credentials are absent so an unsigned-but-valid installer is still produced.

Requirements

• Python 3.10+ (CI-tested on 3.10, 3.11, and 3.12)
• pyyaml>=6.0 (plus the tomli backport on Python 3.10 only)
• pytest>=7.0 for development and test runs

Everything else is standard-library only. Providers use urllib; MCP, gateway, and execution backends accept injected clients so no third-party network dependency is required to run the test suite.

Layout

src/proseforge_agent/       importable package (src-layout)
  cli.py                    pf-agent command entry point (71 command groups)
  config.py                 YAML config loading and validation
  workspace.py              project workspace helpers
  capabilities.py           capability flags and safe-mode boot
  concurrency.py            cross-platform advisory file lock + sqlite retry
  demo.py                   end-to-end offline demo runner
  errors.py                 typed exception hierarchy

  proseforge/               adapter boundary for the ProseForge engine

  llm/                      provider contracts, registry, HTTP transport,
                            certification, fallback router, streaming,
                            usage metering + budgets
  llm/providers/            10 native provider profiles

  memory/                   schema, store, ingestion, compaction, review,
                            user model + memory nudges
  retrieval/                embeddings, vector store, hybrid retrieval,
                            RAG ingestion, evidence packs, evaluation

  planning/                 intake parsing and phase plan generation
  daily/                    daily workbook and recommendations
  chapter/                  context, draft, review, rewrite, accept lifecycle
  workflow/                 workflow state and recovery
  novel/                    novel project manifest, artifact graph, scene
                            workflow, canon bible, timeline, plot threads,
                            character arcs, style/tone, quality gates,
                            editorial pipeline, backups, approvals
  reports/                  Markdown, JSON, and terminal report rendering
  extensions/               extension registry and hook base classes

  agent/                    kernel, intent router, modes, tools,
                            permissions, events, safety, loop, planner,
                            reflection, sandbox, subagent, profiles,
                            attachments, audit, context window, prompt
                            templates, function calling, structured
                            output repair, provider fallback, degradation,
                            offline, request cache, observability,
                            middleware, execution guard

  chat/                     session store, repl, prompts, retrieval,
                            memory, handoff, lifecycle, cross-session
                            search, export/import, branching, merge,
                            multi-context switch

  mcp/                      client, server registry, schema, policy,
                            approval, credentials

  gateway/                  messaging core, delivery reliability, media/
                            voice ingestion
  gateway/platforms/        Telegram / Slack / Discord / Signal / Email /
                            mobile e-mail adapters
  gateway/relay/            relay auth + pairing

  environments/             execution env abstraction, local / Docker /
                            SSH / Singularity / Modal / Daytona backends,
                            file sync + checkpoints, process registry,
                            serverless

  tools/                    managed tool gateway foundation, web
                            search/URL safety, cloud browser, media gen +
                            transcription, tool result artifacts

  skills/                   skill registry, hub install/sync, autonomous
                            creation, self-improvement, usage analytics,
                            safety audit

  cron/                     hosted cron verification + scale-to-zero plan

  eval/                     trajectory datasets + research-ready export

  notifications/            core, desktop, webhook

  plugins/                  manifest, discovery, permission, sandbox,
                            dependency, hooks, harness

  install/                  app dirs, doctor, package checks, binary
                            packaging manifest, binary_build.py (Task 184),
                            installers.py (Task 185), native OS support

  release/                  release gate, publish.py (Task 183)

  service/                  local agent service API
  setup/                    guided setup wizard, modes, config generator
  tui/                      terminal UI foundation, slash commands,
                            streaming tool output
  testing/                  canonical fakes shared by contract/golden tests

configs/                    agent and provider example configs
docs/                       operator, developer, implementation, and
                            task-card plans (docs/superpowers/plans/)
samples/                    sample extensions
tests/                      pytest suite (1301 tests) + contract / golden
                            tiers + fixtures
.github/workflows/ci.yml    cross-platform CI (Windows, macOS, Linux)

Development

Run the full test suite. No install is required — pythonpath is configured in pyproject.toml:

python -m pytest -q

Invoke the CLI directly:

python -m proseforge_agent.cli --help

After an editable install, the pf-agent command is on your path:

python -m pip install -e ".[dev]"
pf-agent --help

CLI Highlights

The pf-agent command exposes 71 command groups. Every group shares --format (markdown / json / terminal), --write, --dry-run, and --out output flags (except a few groups that own their own --format, such as telemetry and trajectories which emit JSONL).

Render the full command reference:

pf-agent report command-reference --format terminal

Inspect provider routing with the offline fake provider:

pf-agent provider --providers configs/providers.example.yaml

Chat REPL (add --stream for incremental output):

python -m proseforge_agent.chat.repl
pf-agent chat --message "draft an opening" --provider fake

Autonomous, goal-directed loop:

pf-agent run --goal "draft a one-line opening" --provider fake --max-iterations 5
pf-agent run --goal "写满 200 字的开头" --provider fake --verify

Rewrite strategies:

pf-agent rewrite strategies list
pf-agent rewrite --slug my-novel --strategy condense --chapter ch_001

Novel operations, canon, timeline, plot threads, style/quality gates:

pf-agent scene draft --slug my-novel --chapter ch_001 --scene sc_01
pf-agent bible refresh --slug my-novel
pf-agent timeline check --slug my-novel
pf-agent quality report --slug my-novel

MCP integration:

pf-agent mcp list
pf-agent mcp inspect filesystem
pf-agent mcp tools filesystem

Skills:

pf-agent skills list
pf-agent skills install <name>

Hosted cron + scale-to-zero:

pf-agent cron add "daily report" --schedule "0 9 * * *" --dry-run
pf-agent cron fire --fixture demo --provider fake

Observability & research exports:

pf-agent telemetry export --input .pf-agent/telemetry.jsonl --output out.jsonl --format jsonl --redact
pf-agent trajectories export --input .pf-agent/trajectories.jsonl --output ds.jsonl --format jsonl --redact

Health, capabilities, usage, QA:

pf-agent doctor
pf-agent status --capabilities
pf-agent usage report --since today
pf-agent qa ci --check

End-to-end offline demo:

python -m proseforge_agent.demo

Provider Profiles

Provider profiles live under configs/providers/ with matching adapters and tests under src/proseforge_agent/llm/providers/ and tests/. Current profiles: OpenAI, Anthropic, Gemini, xAI/Grok, DeepSeek, Qwen, GLM, Mimo, MiniMax, Doubao.

Keep secrets in environment variables or local ignored config files. Do not commit API keys into provider YAML.

What is production-ready and what is contract-only

The project intentionally splits "contract + fake" from "real wiring" so tests stay deterministic and offline. A few tracks are contract-complete with fake/dry-run implementations that the operator wires to real transports at call time:

• Production ready today — LLM providers (real HTTP via urllib, 10 profiles), memory (SQLite), workspace + config, planning / daily / chapter / workflow, tool registry + permission policy, attachment ingestion, embeddings + retrieval + RAG, skills, hosted cron verification, telemetry & trajectory export, the real publish/binary/installer chain (tag-triggered release workflow: PyPI + GHCR + GitHub Release), MCP stdio (real JSON-RPC subprocess transport: pf-agent mcp inspect/tools/call), the local execution environment (real subprocess execution behind pf-agent environments exec, permission + approval gated), the Telegram gateway (real Bot API long-polling: pf-agent gateway telegram start), and the local HTTP service (pf-agent service serve on port 8765, bearer-token auth for non-loopback binds — this is what the Docker image runs).
• Contract-complete with pluggable transport — MCP over HTTP/SSE (placeholder; stdio is real), remote execution environments (Docker/Modal/Daytona/SSH/Singularity emit deterministic dry-run plans that a real backend can execute), and the remaining gateway platform adapters (Slack/Discord/Signal/Email ship contract shape + fake transport; wire real HTTP at call time like the Telegram adapter does).

The 1301-test suite covers both tracks via injectable runners and fake transports, so python -m pytest -q is fully offline and requires no credentials.

ProseForge Engine Boundary

Set PROSEFORGE_ROOT when commands need to discover or call the canonical ProseForge engine. This repository remains the orchestration layer and should avoid duplicating engine-owned project, pipeline, guard, export, or report logic.

License

Apache-2.0. See LICENSE.