pagespring

Acquire and normalize online documentation into clean, convertible source files — the acquisition front-end to pagespeak.

Point it at a manual's URL. A pattern recognizes the source type, acquires the raw pages (stdlib urllib), and normalizes them into ONE clean HTML/markdown file with absolute asset URLs under incoming/<slug>/. That clean file is the deliverable; converting it into the finished RAG corpus is a separate step (pagespeak) that consumes incoming/ on its own — pagespring never runs it.

Lean by design: pf-core[cli] (PyPI) + beautifulsoup4, stdlib fetch, no ML stack.

Intended use

pagespring is for publicly available documentation — vendor manuals, help centers, open textbooks, API specs. It fetches only what the source serves to any reader: there is no login/session handling, no paywall traversal, and no bot-detection evasion. It is a polite client: it identifies itself with a pagespring/<version> User-Agent (PAGESPRING_UA overrides it), honors 429 Retry-After, backs off on server errors, paces crawl requests, and caps crawl sizes.

It is a user-invoked, one-manual-at-a-time archiver — closer to "Save Page As" than to an autonomous crawler — so it does not consult robots.txt (which governs bots that discover URLs on their own; you supply the URL). Before mirroring a site, check its terms of use. What you may do with the acquired copy (personal RAG corpus, internal search, redistribution) is governed by the source's license — the deliverable under incoming/ stays on your machine, and nothing is re-published by this tool.

Install

pip install pagespring

Quick start

pagespring ingest https://docs.tableplus.com   # acquire + normalize → incoming/tableplus/
pagespring renormalize <slug>                   # replay normalize from kept raw/ — no re-crawl
pagespring refresh --all                        # re-check every manual against its source
pagespring localize <slug>                      # pull a deliverable's images later (resumable; --all)
pagespring patterns                             # list the source patterns
pagespring classify <url>                       # which pattern handles a URL (no fetch)
pagespring status                               # what's been acquired

Deliverables land in ./incoming/<slug>/ under the directory you run from.

Dev

bin/setup   # clone → venv + editable install with dev extras
bin/test    # pytest
bin/lint    # ruff check + ruff format --check + mypy (strict)

See docs/usage.md for the full command set and docs/architecture.md for the acquire → normalize flow and how to add a new source pattern.

Development Status
- 4 - Beta
Intended Audience
- Developers
Operating System
- OS Independent
Programming Language
Topic
- Text Processing :: Markup :: Markdown
Typing
- Typed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pagespring-0.4.0.tar.gz (78.9 kB view details)

Uploaded Jul 20, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

pagespring-0.4.0-py3-none-any.whl (63.1 kB view details)

Uploaded Jul 20, 2026 Python 3

File details

Details for the file pagespring-0.4.0.tar.gz.

File metadata

Download URL: pagespring-0.4.0.tar.gz
Upload date: Jul 20, 2026
Size: 78.9 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for pagespring-0.4.0.tar.gz
Algorithm	Hash digest
SHA256	`e1059194c775dc9ccd45a085cd173a78ec6f5ea3cfd9e3f26d85fe9205c340e5`
MD5	`d294a6184f76e9e3aeb009e3d3158850`
BLAKE2b-256	`434d0f68469e31f07b35537fea80da08245c911f69829fd4a21b35cc072a72b9`

See more details on using hashes here.

Provenance

The following attestation bundles were made for pagespring-0.4.0.tar.gz:

Publisher: publish.yml on phierceweb/pagespring

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: pagespring-0.4.0.tar.gz
- Subject digest: e1059194c775dc9ccd45a085cd173a78ec6f5ea3cfd9e3f26d85fe9205c340e5
- Sigstore transparency entry: 2204853860
- Sigstore integration time: Jul 20, 2026
Source repository:
- Permalink: phierceweb/pagespring@f6cec67dc8d56c39cd79ce9593dade3bce27bf66
- Branch / Tag: refs/tags/v0.4.0
- Owner: https://github.com/phierceweb
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@f6cec67dc8d56c39cd79ce9593dade3bce27bf66
- Trigger Event: push