Skip to main content

classics-index

从扫描本的版心(页边书眉)OCR 里建章回索引:卷 → 篇 → 回。不依赖任何在线服务,纯 Python,无第三方运行时依赖(OCR 引擎是可选的)。

pip install -e .
classics-index volumes --headers heads.jsonl --out toc.json
classics-index pian    --toc toc.json --band titles.jsonl --out toc-pian.json
classics-index hui     --ocr hui.jsonl --names "1=西游记,2=水浒传" --out hui.json
classics-index find    --toc toc.json --q 卷3

与数据管线的关系:本包是私有工程 rag-build(OCR → 切块 → 向量 → RAG 数据库)的构建依赖。 数据管线的代码不在 socratic-tutor 仓库里,本包可独立使用,也可被任何索引管线复用。

它解决什么问题

点校本、影印本没有文本层,目录页也常常 OCR 不可用。但每一页的页眉都印着卷次("史記卷二")或回目("灵根育孕源流出心性修持大道生")。于是:

输入 输出
每页页眉带的 OCR({"page": n, "vol": 2}) 卷的起止页,连续覆盖全书
每页右侧标题带的 OCR({"page": n, "lines": [...]}) 篇名页与篇页码范围
章回小说每页页眉 OCR({"page": n, "texts": [...]}) 回目分段、附录标记、多部书切分

设计上的三个关键判据

  1. 卷靠"连续 ≥5 页":目录页也会写出"史記卷十三",那种孤页会被丢掉。
  2. 篇靠"行的形态"而不是字号:实测卷眉(48 px 宽)比篇名(46 px)还宽,宽度阈值不可靠;可靠的是"短行 + 序号结尾干净"(目录行末尾是":"或省略号)。
  3. 回靠"页眉标题变化":章回小说每页页眉都印着当前回目,标题一变就是新回;附录("附/录")重复上一回页眉,不能占回号。

数据从哪来

配套的 OCR 脚本在本仓库的 eval/p3/(vertical_header_ocr.py、vertical_title_ocr.py、novel_hui_ocr.py),它们只是 PaddleOCR 的一层薄封装:

  • 版心带:页面右侧 18%、高度 8%–55%,1.2 s/页;
  • 标题带:右侧 42%、高度 3%–65%,3.5 s/页;
  • 章回页眉:全宽、高度 3%–15%,0.55 s/页。

把任意 OCR 引擎的结果整理成本包要的 JSONL 也行。

已在什么数据上验证

书 规模 结果
《史记》点校本 一 358 页 7 卷 + 7 篇,覆盖 358/358
《三国志》二 140 页 卷10–14 五篇篇名正确
四大名著合订本 2745/5291 页 215 回目段 / 3 部(过切 +1%)

限制

  • 版心位置是按中华书局版式调的,换版式要调裁切参数;
  • 章回小说的"换书边界"会晚一回(下一部第一回的页眉常没有"第一回");
  • 只处理竖排/横排的中文印刷体,不做手写、不做多栏学术期刊。

许可

尚未选定(建议 MIT);选好之前不要发布到任何包索引。

Metadata

Release files for classics-index 0.1.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for classics-index 0.1.3
File Size Uploaded Provenance Actions
classics_index-0.1.3.tar.gz 17.1 kB Without provenance Details for classics_index-0.1.3.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for classics-index 0.1.3
File Interpreter ABI Platform Actions
classics_index-0.1.3-py3-none-any.whl Python 3 none any Details for classics_index-0.1.3-py3-none-any.whl

Total release size: 31.1 kB

Release files / classics_index-0.1.3.tar.gz

Download URL classics_index-0.1.3.tar.gz
Size 17.1 kB
Tags Source
SHA-256 checksum
How to use checksums
f93270bc4bda2b6d7a4624a7613cb826179571f2068e7d089586fbdc3253bbfe
BLAKE2b-256 checksum
How to use checksums
9baba108971b7a27b54a7ab8ac7c7f276851a346755b7cb7b4a4c728c208400c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.14

Release files / classics_index-0.1.3-py3-none-any.whl

Download URL classics_index-0.1.3-py3-none-any.whl
Size 13.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
c654327e4e1201cfc929f561d582b6b838673e880795b8dc7d18105fbf338209
BLAKE2b-256 checksum
How to use checksums
13bfb76cb95dfbbf2969d2bc469e5921227de9cd60c79bf309c96060be57ccc9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.14

Security for classics-index 0.1.3

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page