lncrawl-sourcelib

Interpreter for declarative light-novel source definitions.

A source definition describes how to read one website as data rather than as code: one YAML document per host, validated against a published schema and interpreted at runtime. RFC-0001 in the lncrawl/sources repository is the normative definition of the format. This package is written against it, and where the two disagree the RFC wins.

pip install lncrawl-sourcelib

What it does

sourcelib check                          # resolve and validate every document
sourcelib resolve specs/example.com.yaml # what a spec actually says once merged
sourcelib schema -o schema.json          # the JSON Schema editors read

from sourcelib.spec.checks import check_resolved
from sourcelib.spec.resolve import resolve_file

spec = resolve_file("specs/example.com.yaml", root=".")
for problem in check_resolved(spec):
    print(problem)

resolve matters more than it looks. A spec can inherit through a chain of bases, so "what am I actually running" has to be answerable in one command or a deep chain becomes undebuggable.

Why it is its own package

It is not a feature of the crawler; it is the crawler's next core. Built inside the crawler it would grow references into the code it replaces, and swapping it in would become a rewrite instead of a dependency bump.

This package never imports the crawler, and its CI asserts that. It also means the definitions repository can validate itself with no application in the loop.

Licence

Apache-2.0. The crawler is GPL-3.0-or-later, and Apache-2.0 feeds a GPL-3 application without friction. Nothing here is copied or adapted from the crawler's sources.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lncrawl_sourcelib-0.1.1.tar.gz (251.0 kB view details)

Uploaded Aug 4, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

lncrawl_sourcelib-0.1.1-py3-none-any.whl (78.6 kB view details)

Uploaded Aug 4, 2026 Python 3

File details

Details for the file lncrawl_sourcelib-0.1.1.tar.gz.

File metadata

Download URL: lncrawl_sourcelib-0.1.1.tar.gz
Upload date: Aug 4, 2026
Size: 251.0 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for lncrawl_sourcelib-0.1.1.tar.gz
Algorithm	Hash digest
SHA256	`5075caa7acbbfa2dc25ee65f6edc2bb3f906af1ed7ddc9f9b089c135e5b36f53`
MD5	`56a3f013a9cddcebb1fc6a5a90d2aabc`
BLAKE2b-256	`df0a622b5fabcfce88a67a787aa6810b7c7a71104fb119156e5cde14039db4fc`

See more details on using hashes here.

Provenance

The following attestation bundles were made for lncrawl_sourcelib-0.1.1.tar.gz:

Publisher: publish.yml on lncrawl/sourcelib

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: lncrawl_sourcelib-0.1.1.tar.gz
- Subject digest: 5075caa7acbbfa2dc25ee65f6edc2bb3f906af1ed7ddc9f9b089c135e5b36f53
- Sigstore transparency entry: 2340628663
- Sigstore integration time: Aug 4, 2026
Source repository:
- Permalink: lncrawl/sourcelib@1b5d4c4a0fa36744cb14a9be0e26ce2f62bc60eb
- Branch / Tag: refs/heads/main
- Owner: https://github.com/lncrawl
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@1b5d4c4a0fa36744cb14a9be0e26ce2f62bc60eb
- Trigger Event: workflow_dispatch