Skip to main content

passageway-connector-llmstxt

A passageway connector that indexes the llms-full.txt of a list of websites, a single markdown file such as a SKILL.md, or the pages an llms.txt index links to. Most documentation sites publish one of these for language models, and this connector makes them searchable without crawling.

What it does

  • Each top-level heading is one item. A llms-full.txt that concatenates a whole site becomes one item per page, with the sections below each heading as passages. Its id is the file's address plus the heading (https://docs.example.org/llms-full.txt#install), and its version is a hash of the text, so an edit re-indexes only that page. A # inside a code block is not a heading, and front matter at the top of a file is not indexed.
  • A front matter block with a url: key starts a page, wherever it is in the file. Some llms-full.txt files (the VitePress llms plugin writes them) open every page with ---, url: /howto/install.md and ---. Each such page is one item that runs to the next block, however many headings it has. Its hit links to the page itself (https://docs.example.org/howto/install, the .md dropped), its title is the page's first heading (else the last part of the path), and its path is the host and the folders of the page, so a path: filter works. Its id is the file's address plus the url: value. A file without such blocks is split by heading, as above.
  • A site address reads its llms-full.txt. https://docs.example.org is read as https://docs.example.org/llms-full.txt.
  • A file address is read as it is: https://docs.example.org/llms-full.txt, https://getpassageway.com/SKILL.md, any markdown or text file.
  • An llms.txt index is followed. Every link in it that stays on the same host is fetched and indexed as a file of its own. Links to other hosts are not followed.
  • Any failure fails the whole listing. A page that returns an error, or a server that is down, stops the sync with a message that names it, so a half-read site never deletes what was indexed before.

Install

Add the connector to the environment that runs passageway (see Install):

uv tool install --python 3.14 "passageway[llmstxt]"

An example

A file https://docs.example.org/llms-full.txt:

# Install

Run the installer.

## Linux

Use the package manager.

becomes the item:

{
  "title": "Install",
  "sections": [
    {
      "heading": null,
      "text": "Run the installer."
    },
    {
      "heading": "Linux",
      "text": "Use the package manager."
    }
  ]
}

and is configured as:

[[connector]]
name = "docs"
command = ["/path/to/python", "-m", "passageway_connector_llmstxt"]
env = { URLS = "https://docs.example.org|https://getpassageway.com/SKILL.md" }

For a site whose llms-full.txt opens every page with a url: front matter block, and whose release notes should rank below its how-to pages:

[[connector]]
name = "docs"
command = ["/path/to/python", "-m", "passageway_connector_llmstxt"]
env = { URLS = "https://docs.example.org", WEIGHTS = "changelog=0.5" }

Settings

All settings are strings in the connector's env.

Setting Meaning Default
URLS Required. Site addresses, file addresses and llms.txt addresses, separated by |.
MAX_BYTES The largest file that is read, in bytes. A larger one fails the sync. 20000000
WEIGHTS Lowers or raises the weight of the pages whose path starts with a prefix: changelog=0.5,archive/=0.3 (comma-separated prefix=weight, the weight above 0, the first match wins). Release notes repeat the vocabulary of how-to pages and push them down the results; a weight below 1 ranks them lower. The path is the page's address without the host. all pages 1.0
MAX_PAGES The most pages followed from one llms.txt index. The rest are skipped with a warning. 500

Requests carry the user agent passageway-connector-llmstxt, follow redirects and verify HTTPS against the operating system's trust store.

Metadata

Release files for passageway-connector-llmstxt 0.4.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for passageway-connector-llmstxt 0.4.0
File Size Uploaded Provenance Actions
passageway_connector_llmstxt-0.4.0.tar.gz 8.9 kB Without provenance Details for passageway_connector_llmstxt-0.4.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for passageway-connector-llmstxt 0.4.0
File Interpreter ABI Platform Actions
passageway_connector_llmstxt-0.4.0-py3-none-any.whl Python 3 none any Details for passageway_connector_llmstxt-0.4.0-py3-none-any.whl

Total release size: 15.3 kB

Release files / passageway_connector_llmstxt-0.4.0.tar.gz

Download URL passageway_connector_llmstxt-0.4.0.tar.gz
Size 8.9 kB
Tags Source
SHA-256 checksum
How to use checksums
3c570c87e1a7129960167f313530b527773f4d534fdaa88547fc260b6c4203fe
BLAKE2b-256 checksum
How to use checksums
2d5b2b5a3e6873919376f67bfc2385f5261c19a69902d4b3ec6f8ea6533f2f4f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / passageway_connector_llmstxt-0.4.0-py3-none-any.whl

Download URL passageway_connector_llmstxt-0.4.0-py3-none-any.whl
Size 6.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
c1aa5f3e69cc3938fdcd15bf95aae66c1f8f370828bbcf24cda047ca76671b6d
BLAKE2b-256 checksum
How to use checksums
ea0b90d413b955c67566153fcb36f344ab51fcb030eefe0f0f33aaa5b2732e2e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for passageway-connector-llmstxt 0.4.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page