Skip to main content

passageway-connector-github

A passageway connector that indexes GitHub repositories: by default their documentation (the README and every markdown file), and their code when you ask for it. Name the repositories one by one, or every repository of a user or an organisation.

What it does

  • Each file is one item, read from the default branch (or the branch you name). Its id is the repository and the path (acme/docs/guide/setup.md), its address the file on github.com, and its version the file's git blob sha, so listing a repository costs one call and reads no file, and an edit re-indexes only that file.
  • Markdown files are cut at their headings. The first # heading is the title (else the file name; a README at the top of the repository is titled by the repository), front matter is left out, and the repository and folders are the breadcrumb (path), so a path: filter works.
  • Code, when FILES asks for it, is cut into passages of 40 to 80 lines, at a blank line where one comes, each headed by its line range (Lines 41–96). Files that are not UTF-8 text (images, archives) are skipped with a warning.
  • owner/* is every repository of a user or an organisation. Forks and archived repositories are left out unless FORKS or ARCHIVED says otherwise. With a TOKEN, an organisation's private repositories are included where the token can read them, and so are the token owner's own.
  • Any failure fails the whole listing. A repository that is missing or unreadable, a refused token, a used-up rate limit or a server that is down stops the sync with a message that says what to do, so a half-read repository never deletes what was indexed before. A rate limit names the time it resets.

Without a token GitHub allows 60 API calls an hour from one address: enough to list a few repositories, not to read many files. Indexing reads each file once (one call per file) and then only the files that changed, so give it a token for anything but a small public repository. A fine-grained personal access token with read access to Contents and Metadata of the repositories is enough.

Install

Add the connector to the environment that runs passageway (see Install):

uv tool install --python 3.14 "passageway[github]"

An example

A file guide/setup.md in acme/docs:

# Setup

Install it.

## Linux

Use the package manager.

becomes the item:

{
  "id": "acme/docs/guide/setup.md",
  "title": "Setup",
  "url": "https://github.com/acme/docs/blob/main/guide/setup.md",
  "path": ["acme/docs", "guide"],
  "sections": [
    {
      "heading": null,
      "text": "Install it."
    },
    {
      "heading": "Linux",
      "text": "Use the package manager."
    }
  ]
}

and is configured as:

[[connector]]
name = "github"
command = ["/path/to/python", "-m", "passageway_connector_github"]
env = { REPOS = "acme/docs|acme/api@release|ann/*" }
secrets = { TOKEN = "env:GITHUB_TOKEN" }

To index the code of a repository as well as its documentation:

env = { REPOS = "acme/api", FILES = "*.md|src/**/*.py", EXCLUDE = "src/**/test_*.py" }

Settings

All settings are strings in the connector's env; TOKEN belongs in secrets.

Setting Meaning Default
REPOS Required. Repositories separated by |: owner/repo, owner/repo@branch (the branch is then part of every id), or owner/* for every repository of a user or an organisation.
TOKEN A GitHub personal access token, as a secret reference. Needed for private repositories and for more than 60 API calls an hour. none
FILES The files to index, as glob patterns separated by |, matched against the path in the repository: ** crosses folders, * stays in one, and a pattern without / matches the file name in any folder. A file that does not end in .md, .mdx or .markdown is indexed as code. *.md|*.mdx|*.markdown
EXCLUDE Files to leave out, as glob patterns like FILES. none
FORKS true to include forks in owner/*. false
ARCHIVED true to include archived repositories in owner/*. false
MAX_FILES The most files indexed from one repository; the rest are skipped with a warning. 5000
MAX_FILE_BYTES The largest file that is indexed, in bytes; larger ones are skipped with a warning. 1000000
API_URL The REST API address, for GitHub Enterprise Server: https://github.example.org/api/v3. https://api.github.com

Requests carry the user agent passageway-connector-github and verify HTTPS against the operating system's trust store.

Metadata

Release files for passageway-connector-github 0.5.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for passageway-connector-github 0.5.0
File Size Uploaded Provenance Actions
passageway_connector_github-0.5.0.tar.gz 10.9 kB Without provenance Details for passageway_connector_github-0.5.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for passageway-connector-github 0.5.0
File Interpreter ABI Platform Actions
passageway_connector_github-0.5.0-py3-none-any.whl Python 3 none any Details for passageway_connector_github-0.5.0-py3-none-any.whl

Total release size: 18.4 kB

Release files / passageway_connector_github-0.5.0.tar.gz

Download URL passageway_connector_github-0.5.0.tar.gz
Size 10.9 kB
Tags Source
SHA-256 checksum
How to use checksums
0035e14fa9525a8ca2c7f816e74fd7dd2e00660a08778c98a8adc052b455ea30
BLAKE2b-256 checksum
How to use checksums
f5a3d5aad7fc7f5511bce91e9260c9f71f1105a0c25b653f753701ac4cd252dc
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / passageway_connector_github-0.5.0-py3-none-any.whl

Download URL passageway_connector_github-0.5.0-py3-none-any.whl
Size 7.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
3c4e6e8564145197827962b5be31c1e034a1c727925dae783662249c28194b3c
BLAKE2b-256 checksum
How to use checksums
1cc8dcebf95ad8cf425c1758d2d2e581dcf71e6f84ff993edc86e7b1bf9fdda3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for passageway-connector-github 0.5.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.5.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page