Skip to main content

tai42-storage-github

License: Apache 2.0

A GitHub-backed Storage provider for the TAI ecosystem. It stores content as files in a GitHub repository and serves the full tai42_contract.storage.Storage surface — the text methods (load / list / upload / delete / delete_dir) plus the metadata read stat, the bounded read load_range and the binary methods (load_bytes / upload_bytes / load_object).

The TAI ecosystem

TAI is an open-source runtime for MCP tools, agents, and workflows. A Storage backend is "where content physically lives" — a pluggable provider the runtime's ResourceManager loads and renders content over. This package is one such provider (GitHub); siblings back the same contract with S3 (tai42-storage-s3) and the local filesystem (tai42-storage-local). The ecosystem is open-ended: any package can back the same contract, so this repo is this provider's own full doc home, and the documentation site covers the platform-level story:

Install

Requires Python 3.13+. Install from PyPI into the environment that runs the server:

uv add tai42-storage-github

Or from source — clone this repo and add it as an editable dependency; the tai42-* dependencies resolve in-tree from the workspace.

git clone https://github.com/tai42ai/tai42   # next to your app checkout
cd /path/to/your/app
uv add --editable ../tai42/plugins/storage-github

Use

The backend is loaded by import side-effect: importing the tai42_storage_github package runs its @tai42_app.storage.register_storage decorator, making GithubStorage the active storage provider. Point a manifest's storage_module at the package:

# manifest.yml
storage_module: tai42_storage_github

Configuration

Configure through STORAGE_GITHUB_-prefixed environment variables:

Variable Default Description
STORAGE_GITHUB_USERNAME — Repository owner (user or org).
STORAGE_GITHUB_REPO — Repository name.
STORAGE_GITHUB_BRANCH main Branch to read/write.
STORAGE_GITHUB_TOKEN — Access token; omit for a public repository.
STORAGE_GITHUB_TIMEOUT_TOTAL 15.0 HTTP timeout (seconds).
STORAGE_GITHUB_MAX_CONNECTIONS 200 Connection-pool ceiling.
STORAGE_GITHUB_MAX_KEEPALIVE_CONNECTIONS 50 Keep-alive pool size.
STORAGE_GITHUB_KEEPALIVE_EXPIRY 300.0 Keep-alive expiry (seconds).

The token is held as a SecretStr; it authenticates requests but never appears in a log line, repr, or error message.

Reads and writes

  • Reads (load, load_bytes, load_object) are one Contents API call each at the configured branch, asking for the raw form (Accept: application/vnd.github.raw+json), which serves every byte of a file of any size. A directory answers its JSON listing and reads as FileNotFoundError. load_object infers the content type from the path suffix, as stat does.
  • Ranged reads (load_range) resolve the branch's head commit through the Git refs API, then read raw.githubusercontent.com at that commit with a Range header and Accept-Encoding: identity (the raw host compresses, and a compressed answer ranges over the encoded bytes): two calls, one of them REST. Reading at the commit, never at the branch name, means a ranged read answers the branch's current bytes, not a cached copy. An answer that ignores the range or is still encoded raises; there is no whole-object read in its place. A zero-length range sends no request.
  • Rate limit: GitHub allows 60 REST calls an hour without a token and 5000 with one; stat, load and load_bytes cost one REST call each and load_range one.
  • Writes (upload, upload_bytes) go through the Contents API (base64) and are capped conservatively at 1 MiB. A larger payload raises ValueError before the request rather than truncating; the API's own 422 is the backstop.
  • Listing uses the recursive Git Trees API (one request) and raises loudly on a truncated response rather than acting on a partial listing. list(directory) reads the tree of that directory alone (<branch>:<directory>), so it grows with the directory, not the repository; a directory that does not exist answers [] once the branch is confirmed to exist. delete_dir lists the directory the same way.
  • stat is one Contents API call asking for the object form (Accept: application/vnd.github.object+json): it answers the file's size, and infers content_type from the path suffix — GitHub stores no per-object content-type. A missing path and a directory raise FileNotFoundError.

Development

uv venv --python 3.13
uv pip install --no-sources --group dev --editable .
uv run --no-sync pytest --cov --cov-report=term-missing
uv run --no-sync ruff check .
uv run --no-sync ruff format --check .
uv run --no-sync pyright

License

Apache-2.0. See LICENSE and NOTICE.

Metadata

Release files for tai42-storage-github 3.0.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for tai42-storage-github 3.0.2
File Size Uploaded Provenance Actions
tai42_storage_github-3.0.2.tar.gz 29.5 kB Without provenance Details for tai42_storage_github-3.0.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for tai42-storage-github 3.0.2
File Interpreter ABI Platform Actions
tai42_storage_github-3.0.2-py3-none-any.whl Python 3 none any Details for tai42_storage_github-3.0.2-py3-none-any.whl

Total release size: 48.1 kB

Release files / tai42_storage_github-3.0.2.tar.gz

Download URL tai42_storage_github-3.0.2.tar.gz
Size 29.5 kB
Tags Source
SHA-256 checksum
How to use checksums
59430077cb387aa939796451b8f69c60e86778921f313ea68435ab86d3549cde
BLAKE2b-256 checksum
How to use checksums
4048af711ecfd9bdb5e83804a86f4dc1c116d879a5f99f11b69c1189a92c80ad
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / tai42_storage_github-3.0.2-py3-none-any.whl

Download URL tai42_storage_github-3.0.2-py3-none-any.whl
Size 18.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b84de8a681021828778615600d2ff2ee87e83ac2ee3e20ec3a008b108c596aa2
BLAKE2b-256 checksum
How to use checksums
f2915147f5472ddc6ddaf64a6b427a4befee4f6d851a2cf1e8451e69b0478a46
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Security for tai42-storage-github 3.0.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

3.0.2 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page