Skip to main content

tai42-storage-github

License: Apache 2.0

A GitHub-backed Storage provider for the TAI ecosystem. It stores content as files in a GitHub repository and serves the full tai42_contract.storage.Storage surface — the text methods (load / list / upload / delete / delete_dir) plus the metadata read stat, the bounded read load_range and the binary methods (load_bytes / upload_bytes / load_object).

The TAI ecosystem

TAI is an open-source runtime for MCP tools, agents, and workflows. A Storage backend is "where content physically lives" — a pluggable provider the runtime's ResourceManager loads and renders content over. This package is one such provider (GitHub); siblings back the same contract with S3 (tai42-storage-s3) and the local filesystem (tai42-storage-local). The ecosystem is open-ended: any package can back the same contract, so this repo is this provider's own full doc home, and the documentation site covers the platform-level story:

Install

Requires Python 3.13+. Install from PyPI into the environment that runs the server:

uv add tai42-storage-github

Or from source — clone this repo and add it as an editable dependency; the tai42-* dependencies resolve in-tree from the workspace.

git clone https://github.com/tai42ai/tai42   # next to your app checkout
cd /path/to/your/app
uv add --editable ../tai42/plugins/storage-github

Use

The backend is loaded by import side-effect: importing the tai42_storage_github package runs its @tai42_app.storage.register_storage decorator, making GithubStorage the active storage provider. Point a manifest's storage_module at the package:

# manifest.yml
storage_module: tai42_storage_github

Configuration

Configure through STORAGE_GITHUB_-prefixed environment variables:

Variable Default Description
STORAGE_GITHUB_USERNAME — Repository owner (user or org).
STORAGE_GITHUB_REPO — Repository name.
STORAGE_GITHUB_BRANCH main Branch to read/write.
STORAGE_GITHUB_TOKEN — Access token; omit for a public repository.
STORAGE_GITHUB_TIMEOUT_TOTAL 15.0 HTTP timeout (seconds).
STORAGE_GITHUB_MAX_CONNECTIONS 200 Connection-pool ceiling.
STORAGE_GITHUB_MAX_KEEPALIVE_CONNECTIONS 50 Keep-alive pool size.
STORAGE_GITHUB_KEEPALIVE_EXPIRY 300.0 Keep-alive expiry (seconds).

The token is held as a SecretStr; it authenticates requests but never appears in a log line, repr, or error message.

Reads and writes

  • Reads (load, load_bytes, load_object) are one Contents API call each at the configured branch, asking for the raw form (Accept: application/vnd.github.raw+json), which serves every byte of a file of any size. A directory answers its JSON listing and reads as FileNotFoundError. load_object infers the content type from the path suffix, as stat does.
  • Ranged reads (load_range) resolve the branch's head commit through the Git refs API, then read raw.githubusercontent.com at that commit with a Range header and Accept-Encoding: identity (the raw host compresses, and a compressed answer ranges over the encoded bytes): two calls, one of them REST. Reading at the commit, never at the branch name, means a ranged read answers the branch's current bytes, not a cached copy. An answer that ignores the range or is still encoded raises; there is no whole-object read in its place. A zero-length range sends no request.
  • Rate limit: GitHub allows 60 REST calls an hour without a token and 5000 with one; stat, load and load_bytes cost one REST call each and load_range one.
  • Writes (upload, upload_bytes) go through the Contents API (base64) and are capped conservatively at 1 MiB. A larger payload raises ValueError before the request rather than truncating; the API's own 422 is the backstop.
  • Listing uses the recursive Git Trees API (one request) and raises loudly on a truncated response rather than acting on a partial listing. list(directory) reads the tree of that directory alone (<branch>:<directory>), so it grows with the directory, not the repository; a directory that does not exist answers [] once the branch is confirmed to exist. delete_dir lists the directory the same way.
  • stat is one Contents API call asking for the object form (Accept: application/vnd.github.object+json): it answers the file's size, and infers content_type from the path suffix — GitHub stores no per-object content-type. A missing path and a directory raise FileNotFoundError.

Development

uv venv --python 3.13
uv pip install --no-sources --group dev --editable .
uv run --no-sync pytest --cov --cov-report=term-missing
uv run --no-sync ruff check .
uv run --no-sync ruff format --check .
uv run --no-sync pyright

License

Apache-2.0. See LICENSE and NOTICE.

Metadata

Release files for tai42-storage-github 3.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for tai42-storage-github 3.0.0
File Size Uploaded Provenance Actions
tai42_storage_github-3.0.0.tar.gz 29.5 kB Without provenance Details for tai42_storage_github-3.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for tai42-storage-github 3.0.0
File Interpreter ABI Platform Actions
tai42_storage_github-3.0.0-py3-none-any.whl Python 3 none any Details for tai42_storage_github-3.0.0-py3-none-any.whl

Total release size: 48.1 kB

Release files / tai42_storage_github-3.0.0.tar.gz

Download URL tai42_storage_github-3.0.0.tar.gz
Size 29.5 kB
Tags Source
SHA-256 checksum
How to use checksums
bbf69ac0ac8851c3314d7deda3062c0dfc20caccea789d5e644041ad35b4c386
BLAKE2b-256 checksum
How to use checksums
b4aec1952e5e077ddf2ef3c5f259bf8d3c146f37ea6153f542892149844ec481
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / tai42_storage_github-3.0.0-py3-none-any.whl

Download URL tai42_storage_github-3.0.0-py3-none-any.whl
Size 18.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
2a87b528c9a29c5afc884bb6ae9859e9de6e38c128d31437c919093cdeeedd69
BLAKE2b-256 checksum
How to use checksums
408f4ee4dc4cdf61cfde02f6b677f2f685c6e4ad69ffd58a53a07bfdba7e031c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Security for tai42-storage-github 3.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

3.0.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page