Skip to main content

tai42-storage-github

License: Apache 2.0

A GitHub-backed Storage provider for the TAI ecosystem. It stores content as files in a GitHub repository and serves the full tai42_contract.storage.Storage surface — the text methods (load / list / upload / delete / delete_dir) plus the metadata read stat, the bounded read load_range and the binary methods (load_bytes / upload_bytes).

The TAI ecosystem

TAI is an open-source runtime for MCP tools, agents, and workflows. A Storage backend is "where content physically lives" — a pluggable provider the runtime's ResourceManager loads and renders content over. This package is one such provider (GitHub); siblings back the same contract with S3 (tai42-storage-s3) and the local filesystem (tai42-storage-local). The ecosystem is open-ended: any package can back the same contract, so this repo is this provider's own full doc home, and the documentation site covers the platform-level story:

Install

Requires Python 3.13+. Install from PyPI into the environment that runs the server:

uv add tai42-storage-github

Or from source — clone this repo and add it as an editable dependency; the tai42-* dependencies resolve in-tree from the workspace.

git clone https://github.com/tai42ai/tai42   # next to your app checkout
cd /path/to/your/app
uv add --editable ../tai42/plugins/storage-github

Use

The backend is loaded by import side-effect: importing the tai42_storage_github package runs its @tai42_app.storage.register_storage decorator, making GithubStorage the active storage provider. Point a manifest's storage_module at the package:

# manifest.yml
storage_module: tai42_storage_github

Configuration

Configure through STORAGE_GITHUB_-prefixed environment variables:

Variable Default Description
STORAGE_GITHUB_USERNAME — Repository owner (user or org).
STORAGE_GITHUB_REPO — Repository name.
STORAGE_GITHUB_BRANCH main Branch to read/write.
STORAGE_GITHUB_TOKEN — Access token; omit for a public repository.
STORAGE_GITHUB_TIMEOUT_TOTAL 15.0 HTTP timeout (seconds).
STORAGE_GITHUB_MAX_CONNECTIONS 200 Connection-pool ceiling.
STORAGE_GITHUB_MAX_KEEPALIVE_CONNECTIONS 50 Keep-alive pool size.
STORAGE_GITHUB_KEEPALIVE_EXPIRY 300.0 Keep-alive expiry (seconds).

The token is held as a SecretStr; it authenticates requests but never appears in a log line, repr, or error message.

Reads and writes

  • Reads (load, load_bytes) are one Contents API call each at the configured branch, asking for the raw form (Accept: application/vnd.github.raw+json), which serves every byte of a file of any size. A directory answers its JSON listing and reads as FileNotFoundError.
  • Ranged reads (load_range) resolve the branch's head commit through the Git refs API, then read raw.githubusercontent.com at that commit with a Range header and Accept-Encoding: identity (the raw host compresses, and a compressed answer ranges over the encoded bytes): two calls, one of them REST. Reading at the commit, never at the branch name, means a ranged read answers the branch's current bytes, not a cached copy. An answer that ignores the range or is still encoded raises; there is no whole-object read in its place. A zero-length range sends no request.
  • Rate limit: GitHub allows 60 REST calls an hour without a token and 5000 with one; stat, load and load_bytes cost one REST call each and load_range one.
  • Writes (upload, upload_bytes) go through the Contents API (base64) and are capped conservatively at 1 MiB. A larger payload raises ValueError before the request rather than truncating; the API's own 422 is the backstop.
  • Listing uses the recursive Git Trees API (one request) and raises loudly on a truncated response rather than acting on a partial listing.
  • stat is one Contents API call asking for the object form (Accept: application/vnd.github.object+json): it answers the file's size, and infers content_type from the path suffix — GitHub stores no per-object content-type. A missing path and a directory raise FileNotFoundError.

Development

uv venv --python 3.13
uv pip install --no-sources --group dev --editable .
uv run --no-sync pytest --cov --cov-report=term-missing
uv run --no-sync ruff check .
uv run --no-sync ruff format --check .
uv run --no-sync pyright

License

Apache-2.0. See LICENSE and NOTICE.

Metadata

Release files for tai42-storage-github 2.0.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for tai42-storage-github 2.0.1
File Size Uploaded Provenance Actions
tai42_storage_github-2.0.1.tar.gz 27.7 kB Without provenance Details for tai42_storage_github-2.0.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for tai42-storage-github 2.0.1
File Interpreter ABI Platform Actions
tai42_storage_github-2.0.1-py3-none-any.whl Python 3 none any Details for tai42_storage_github-2.0.1-py3-none-any.whl

Total release size: 45.4 kB

Release files / tai42_storage_github-2.0.1.tar.gz

Download URL tai42_storage_github-2.0.1.tar.gz
Size 27.7 kB
Tags Source
SHA-256 checksum
How to use checksums
188abc2c773d1b45ca489010e03090ebb29c2c09af312500e635866080d08100
BLAKE2b-256 checksum
How to use checksums
13c3171220655ecf72e546566c70fb03194fa16b483d2e566431ea1b858e6c11
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / tai42_storage_github-2.0.1-py3-none-any.whl

Download URL tai42_storage_github-2.0.1-py3-none-any.whl
Size 17.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
3fc41260b6e3212bcd4e5c87f4454ec174fd3376f57680aafdcc42a9f77b4455
BLAKE2b-256 checksum
How to use checksums
859d17c2b5394293d911c2deafac6e58479960ba0c0f0168196e03a78f30a939
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Security for tai42-storage-github 2.0.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

2.0.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page