tai42-storage-github
A GitHub-backed Storage provider for the TAI ecosystem. It stores content as
files in a GitHub repository and serves the full tai42_contract.storage.Storage
surface — the text methods (load / list / upload / delete / delete_dir)
plus the metadata read stat, the bounded read load_range and the binary
methods (load_bytes / upload_bytes).
The TAI ecosystem
TAI is an open-source runtime for MCP tools, agents, and workflows. A Storage
backend is "where content physically lives" — a pluggable provider the runtime's
ResourceManager loads and renders content over. This package is one such
provider (GitHub); siblings back the same contract with S3 (tai42-storage-s3) and
the local filesystem (tai42-storage-local). The ecosystem is open-ended: any
package can back the same contract, so this repo is this provider's own full doc
home, and the documentation site covers the platform-level story:
- Storage & resources concept: https://tai42.ai/concepts/storage-and-resources
- Build a storage provider (author guide): https://tai42.ai/guides/authors/storage-provider
- Ecosystem catalog: https://tai42.ai/reference/catalog
Install
Requires Python 3.13+. Install from PyPI into the environment that runs the server:
uv add tai42-storage-github
Or from source — clone this repo and add it as an editable dependency; the
tai42-* dependencies resolve in-tree from the workspace.
git clone https://github.com/tai42ai/tai42 # next to your app checkout
cd /path/to/your/app
uv add --editable ../tai42/plugins/storage-github
Use
The backend is loaded by import side-effect: importing the tai42_storage_github
package runs its @tai42_app.storage.register_storage decorator, making
GithubStorage the active storage provider. Point a manifest's storage_module
at the package:
# manifest.yml
storage_module: tai42_storage_github
Configuration
Configure through STORAGE_GITHUB_-prefixed environment variables:
| Variable | Default | Description |
|---|---|---|
STORAGE_GITHUB_USERNAME |
— | Repository owner (user or org). |
STORAGE_GITHUB_REPO |
— | Repository name. |
STORAGE_GITHUB_BRANCH |
main |
Branch to read/write. |
STORAGE_GITHUB_TOKEN |
— | Access token; omit for a public repository. |
STORAGE_GITHUB_TIMEOUT_TOTAL |
15.0 |
HTTP timeout (seconds). |
STORAGE_GITHUB_MAX_CONNECTIONS |
200 |
Connection-pool ceiling. |
STORAGE_GITHUB_MAX_KEEPALIVE_CONNECTIONS |
50 |
Keep-alive pool size. |
STORAGE_GITHUB_KEEPALIVE_EXPIRY |
300.0 |
Keep-alive expiry (seconds). |
The token is held as a SecretStr; it authenticates requests but never appears in
a log line, repr, or error message.
Reads and writes
- Reads (
load,load_bytes) are one Contents API call each at the configured branch, asking for the raw form (Accept: application/vnd.github.raw+json), which serves every byte of a file of any size. A directory answers its JSON listing and reads asFileNotFoundError. - Ranged reads (
load_range) resolve the branch's head commit through the Git refs API, then readraw.githubusercontent.comat that commit with aRangeheader andAccept-Encoding: identity(the raw host compresses, and a compressed answer ranges over the encoded bytes): two calls, one of them REST. Reading at the commit, never at the branch name, means a ranged read answers the branch's current bytes, not a cached copy. An answer that ignores the range or is still encoded raises; there is no whole-object read in its place. A zero-length range sends no request. - Rate limit: GitHub allows 60 REST calls an hour without a token and 5000
with one;
stat,loadandload_bytescost one REST call each andload_rangeone. - Writes (
upload,upload_bytes) go through the Contents API (base64) and are capped conservatively at 1 MiB. A larger payload raisesValueErrorbefore the request rather than truncating; the API's own422is the backstop. - Listing uses the recursive Git Trees API (one request) and raises loudly on
a
truncatedresponse rather than acting on a partial listing. statis one Contents API call asking for the object form (Accept: application/vnd.github.object+json): it answers the file'ssize, and inferscontent_typefrom the path suffix — GitHub stores no per-object content-type. A missing path and a directory raiseFileNotFoundError.
Development
uv venv --python 3.13
uv pip install --no-sources --group dev --editable .
uv run --no-sync pytest --cov --cov-report=term-missing
uv run --no-sync ruff check .
uv run --no-sync ruff format --check .
uv run --no-sync pyright
License
Apache-2.0. See LICENSE and NOTICE.
Metadata
Release files for tai42-storage-github 2.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| tai42_storage_github-2.0.1.tar.gz | 27.7 kB | Without provenance | Details for tai42_storage_github-2.0.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| tai42_storage_github-2.0.1-py3-none-any.whl | Python 3 | none | any | Details for tai42_storage_github-2.0.1-py3-none-any.whl |
Total release size: 45.4 kB
Release files / tai42_storage_github-2.0.1.tar.gz
| Download URL | tai42_storage_github-2.0.1.tar.gz |
|---|---|
| Size | 27.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
188abc2c773d1b45ca489010e03090ebb29c2c09af312500e635866080d08100
|
|
BLAKE2b-256 checksum How to use checksums |
13c3171220655ecf72e546566c70fb03194fa16b483d2e566431ea1b858e6c11
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Release files / tai42_storage_github-2.0.1-py3-none-any.whl
| Download URL | tai42_storage_github-2.0.1-py3-none-any.whl |
|---|---|
| Size | 17.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
3fc41260b6e3212bcd4e5c87f4454ec174fd3376f57680aafdcc42a9f77b4455
|
|
BLAKE2b-256 checksum How to use checksums |
859d17c2b5394293d911c2deafac6e58479960ba0c0f0168196e03a78f30a939
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Security for tai42-storage-github 2.0.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.