This release is a pre-release and may not be stable for production use.
theblob
An S3-compatible object store written in Python. It starts as one command on one machine, and is designed to grow into a redundant cluster.
It is meant to replace self-hosted MinIO: single-server installs first, then clusters (not there yet).
Quick start
uv sync
uv run theblob serve --data-dir ./data
The first start creates an access key and prints it once. Use it with any S3 client:
import boto3
s3 = boto3.client(
"s3",
endpoint_url="http://127.0.0.1:9000",
aws_access_key_id="...",
aws_secret_access_key="...",
region_name="us-east-1",
)
s3.create_bucket(Bucket="my-bucket")
s3.put_object(Bucket="my-bucket", Key="hello.txt", Body=b"hello")
Other commands: theblob key create|list|delete|grant|revoke and theblob fsck. Run theblob serve --help for the server options, including TLS and virtual-hosted addressing.
Presigned URLs need SigV4. boto3 presigns with SigV2 by default, which theblob refuses, so create the client with config=Config(signature_version="s3v4").
Moving from MinIO or Garage
With a rclone remote for each store (provider = Other, force_path_style = true), and the bucket created on theblob:
rclone sync --metadata old:my-bucket theblob:my-bucket
rclone check old:my-bucket theblob:my-bucket
tests/bench/run_migration.py checks this from both: the bytes, Content-Type, Cache-Control and x-amz-meta-* arrive, including multipart objects and unusual keys. These don't, and need setting again: object tags, ACLs, lifecycle rules, CORS, keys, and any version but the current one. rclone also adds mtime and btime to the metadata. An object's time on theblob is when it was copied.
In a container
docker build -t theblob .
docker run -d --name theblob -p 9000:9000 -v theblob-data:/data theblob
docker logs theblob # the first start prints an access key once
Set --workers after the image name if the default, one per CPU, uses more memory than you want. The data format is upgraded when a newer version starts: back up first, with theblob backup. A store written by an earlier release is tested to open in this one, from samples in tests/upgrade/stores/.
Access keys
A key has full access unless it is created as scoped. A scoped key has no rights until it is granted some, per bucket, at one of three levels: read, write (which includes read) or admin (which also allows deleting the bucket).
uv run theblob key create --scoped --data-dir ./data
uv run theblob key grant ACCESS_KEY my-bucket read --data-dir ./data
uv run theblob key revoke ACCESS_KEY my-bucket --data-dir ./data
A grant or a revocation applies from the next request. A scoped key cannot create buckets, sees only its own buckets when listing, and cannot tell a bucket it has no grant on from one that does not exist. Grants disappear with their bucket, so a bucket deleted and created again starts with none.
Development
make test # the whole test pyramid
make lint # ruff, ty, pyrefly, mypy
Tests are in tests/: a_unit for pure logic, b_integration for real SQLite and files, c_e2e for real S3 clients (boto3, AWS CLI, rclone, s5cmd) against a real server over HTTP and HTTPS. The rclone, s5cmd and AWS CLI v2 tests are skipped if those tools are not installed.
License
Apache-2.0: see LICENSE.
Metadata
Release files for theblob 0.4.0.dev1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| theblob-0.4.0.dev1.tar.gz | 95.3 kB | Without provenance | Details for theblob-0.4.0.dev1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| theblob-0.4.0.dev1-py3-none-any.whl | Python 3 | none | any | Details for theblob-0.4.0.dev1-py3-none-any.whl |
Total release size: 204.1 kB
Release files / theblob-0.4.0.dev1.tar.gz
| Download URL | theblob-0.4.0.dev1.tar.gz |
|---|---|
| Size | 95.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9dcc3f2a1ce0090000d75d73ca4b99b9eba1e7f1cca0178346d2f293be0ef985
|
|
BLAKE2b-256 checksum How to use checksums |
b61476cfd3af8e295701ab4a50059fda2796f9a2490eeda4062e8673aab03845
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.4 {"installer":{"name":"uv","version":"0.13.4","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / theblob-0.4.0.dev1-py3-none-any.whl
| Download URL | theblob-0.4.0.dev1-py3-none-any.whl |
|---|---|
| Size | 108.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
aaa1febcc85445a9185c0bae103e8b34f8cae90457578a6a4e964cf70b63d35c
|
|
BLAKE2b-256 checksum How to use checksums |
752e1f098e07dcde1ea74b9be5bb6a1da903af1f024da032d567982ccb3d4111
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.4 {"installer":{"name":"uv","version":"0.13.4","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for theblob 0.4.0.dev1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.