s2-bucket-proxy 0.1.0

C engine-based bucket proxy server for S2 storage — streaming, decryption, and downloads

S2 Bucket Proxy

C engine-powered proxy server for streaming and downloading encrypted files from S2 storage buckets. Decrypts AES-256-CBC chunks on-the-fly and serves them over HTTP with full Range request support.

Features

• C Engine — high-performance HTTP server (libmicrohttpd, epoll, thread pool)
• AES-256-CBC Decryption — real-time chunk decryption via OpenSSL
• Full Range Support — byte-range seeking for video/audio players
• ETag & Conditional Requests — If-None-Match, If-Range, 304 Not Modified
• CORS — full preflight support for browser-based streaming
• Multi-Replica Fallback — automatic retry across chunk replicas
• SQLite3 Config — persistent config, buckets, and stats
• Docker Support — production-ready container with env var configuration
• Interactive CLI — terminal UI for managing buckets and server

Install System Dependencies

The C engine compiles from source on pip install, so you need build tools and libraries installed first.

Ubuntu / Debian

sudo apt update
sudo apt install build-essential libcurl4-openssl-dev libssl-dev libmicrohttpd-dev pkg-config

Fedora / RHEL / CentOS

sudo dnf install gcc make libcurl-devel openssl-devel libmicrohttpd-devel pkgconf-pkg-config

Arch Linux / Manjaro

sudo pacman -S base-devel curl openssl libmicrohttpd pkgconf

macOS (Homebrew)

brew install curl openssl libmicrohttpd pkg-config

Termux (Android)

pkg update
pkg install clang make libcurl openssl libmicrohttpd pkg-config

Alpine Linux (Docker)

apk add build-base curl-dev openssl-dev libmicrohttpd-dev pkgconf

Installation

From PyPI

pip install s2-bucket-proxy

The C engine compiles automatically during install (~30s). Make sure system dependencies above are installed first.

From Source

git clone <repo>
cd bucket_proxy/v2
pip install -e .

Docker (no system deps needed)

docker build -t s2-bucket-proxy .
docker run -p 8001:8001 \
  -e S2_MAIN_SERVER=https://your-server.com \
  -e S2_TOKEN=bkt_your_token_here \
  -v proxy_data:/data \
  s2-bucket-proxy

Or with Docker Compose:

S2_MAIN_SERVER=https://your-server.com S2_TOKEN=bkt_xxx docker compose up -d

Quick Start

Interactive Mode

s2-proxy
# → proxy> add bkt_your_token_here
# → proxy> start

Non-Interactive

# Add bucket + start in one command
s2-proxy start --server https://your-server.com --token bkt_xxx

# Add a bucket
s2-proxy add bkt_your_token_here

# Start as daemon
s2-proxy start -d

# Stop daemon
s2-proxy stop

Docker Container Management

# Add a bucket to running container
docker exec s2_bucket_proxy bucket_proxy_engine --docker add-bucket --token bkt_xxx

# List buckets
docker exec s2_bucket_proxy bucket_proxy_engine --docker list-buckets

# View status
docker exec s2_bucket_proxy bucket_proxy_engine --docker status

API Endpoints

Method	Route	Description
GET	/health	Health check
GET	/status	Server stats + buckets
GET	/buckets	List registered buckets
GET	/{bucket}/files?limit=N&offset=N	List files in bucket
GET	/{bucket}/stream/{file_id}	Stream file (Range supported)
GET	/{bucket}/download/{file_id}	Download file
HEAD	/{bucket}/stream/{file_id}	Get file metadata
HEAD	/{bucket}/download/{file_id}	Get download metadata
OPTIONS	*	CORS preflight

Streaming

# Full file
curl http://localhost:8001/mybucket/stream/file123

# Range request (video seeking)
curl -H "Range: bytes=1000000-2000000" http://localhost:8001/mybucket/stream/file123

# Private file
curl http://localhost:8001/mybucket/stream/file123?token=ftk_xxx

Download

curl -OJ http://localhost:8001/mybucket/download/file123

Architecture

┌──────────────────────────────────────────────────┐
│                    Python CLI                     │
│  s2-proxy (interactive) / s2-proxy start (cmd)   │
└──────────────────┬───────────────────────────────┘
                   │ subprocess
┌──────────────────▼───────────────────────────────┐
│              C Engine (bucket_proxy_engine)       │
│                                                   │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐       │
│  │ server.c │  │  api.c   │  │ crypto.c │       │
│  │ HTTP svr │  │ curl GET │  │ AES-256  │       │
│  │ epoll    │  │ validate │  │ decrypt  │       │
│  │ Range    │  │ info     │  │ PKCS7    │       │
│  │ CORS     │  │ download │  │          │       │
│  └──────────┘  └──────────┘  └──────────┘       │
│                                                   │
│  ┌──────────┐  ┌──────────┐                      │
│  │  db.c    │  │  main.c  │                      │
│  │ SQLite3  │  │ CLI args │                      │
│  │ config   │  │ daemon   │                      │
│  │ stats    │  │ docker   │                      │
│  └──────────┘  └──────────┘                      │
└──────────────────────────────────────────────────┘

Docker Environment Variables

Variable	Description	Default
S2_MAIN_SERVER	Main storage server URL	http://localhost:8000
S2_PORT	Proxy listen port	8001
S2_TOKEN	Bucket token (auto-registers)	—
S2_DB_PATH	SQLite database path	/data/proxy.db

License

MIT