tiny-object-storage 0.1.1

A lightweight S3-compatible object storage server for local development and learning.

🪣 tiny-object-storage

A lightweight, S3-compatible object storage server for local development, testing, and learning.

---

📖 Overview

tiny-object-storage is a fast, minimalistic object storage server written in Python using FastAPI. It acts as a local emulator that mimics the core functionality of Amazon S3, allowing developers to test S3-integrated applications locally without relying on external cloud services or heavy containers like MinIO.

✨ Features

• S3 Compatibility: Supports core S3 XML responses and basic HTTP methods (GET, PUT, HEAD, DELETE).
• Bucket Management: Create, list, and delete buckets.
• Object Operations: Upload, download, metadata lookup, listing (list-type=2), and deletion.
• Persistent Storage: Filesystem-based persistence with dedicated metadata and data separation.
• Developer Experience: Interactive Swagger UI and OpenAPI schema built-in.
• Enterprise Code Quality: Enforced static typing (Mypy strict), linting (Ruff), and security analysis (Bandit).
• Flexible Configuration: Configurable via CLI arguments, environment variables, or .env files.

---

🚀 Quick Start

Installation

From PyPI (Recommended) Install the package globally or in a virtual environment using pip:

pip install tiny-object-storage

From Source (Development) Clone the repository and install it in a virtual environment:

git clone https://github.com/EhsanBitaraf/tiny-object-storage.git
cd tiny-object-storage
python -m venv .venv

# On Linux/macOS:
source .venv/bin/activate
# On Windows:
.venv\Scripts\activate

# Install the application and development dependencies
pip install -e ".[dev]"

Running the Server

Start the server using the built-in CLI:

tos-server

Or run it directly with uvicorn:

uvicorn tiny_object_storage.main:app --reload --host 0.0.0.0 --port 9000

---

⚙️ Configuration

The server reads configuration in the following priority (highest to lowest):

1. Command-line arguments
2. Environment variables
3. .env file

CLI Usage Examples

# View all available options
tos-server --help

# Load from a specific .env file
tos-server --env-file .env.custom

# Override host and port
tos-server --host 127.0.0.1 --port 9001 --reload

# Customize storage directories and log level
tos-server --data-dir ./storage_data --log-dir ./logs --log-level debug

---

🔌 S3 Client Compatibility

tiny-object-storage acts as a local emulator and has been successfully verified against the following official SDKs and tools:

• MinIO Python SDK
• Boto3 (AWS SDK for Python)
• AWS CLI

Note: For official SDKs to interact locally, configure them with secure=False (or --no-verify-ssl) and point the endpoint_url to http://127.0.0.1:9000 with any dummy credentials.

Sample scripts for verifying SDK interactions are included in the repository root (e.g., test_boto3_sdk.py, test_minio_sdk.py).

---

🛠 Testing & Code Quality

The project includes an extensive test suite built with pytest achieving 100% test pass rate over 63 edge-case scenarios.

Running Tests

pytest

(Optionally, use pytest --cov=src to generate a coverage report).

Code Quality Tools

We use pre-commit to maintain our code quality. The suite includes:

• Ruff: For ultra-fast linting and code formatting.
• Mypy: For strict static type checking.
• Bandit: For security vulnerability scanning.

To run checks manually on all files:

pre-commit run --all-files

---

🌐 API Endpoints (Examples)

After starting the server, visit /docs for the interactive Swagger UI. Below are standard curl examples targeting common S3 actions:

Health check

curl http://127.0.0.1:9000/health

Create a bucket

curl -X PUT http://127.0.0.1:9000/photos

Upload an object

curl -X PUT --data-binary @"cat.jpg" http://127.0.0.1:9000/photos/cat.jpg

Download an object

curl http://127.0.0.1:9000/photos/cat.jpg --output cat.jpg

List objects (S3 Format)

curl "http://127.0.0.1:9000/photos?list-type=2"

Debug endpoints (JSON format)

curl http://127.0.0.1:9000/_debug/buckets
curl http://127.0.0.1:9000/_debug/photos

---

🤝 Contributing

Contributions are welcome and highly appreciated! Please review our Contributing Guidelines to get started. By participating in this project, you agree to abide by the Code of Conduct.

🔒 Security

For instructions on reporting security vulnerabilities, please refer to our Security Policy.

📜 License

This project is licensed under the MIT License - see the LICENSE file for details.