mmar-files

Local file storage abstraction with content deduplication for MAESTRO.

Installation

Base functionality (local/basic storage only):

pip install mmar-files

With optional S3 support:

pip install mmar-files[s3]

Testing

Unit Tests

Run unit tests (these use mocks and don't require external services):

make test
# or
uv run pytest

Integration Tests

Integration tests use MinIO (S3-compatible storage) via Docker to test real S3 functionality.

Quick start - Run everything:

make test-integration-full

This will:

1. Start MinIO in Docker
2. Wait for it to be ready
3. Run integration tests
4. Stop MinIO

Manual control:

# Start MinIO in background
make minio-up

# Run integration tests (MinIO must be running)
make test-integration

# View MinIO logs
make minio-logs

# Stop MinIO
make minio-down

MinIO credentials:

• Console: http://localhost:9001
• S3 Endpoint: http://localhost:9000
• Access Key: minioadmin
• Secret Key: minioadmin

The integration tests can be configured via environment variables:

• MINIO_ENDPOINT (default: http://localhost:9000)
• MINIO_ACCESS_KEY (default: minioadmin)
• MINIO_SECRET_KEY (default: minioadmin)

Assumed migration

Before:

from mmar_mapi import FileStorage, FileStorageAPI
from mmar_mimpl import SettingsModel

...
class Config(SettingsModel):
    files_dir: str = "/mnt/data/maestro/files"
...
config: Config = ...
file_storage: FileStorageAPI = FileStorage(config.files_dir)

After

from mmar_mimpl import SettingsModel

from mmar_files import FilesConfigDir, FileStorageAPI, create_file_storage

...
class Config(SettingsModel):
    files: FilesConfig = Field(default=FilesConfigDir(dir="/mnt/data/maestro/files"))
...
config: Config = ...
file_storage: FileStorageAPI = create_file_storage(config.files)

License

MIT