Matyan Backend

REST API and workers for the Matyan experiment-tracking stack (fork of Aim). Serves reads and control operations from FoundationDB; consumes ingestion and control events from Kafka; uses S3/GCS/Azure for artifact blobs. The UI talks to this API; training clients send data via the frontier, which publishes to Kafka consumed by these workers.

Layout

• src/matyan_backend/ — Python package: FastAPI app (app.py), API routes under api/ (runs, experiments, tags, projects, dashboards, reports, streaming), storage/ (FDB + S3/GCS/Azure), workers/ (ingestion + control Kafka consumers), jobs/ (FDB lock, used by CLI cleanup commands), backup/ (export/restore), CLI in cli.py.
• Entrypoints: matyan-backend start (API server, default port 53800), matyan-backend ingest-worker, matyan-backend control-worker; plus one-off CLI commands (reindex, backup, restore, finish-stale, cleanup-orphan-blobs, cleanup-tombstones, convert tensorboard).

Prerequisites

• Python 3.12+. The package uses uv in the repo: uv run matyan-backend or install then matyan-backend CLI.
• Runtime dependencies: FoundationDB (cluster file), Kafka (for workers), blob store. For local dev, typically run FDB + Kafka + S3 (RustFS) via docker-compose.

Run

• API server: uv run matyan-backend start (or matyan-backend start). Options: --host, --port (defaults: 0.0.0.0, 53800). API is under /api/v1; health at /health/ready/, /health/live/, metrics at /metrics/ when enabled.
• Workers: uv run matyan-backend ingest-worker and uv run matyan-backend control-worker. Both require Kafka and FDB; ingestion worker also writes to FDB and reads blob storage config for blob references.
• CLI (one-off): reindex (rebuild indexes), backup / restore, finish-stale, cleanup-orphan-blobs, cleanup-tombstones. See the backend CLI help (matyan-backend cleanup-orphan-blobs --help, matyan-backend cleanup-tombstones --help) and References — CLI for all options. Cleanup commands are intended for CronJobs or cron; use --dry-run to preview and --lock-ttl-seconds for FDB-based single-run locking. Optional: convert tensorboard to convert TensorBoard logs to backup format.

Configuration (environment variables)

Source of truth: config.py.

Production configuration

See docs/PRODUCTION_CONFIG.md for enabling production mode (MATYAN_ENVIRONMENT=production), required overrides, and supplying secrets via env or a secrets backend.

Deployment

• Docker: Build the backend image (context from repo root); run API and workers as separate processes or containers.
• Kubernetes/Helm: The chart in deploy/helm/matyan deploys the backend API, ingestion worker, and control worker as separate Deployments; optional CronJobs for cleanup-orphan-blobs and cleanup-tombstones. Configure FDB, blob storage (S3, GCS, Azure), and Kafka via chart values; see the chart README. Set MATYAN_ENVIRONMENT=production and required env for production.

Related

• UI: matyan-ui calls this backend REST API.
• Frontier: matyan-frontier publishes to Kafka; backend workers consume.
• API models: matyan-api-models shared types (Kafka messages, run creation, etc.).
• Monorepo: This package lives under extra/matyan-backend in the matyan-core repo.