whatsnext 0.0.2

A job queue and task management system with client-server architecture

Warning This is a private project in early development. It has not been security audited and is strongly advised against running on public-facing servers. Use only in trusted, internal environments. You may also experience several bugs at this time due to the early development stage of this project.

A simple, powerful job queue for Python applications.

WhatsNext helps you manage and execute background jobs across multiple machines. Add jobs to a queue, and workers pick them up and run them.

Features

• Simple: Just a few lines of code to get started
• Reliable: Jobs are stored in PostgreSQL, so nothing gets lost
• Scalable: Run multiple workers on different machines
• Flexible: Works with any Python code, SLURM clusters, or Kubernetes
• Lightweight Client: No database, no caching, no background processes - the client is pure HTTP and runs in restricted environments

Quick Example

from whatsnext.api.client import Server, Job, Client, CLIFormatter

# Connect and get a project
server = Server("localhost", 8000)
project = server.get_project("my-experiments")

# Add a job to the queue
project.append_queue(Job(
    name="experiment-1",
    task="train-model",
    parameters={"learning_rate": 0.01, "epochs": 100}
))

# Create a worker and process jobs
formatter = CLIFormatter(executable="python", script="train.py")
client = Client(
    entity="ml-team",
    name="gpu-worker-1",
    project=project,
    formatter=formatter
)
client.work()

Installation

# Install with uv (recommended)
uv add whatsnext[all]

# Or with pip
pip install whatsnext[all]

Requirements

• Python 3.10+
• PostgreSQL 14+

Running the Server

1. Create a .env file with your database configuration:

database_hostname=localhost
database_port=5432
database_user=postgres
database_password=postgres
database_name=whatsnext

2. Start the server:

uvicorn whatsnext.api.server.main:app --reload

3. Visit http://localhost:8000/docs for interactive API documentation.

Documentation

Full documentation is available at the project docs site. Build locally with:

uv run mkdocs serve

Development

# Install dependencies
uv sync --all-extras --all-groups

# Run quality checks
uv run ruff format .
uv run ruff check . --fix
uv run ty check whatsnext
uv run pytest -v

License

AGPL-3.0 - See LICENSE for details.