molq

Unified job queue — one submission API for local, SLURM, PBS, and LSF

CI PyPI Python License Ruff

Documentation  ·  Quick start  ·  Ecosystem

molq is a unified job queue for Python workloads that need the same submission API on a laptop, a workstation, or an HPC cluster. A Cluster says where jobs run, a Submitor tracks how they progress — and the same code runs against local subprocesses or remote schedulers over SSH.

Under active development. Public APIs may change between minor releases.

Capabilities

Module Capability
cluster Cluster — destination spec: scheduler kind × transport × scheduler options, plus live queue snapshots
submitor Submitor + JobHandle — single entry point for submitting, tracking, and waiting on jobs
scheduler Scheduler protocol plus one module per backend (shell, slurm, pbs, lsf), each routing shell calls through a transport and owning its own directive and dependency syntax
transport LocalTransport / SshTransport — runs shell and file ops here or on a remote host via OpenSSH, sharing one multiplexed connection
store JobStore — SQLite persistence with WAL mode, UUID job identity, schema versioning, and v1 auto-migration
validation Checks a request against the backend's declared capability matrix before submitting
jobpaths Job directory layout, log path resolution, script staging, and manifests
dependencies Resolves molq job ids + conditions into the backend's submit syntax
retry / retention Retry eligibility and backoff; expiry of job directories and old records
artifacts Pulls logs and job directories back from the cluster
reconciler JobReconciler — batch-queries schedulers, diffs against the store, syncs job state
monitor Blocking waits and polling engine driven by pluggable strategies
strategies Pluggable polling strategies; exponential backoff by default
callbacks EventBus — synchronous pub/sub for job lifecycle events with handler isolation
models Job data models — JobRecord, RetryPolicy, RetentionPolicy, JobDependency, SubmitorDefaults
types Frozen value types — Memory, Duration, Script, JobResources, JobScheduling, JobExecution
options Per-scheduler frozen option dataclasses (Local, Slurm, PBS, LSF) — no untyped dicts
config Profile and config loading from molcfg (~/.molcrafts/molq/config/config.toml, or MOLCRAFTS_HOME)
plugin MolqPlugin host — official builtins + third-party entry points (molq.plugins)
plugins Official plugins (e.g. nerve → local Nerve menu-bar status; fail-open)
workspace Workspace / Project — directory handles over a cluster's filesystem (local or remote)
ssh_config Surfaces ~/.ssh/config hosts as cluster candidates
serde Serialization helpers for stored requests and config-driven values
errors Unified MolqError exception hierarchy with typed context
status JobState enum with terminal-state semantics
merge Pure function that merges per-submit parameters with Submitor defaults
dashboard Full-screen terminal dashboard for monitoring runs and jobs
testing FakeScheduler and make_submitor for tests and runnable examples without a real cluster
cli Typer + Rich CLI: jobs (submit/list/status/logs/cancel/…), live (watch/monitor/daemon), setup (clusters/workspace/plugins)

Install

pip install molcrafts-molq

Requires Python 3.12+. Depends on typer, rich, molcrafts-mollog, and molcrafts-molcfg.

Quick start

import molq as mq

# Cluster = destination (where to run). Submitor = lifecycle (how jobs are tracked).
cluster = mq.Cluster("devbox", "local")

with mq.Submitor(target=cluster) as queue:
    handle = queue.submit_job(
        argv=["python", "-c", "print('hello from molq')"]
    )
    record = handle.wait()

print(record.state)

Swap to a cluster by changing one line — mq.Cluster("hpc", "slurm", host="user@hpc.example.com") — and the rest of the code is unchanged. See the documentation for retries, dependencies, profiles, and the CLI.

Documentation

MolCrafts ecosystem

Project Role
molpy Python toolkit — the shared molecular data model & workflow layer
molrs Rust core — molecular data structures & compute kernels (native + WASM)
molpack Packmol-grade molecular packing (Rust + Python)
molvis WebGL molecular visualization & editing
molexp Workflow & experiment-management platform
molnex Molecular machine-learning framework
molq Unified job queue — local / SLURM / PBS / LSF — this repo
molcfg Layered configuration library
mollog Structured logging, stdlib-compatible
molhub Molecular dataset hub
molmcp MCP server for the ecosystem
molrec Atomistic record specification

Contributing

Issues and pull requests are welcome — see CONTRIBUTING.md for development setup.

License

MIT — see LICENSE.

Crafted with 💚 by MolCrafts

Project links

Data verified by PyPI on Aug 7, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.8.1)

Owner

PyPI data
Data sourced directly from PyPI's database.

Credits

Author: Roy Kid

GitHub Statistics

Data verified by PyPI on Aug 7, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

MIT
View SPDX License List

Requires

Python >=3.12

Provides Extra

dev docs

Tags

job queue cluster slurm hpc scheduler

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

molcrafts_molq-0.7.0.tar.gz (172.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

molcrafts_molq-0.7.0-py3-none-any.whl (124.1 kB view details)

Uploaded Python 3

File details

Details for the file molcrafts_molq-0.7.0.tar.gz.

File metadata

  • Download URL: molcrafts_molq-0.7.0.tar.gz
  • Upload date:
  • Size: 172.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for molcrafts_molq-0.7.0.tar.gz
Algorithm Hash digest
SHA256 0a9ab0ce27ecbc962903a31814876144c39d94d3b47986fe51c4e05d5ce96b04
MD5 074b4ed1818cad6e08c0f11da70e9b7c
BLAKE2b-256 2307ec4b2f3cfe5bfa333d20acbed6255cead078b724b0470c79b70a9fbc06fc

See more details on using hashes here.

Provenance

The following attestation bundles were made for molcrafts_molq-0.7.0.tar.gz:

Publisher: release.yml on MolCrafts/molq

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file molcrafts_molq-0.7.0-py3-none-any.whl.

File metadata

  • Download URL: molcrafts_molq-0.7.0-py3-none-any.whl
  • Upload date:
  • Size: 124.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for molcrafts_molq-0.7.0-py3-none-any.whl
Algorithm Hash digest
SHA256 2e23bd03519b4a7bd50741f61f61bb09d4acdafae8d4d5764760cb010fa89901
MD5 e306a72a42460654f016d73bcfce3d61
BLAKE2b-256 d0381df8a468615336f599a619e8843306ffa9a6599c55af0727213aa147d818

See more details on using hashes here.

Provenance

The following attestation bundles were made for molcrafts_molq-0.7.0-py3-none-any.whl:

Publisher: release.yml on MolCrafts/molq

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.8.1

2 files

0.8.0

2 files

This release

0.7.0 This release

2 files

0.6.0

2 files

0.4.0

2 files

0.2.0

2 files

0.1.0

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page