molq

Unified job queue — one submission API for local, SLURM, PBS, and LSF

CI PyPI Python License Ruff

Documentation  ·  Quick start  ·  Ecosystem

molq is a unified job queue for Python workloads that need the same submission API on a laptop, a workstation, or an HPC cluster. A Cluster says where jobs run, a Submitor tracks how they progress — and the same code runs against local subprocesses or remote schedulers over SSH.

Under active development. Public APIs may change between minor releases.

Capabilities

Module Capability
cluster Cluster — destination spec: scheduler kind × transport × scheduler options, plus live queue snapshots
submitor Submitor + JobHandle — single entry point for submitting, tracking, and waiting on jobs
scheduler Scheduler protocol plus one module per backend (shell, slurm, pbs, lsf), each routing shell calls through a transport and owning its own directive and dependency syntax
transport LocalTransport / SshTransport — runs shell and file ops here or on a remote host via OpenSSH, sharing one multiplexed connection
store JobStore — SQLite persistence with WAL mode, UUID job identity, schema versioning, and v1 auto-migration
validation Checks a request against the backend's declared capability matrix before submitting
jobpaths Job directory layout, log path resolution, script staging, and manifests
dependencies Resolves molq job ids + conditions into the backend's submit syntax
retry / retention Retry eligibility and backoff; expiry of job directories and old records
artifacts Pulls logs and job directories back from the cluster
reconciler JobReconciler — batch-queries schedulers, diffs against the store, syncs job state
monitor Blocking waits and polling engine driven by pluggable strategies
strategies Pluggable polling strategies; exponential backoff by default
callbacks EventBus — synchronous pub/sub for job lifecycle events with handler isolation
models Job data models — JobRecord, RetryPolicy, RetentionPolicy, JobDependency, SubmitorDefaults
types Frozen value types — Memory, Duration, Script, JobResources, JobScheduling, JobExecution
options Per-scheduler frozen option dataclasses (Local, Slurm, PBS, LSF) — no untyped dicts
config Profile and config loading from molcfg (~/.molcrafts/molq/config/config.toml, or MOLCRAFTS_HOME)
plugin MolqPlugin host — official builtins + third-party entry points (molq.plugins)
plugins Official plugins (e.g. nerve → local Nerve menu-bar status; fail-open)
workspace Workspace / Project — directory handles over a cluster's filesystem (local or remote)
ssh_config Surfaces ~/.ssh/config hosts as cluster candidates
serde Serialization helpers for stored requests and config-driven values
errors Unified MolqError exception hierarchy with typed context
status JobState enum with terminal-state semantics
merge Pure function that merges per-submit parameters with Submitor defaults
dashboard Full-screen terminal dashboard for monitoring runs and jobs
testing FakeScheduler and make_submitor for tests and runnable examples without a real cluster
cli Typer + Rich CLI: jobs (submit/list/status/logs/cancel/…), live (watch/monitor/daemon), setup (clusters/workspace/plugins)

Install

pip install molcrafts-molq

Requires Python 3.12+. Depends on typer, rich, molcrafts-mollog, and molcrafts-molcfg.

Quick start

import molq as mq

# Cluster = destination (where to run). Submitor = lifecycle (how jobs are tracked).
cluster = mq.Cluster("devbox", "local")

with mq.Submitor(target=cluster) as queue:
    handle = queue.submit_job(
        argv=["python", "-c", "print('hello from molq')"]
    )
    record = handle.wait()

print(record.state)

Swap to a cluster by changing one line — mq.Cluster("hpc", "slurm", host="user@hpc.example.com") — and the rest of the code is unchanged. See the documentation for retries, dependencies, profiles, and the CLI.

Documentation

MolCrafts ecosystem

Project Role
molpy Python toolkit — the shared molecular data model & workflow layer
molrs Rust core — molecular data structures & compute kernels (native + WASM)
molpack Packmol-grade molecular packing (Rust + Python)
molvis WebGL molecular visualization & editing
molexp Workflow & experiment-management platform
molnex Molecular machine-learning framework
molq Unified job queue — local / SLURM / PBS / LSF — this repo
molcfg Layered configuration library
mollog Structured logging, stdlib-compatible
molhub Molecular dataset hub
molmcp MCP server for the ecosystem
molrec Atomistic record specification

Contributing

Issues and pull requests are welcome — see CONTRIBUTING.md for development setup.

License

MIT — see LICENSE.

Crafted with 💚 by MolCrafts

Project links

Data verified by PyPI on Aug 13, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

Owner

PyPI data
Data sourced directly from PyPI's database.

Credits

Author: Roy Kid

GitHub Statistics

Data verified by PyPI on Aug 13, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

MIT
View SPDX License List

Requires

Python >=3.12

Provides Extra

dev docs

Tags

job queue cluster slurm hpc scheduler

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

molcrafts_molq-0.8.1.tar.gz (176.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

molcrafts_molq-0.8.1-py3-none-any.whl (127.5 kB view details)

Uploaded Python 3

File details

Details for the file molcrafts_molq-0.8.1.tar.gz.

File metadata

  • Download URL: molcrafts_molq-0.8.1.tar.gz
  • Upload date:
  • Size: 176.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for molcrafts_molq-0.8.1.tar.gz
Algorithm Hash digest
SHA256 41e42d12c94823ca68aab4b6037888c66dc93d2048433eb02c836c212f47ad28
MD5 ba8310f4cd401cba082f0b919fcf55cb
BLAKE2b-256 2a1f708a1fd2785496b26f5abb8d8aaf40c6b42f8aea83c247b0f324f17fd435

See more details on using hashes here.

Provenance

The following attestation bundles were made for molcrafts_molq-0.8.1.tar.gz:

Publisher: release.yml on MolCrafts/molq

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file molcrafts_molq-0.8.1-py3-none-any.whl.

File metadata

  • Download URL: molcrafts_molq-0.8.1-py3-none-any.whl
  • Upload date:
  • Size: 127.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for molcrafts_molq-0.8.1-py3-none-any.whl
Algorithm Hash digest
SHA256 d019d972f24168b1980da3dca65d6403bfc4daf7a6209d5bb28f4afd00d8b218
MD5 620fcd9d1a4e5e2e8ce0aacdfa5b44a0
BLAKE2b-256 a57585f02fff6f82a4d1d32e05fcfc428d201f29dd446ecf14f655b763119a5d

See more details on using hashes here.

Provenance

The following attestation bundles were made for molcrafts_molq-0.8.1-py3-none-any.whl:

Publisher: release.yml on MolCrafts/molq

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page