Skip to main content

Worker CLI for the JobDistributor distributed job system

Project description

Worker client (jd-worker)

Workers run jd_worker, which requests jobs from the job server, executes your entry script with parameters as CLI flags, sends heartbeats, and reports DONE / ABORTED.

Full behaviour, environment variables, local paths (~/jd_data/…), and library helpers (jd_upload, checkpoints, jd_job_dir, …) are documented in docs/jd-worker.md.


Install

From the repo root (editable):

cd job-distributor/client
python3 -m venv venv
source venv/bin/activate   # Windows: venv\Scripts\activate
pip install -e .

Or install from GitHub (see docs/jd-worker.md for branch / subdirectory).


Run

jd_worker expId=<experiment_id> entry_script=<your_script.py>

Optional: server=, port=, machine_type=, once=true, etc. — see jd_worker help or docs/jd-worker.md.


Example workload

For an end-to-end ML-style example (MNIST tuning), see
MNIST-parameter-tuning. Point entry_script at that repo’s training script and align expId with the server.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

jd_worker-1.1.0.tar.gz (13.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

jd_worker-1.1.0-py3-none-any.whl (13.7 kB view details)

Uploaded Python 3

File details

Details for the file jd_worker-1.1.0.tar.gz.

File metadata

  • Download URL: jd_worker-1.1.0.tar.gz
  • Upload date:
  • Size: 13.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for jd_worker-1.1.0.tar.gz
Algorithm Hash digest
SHA256 e8abf9af2b93991fb07f1efa6dad11758f519ed52e590036280373380543bbd9
MD5 4afccd4118f98b4596f9828033604bcf
BLAKE2b-256 b695aa3d340a8b6cf2174eb64a6561f9d61799bacaf0bf3fa3469114838a72dd

See more details on using hashes here.

Provenance

The following attestation bundles were made for jd_worker-1.1.0.tar.gz:

Publisher: client-publish.yml on NWSL-UCF/job-distributor

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file jd_worker-1.1.0-py3-none-any.whl.

File metadata

  • Download URL: jd_worker-1.1.0-py3-none-any.whl
  • Upload date:
  • Size: 13.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for jd_worker-1.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 3d45b2e0e7fe21767ed947bf251e4f3993db8f83152e281928e056a5656fb4ce
MD5 f341a7dd23b23aaed4f06de3d7070991
BLAKE2b-256 6f0aac5bc044e6ce5165e99c38e9eb7529b23f5cd30e641f0adf2b34f55db40e

See more details on using hashes here.

Provenance

The following attestation bundles were made for jd_worker-1.1.0-py3-none-any.whl:

Publisher: client-publish.yml on NWSL-UCF/job-distributor

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page