Skip to main content

A real distributed CLI/runtime with workers, brokered execution, and custom commands.

Project description

Lucore

lucore is a real, working distributed CLI/runtime that installs with pip3.

It gives you:

  • lucore broker: the central scheduler
  • lucore worker: a machine that executes jobs
  • lucore run and lucore exec: run raw commands locally or on the cluster
  • lucore submit: queue distributed work without waiting
  • lucore task: define and run project tasks from lucore.toml
  • lucore dev: run the project dev task
  • lucore invoke: run named commands from lucore.toml
  • lucore ps, lucore inspect, and lucore logs: inspect jobs
  • lucore cluster workers and lucore cluster jobs: inspect cluster state
  • lucore config and lucore doctor: inspect local project/runtime state

The transport is newline-delimited JSON over TCP, so workers can run on completely separate machines.

Install

After publishing to PyPI:

pip3 install lucore

For local development from this directory:

pip3 install -e .

Quick start

Start the broker:

lucore broker --listen 127.0.0.1:7000

Start a worker:

lucore worker --broker 127.0.0.1:7000 --name worker-a --label queue=build --label os=mac

Submit a raw remote command:

lucore run --broker 127.0.0.1:7000 --label queue=build --command "uname -a"

Run a local command:

lucore run --local --command "python3 --version"

Create a starter config:

lucore init
cat lucore.toml

Run project tasks:

lucore task list
lucore dev
lucore task run build --broker 127.0.0.1:7000
lucore task run test --broker 127.0.0.1:7000

Run a named command:

lucore invoke hello --broker 127.0.0.1:7000 there
lucore invoke python-version --local

Inspect recent work:

lucore ps --broker 127.0.0.1:7000
lucore cluster workers --broker 127.0.0.1:7000
lucore inspect <job-id> --broker 127.0.0.1:7000
lucore logs <job-id> --broker 127.0.0.1:7000

lucore.toml

Example:

[project]
name = "my-app"

[tasks.dev]
run = "python3 -m http.server 3000 {args}"
mode = "local"

[tasks.build]
run = "echo building app && uname -a"
mode = "distributed"
labels = { queue = "build" }

[tasks.test]
run = "pytest -q {args}"
mode = "distributed"
labels = { queue = "test", os = "linux" }

[commands.hello]
shell = "echo hello from lucore {args}"
labels = { queue = "build" }

{args} is replaced with any extra arguments passed to lucore task run or lucore invoke.

Current behavior

  • Jobs queue until a matching worker is available.
  • Labels route work to the right class of worker.
  • Workers heartbeat every 5 seconds.
  • If a worker disappears mid-job, the broker requeues that job.
  • Tasks can run locally or distributed depending on config and flags.
  • The broker keeps recent job history and stored logs in memory.
  • Named commands are real config-driven commands, not fake placeholders.

Next production steps

If you want to harden this into a serious runtime, the next upgrades are TLS/auth, persistent broker state, streaming logs, retries, and sandboxed execution.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lucore-0.1.0.tar.gz (16.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

lucore-0.1.0-py3-none-any.whl (18.4 kB view details)

Uploaded Python 3

File details

Details for the file lucore-0.1.0.tar.gz.

File metadata

  • Download URL: lucore-0.1.0.tar.gz
  • Upload date:
  • Size: 16.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.1

File hashes

Hashes for lucore-0.1.0.tar.gz
Algorithm Hash digest
SHA256 c4bd74dc71e2b4d7c45e50c0f57f68ccac4c92263a21552e6a0715ed41334d7c
MD5 725ad94fe960c4a8b22caf90223d919e
BLAKE2b-256 2ded48598726f5edd6e73902b88751b669ca87a3490c76eb47f15d54375a17e5

See more details on using hashes here.

File details

Details for the file lucore-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: lucore-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 18.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.1

File hashes

Hashes for lucore-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 47410c7916d849cc4b9e2410386ae7ece056ee8b1ab7cfd04d852abad5b49e71
MD5 a6a6a9d7181cbfca32fa7fdd139e4656
BLAKE2b-256 76671bd2cc363dc41b1360b80a629f21f34ebb8a8c331a7b048c5c42591b7b7e

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page