Skip to main content

A multi-platform cron-like job runner in Python

Project description

Pycroner

Pycroner

Pycroner is a lightweight cron style job runner implemented in Python. Jobs are configured via a YAML file and executed by the runner once their cron schedule matches the current time.

Features

  • Parses standard five field cron expressions (minute, hour, day, month, weekday) using a small built in parser.
  • Jobs can optionally be fanned out into multiple processes. Fanout may be an integer (repeat the job N times) or a list of argument strings that will be appended to the base command.
  • Configuration lives in pycroner.yml by default. The exact format is described in pycroner/spec.md.

Installation

Pycroner requires Python 3.8+ and PyYAML. Clone the repository and install the dependencies:

pip install -r requirements.txt  # or install pyyaml manually

Usage

From code

  1. Create a pycroner.yml file describing your jobs. A simple example is shown below.
  2. Run the job runner from a Python script:
from pycroner.runner import run

run("pycroner.yml")

The runner checks schedules every minute and spawns each job as a subprocess when its cron expression matches the current time.

From CLI

You can also invoke the runner directly from the command line using the pycroner command. By default it looks for pycroner.yml in the current directory:

pycroner

Specify an alternative working directory with --at or a specific configuration file with --config:

pycroner --at /path/to/project
pycroner --config custom.yml

Example Configuration

jobs:
  - id: "index_articles"
    schedule: "*/15 * * * *"
    command: "python index.py"
    fanout: 4

  - id: "daily_etl"
    schedule: "0 2 * * *"
    command: "python etl.py"
    fanout:
      - "--source=internal --mode=full"
      - "--source=external --mode=delta"

  - id: "ping"
    schedule: "* * * * *"
    command: "python ping.py"

Jobs run independently, and any output or error handling is left to your commands. For full details see pycroner/spec.md.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pycroner-0.1.0.tar.gz (18.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

pycroner-0.1.0-py3-none-any.whl (18.2 kB view details)

Uploaded Python 3

File details

Details for the file pycroner-0.1.0.tar.gz.

File metadata

  • Download URL: pycroner-0.1.0.tar.gz
  • Upload date:
  • Size: 18.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.3

File hashes

Hashes for pycroner-0.1.0.tar.gz
Algorithm Hash digest
SHA256 1a0faa71c0b6f363304c6ab43b94360f159314e9625dce5d375c10e4e2720f87
MD5 7feb2b4305f279aaf3b65a1502a798ff
BLAKE2b-256 57f61155804df9b2527085ec9356a0efcc07cc79ef2c578b8594df08cec84b91

See more details on using hashes here.

File details

Details for the file pycroner-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: pycroner-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 18.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.3

File hashes

Hashes for pycroner-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 777556e36c8f2c19cc320c9dd13977cea01bea3f78ac2d106a5c505d0fe33a4e
MD5 2dfa482f877587211d0f065c85213efd
BLAKE2b-256 e28bd664129132a849670410a4462452566f4615482ccf084a1c936731879a50

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page