Skip to main content

A Cron-like system for running tasks

Project description

# CronQ

A cron-like system to run your application tasks across any node, instead of one
special snowflake. This is done by keeping your tasks in MySQL and publishing
them over AMQP to workers that will run your tasks and eventually save the
results back into the DB. This was started as a hackathon project at
[SeatGeek](http://seatgeek.com)

# Installation

pip install cronq

Workers that need to be started

* `cronq-runner` - executes each task
* `cronq-injector` - checks for new tasks in the DB and publishes to AMQP
* `cronq-results` - Saves events from the runner

These take the envvars `RABBITMQ_HOST` and `CRONQ_MYQL`. `CRONQ_MYQL` should
be a MySQL Connector DSN starting with `mysql+mysqlconnector://`.

`cronq-runner` will also use the variable `CRONQ_QUEUE` to determine which
queue to consume. The default is `cronq_jobs`.

The web view is a WSGI app run from `cronq.web:app` and requires the same envvars.

## Runner

The runner requires `/var/log/cronq/` to exist and be writable by the user
executing the runner.


## Categories

Categories allow you to replace a set of jobs with a single API call

```
curl -v 'localhost:5000/api/category/example' -XPUT -H 'content-type: application/json' -d '
{
"category": "example",
"jobs": [{
"name": "Test Job",
"schedule": "R/2013-05-29T00:00:00/PT1M",
"command": "sleep 10",
"queue": "slow"
}]
}'
```

This adds / updates a job named `Test Job` in the `example` category. The time
format is ISO 8601. Any jobs no longer defined for the example category will be
removed. This allows you to script job additions / removes in your VCS.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cronq-0.0.22.tar.gz (126.0 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page