Skip to main content

Do Something and Record It

Project description



DSARI - DO SOMETHING AND RECORD IT


dsari is a lightweight continuous integration (CI) system. It provides
scheduling, concurrency management and trigger capabilities, and is easy
to configure. Job scheduling is handled via dsari-daemon, while
dsari-render may be used to format job run information as HTML.


Requirements

dsari requires Python 3.4 or later, and will run on Unix-based
platforms. It uses the following non-core packages:

- croniter, for parsing cron-style schedule definitions
- python-dateutil, for parsing iCalendar RRULE-style schedule
definitions
- Jinja2, for rendering HTML reports
- psycopg2, for PostgreSQL database support
- mysqlclient (mysqldb), for MySQL database support
- pymongo, for MongoDB database support

All non-core packages are optional, with the following limitations:

- If neither croniter nor python-dateutil are installed, dsari-daemon
will run, but it will not process scheduled runs (i.e. manual
triggers only).
- Jinja2 is only required if you intend to use dsari-render.
- psycopg2, mysqlclient or pymongo are only required if you intend to
use dsari with an alternative database. By default, dsari uses a
SQLite 3 database.


Installation

dsari may be installed as any normal Python package:

$ sudo python3 setup.py install

When this is done, dsari will expect its configuration file --
dsari.json -- in /usr/local/etc/dsari/, and will store its data in
/usr/local/lib/dsari/.

These locations may be customized by passing -c argument to dsari-* to
specify the configuration directory, and the data_dir configuration
option, respectively.

When dsari is installed directly in /usr/ (i.e. as part of distribution
packaging), the default configuration and data directories will be
/etc/dsari/ and /var/lib/dsari/, respectively.

dsari does not need to be installed at all, it can be run directly from
the repository directory. In this case, the default configuration and
data directories will be ~/.dsari/etc/ and ~/.dsari/var/, respectively.

The rest of these documents assume a locally-running setup, i.e.
~/.dsari/.


Configuration

A basic configuration for dsari.json looks as follows:

{
"jobs": {
"sample-job": {
"command": ["/usr/bin/env"],
"schedule": "H/5 * * * *"
}
}
}

This defines a job named "sample-job", which is run every 5 minutes.
Many more configuration options are available in the doc/ directory.


Running

Once dsari is configured, run dsari-daemon. By default, dsari-daemon
will run in the foreground, and can be used with a supervisor (upstart,
systemd, supervisord, etc). If given -d, it will daemonize.

When a job is scheduled to be run, it produces a "run". Runs are
identified by a UUID, the run output is stored in ~/.dsari/var/runs/,
and data related to the run (start time, stop time, exit code, etc) is
stored in a SQLite database at ~/.dsari/var/dsari.sqlite3.

When a run is executed, several environment variables are passed to the
program to be run:

JOB_NAME=sample-job
RUN_ID=fa0490b8-7a8e-4f6b-b73c-160199a9ff75
PREVIOUS_RUN_ID=e5bd61b3-27f3-46ca-8169-372433056fc2
PREVIOUS_SCHEDULE_TIME=1437004689.27
PREVIOUS_START_TIME=1437004689.65
PREVIOUS_STOP_TIME=1437004689.71
PREVIOUS_EXIT_CODE=0

PREVIOUS_* variables are not set if there is no previous run. In
addition, several extra environment variables are present, if the job's
jenkins_environment option is set, to aid with migrations from Jenkins
setups:

BUILD_NUMBER=fa0490b8-7a8e-4f6b-b73c-160199a9ff75
BUILD_ID=fa0490b8-7a8e-4f6b-b73c-160199a9ff75
BUILD_URL=file:///home/user/.dsari/var/runs/sample-job/fa0490b8-7a8e-4f6b-b73c-160199a9ff75/
NODE_NAME=master
BUILD_TAG=dsari-sample-job-fa0490b8-7a8e-4f6b-b73c-160199a9ff75
JENKINS_URL=file:///home/user/.dsari/var/
EXECUTOR_NUMBER=0
WORKSPACE=/home/user/.dsari/var/runs/sample-job/fa0490b8-7a8e-4f6b-b73c-160199a9ff75


Reports

To render HTML reports, run dsari-render occasionally. This will produce
a series of HTML files in ~/.dsari/var/html/. You may then serve these
files, rsync them to a remote server, etc.

The dsari-info command may be used to retrieve information about jobs
and runs.

The dsari-prometheus-exporter command may be used to start a metrics
daemon suitable for ingestion into Prometheus.


License

dsari - Do Something and Record It

Copyright (C) 2015-2016 Ryan Finnie

This program is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 2 of the License, or (at your
option) any later version.

This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
Public License for more details.

You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
dsari-2.0.tar.gz (45.1 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page