SimpleScheduler A Cron Library for Openmetadata
Project description
OpenMetadata Simple Scheduler
We are going to build a library based on simple scheduler for only cron-related ingestion purpose.
Scheduler
openmetadata-simplescheduler
is a flexible python library for building your own cron-like system to schedule jobs, which is to run a tornado process to serve REST APIs and a web ui.
Try it NOW
From source code:
git clone https://github.com/open-metadata/simplescheduler.git
cd simplescheduler
make simple
Open your browser and go to localhost:7777.
How to build Your own cron-replacement
Install simplescheduler
Using pip (from GitHub repo)
#
# Put this in requirements.txt, then run
# pip install -r requirements.txt
#
# If you want the latest build
git+https://github.com/open-metadata/simplescheduler.git#egg=simplescheduler
pip install -e git+https://github.com/open-metadata/simplescheduler.git#egg=simplescheduler
Three things
You have to implement three things for your scheduler, i.e., Settings
, Server
, and Jobs
.
Settings
In your implementation, you need to provide a settings file to override default settings (e.g., settings in simple_scheduler). You need to specify the python import path in the environment variable SIMPLESCHEDULER_SETTINGS_MODULE
before running the server.
All available settings can be found in default_settings.py file.
Server
You need to have a server file to import and run sdscheduler.server.server.SchedulerServer
.
Jobs
Each job should be a standalone class that is a subclass of sdscheduler.job.JobBase
and put the main logic of the job in run()
function.
After you set up Settings
, Server
and Jobs
, you can run the whole thing like this:
SIMPLESCHEDULER_SETTINGS_MODULE=simple_scheduler.settings \
PYTHONPATH=.:$(PYTHONPATH) \
python simple_scheduler/scheduler.py
Install dependencies
# Each time we introduce a new dependency in setup.py, you have to run this
make install
Run unit tests
make test
Clean everything and start from scratch
make clean
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for openmetadata-simplescheduler-0.2.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | a26c20bb3fa2c7aba560f2d3f12c8c0ee253fd9adfefce28bd75b9d958541cd3 |
|
MD5 | 1ba318cfdcfb95d8b634e335b143b2ea |
|
BLAKE2b-256 | ef55e6f19c98c7ba7f8a6334ec67cff1d0977dc43b7c736f26725df91af88d3f |
Hashes for openmetadata_simplescheduler-0.2.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 51584cdbdffefc96443d2eef22092560c4ec6d59ef3dcc363ffcc2f794f103ce |
|
MD5 | f5f3b8c1fd248f72ef67560caa8e0bc8 |
|
BLAKE2b-256 | bb7dd235a68d80731c4897d73395edf76bc1b827540e06b5c1025c016655e44a |