OpenMetadata Simple Scheduler
We are going to build a library based on simple scheduler for only cron-related ingestion purpose.
Scheduler
openmetadata-simplescheduler is a flexible python library for building your own cron-like system to schedule jobs, which is to run a tornado process to serve REST APIs and a web ui.
Try it NOW
From source code:
git clone https://github.com/open-metadata/simplescheduler.git
cd simplescheduler
make simple
Open your browser and go to localhost:7777.
How to build Your own cron-replacement
Install simplescheduler
Using pip (from GitHub repo)
#
# Put this in requirements.txt, then run
# pip install -r requirements.txt
#
# If you want the latest build
git+https://github.com/open-metadata/simplescheduler.git#egg=simplescheduler
pip install -e git+https://github.com/open-metadata/simplescheduler.git#egg=simplescheduler
Three things
You have to implement three things for your scheduler, i.e., Settings, Server, and Jobs.
Settings
In your implementation, you need to provide a settings file to override default settings (e.g., settings in simple_scheduler). You need to specify the python import path in the environment variable SIMPLESCHEDULER_SETTINGS_MODULE before running the server.
All available settings can be found in default_settings.py file.
Server
You need to have a server file to import and run sdscheduler.server.server.SchedulerServer.
Jobs
Each job should be a standalone class that is a subclass of sdscheduler.job.JobBase and put the main logic of the job in run() function.
After you set up Settings, Server and Jobs, you can run the whole thing like this:
SIMPLESCHEDULER_SETTINGS_MODULE=simple_scheduler.settings \
PYTHONPATH=.:$(PYTHONPATH) \
python simple_scheduler/scheduler.py
Install dependencies
# Each time we introduce a new dependency in setup.py, you have to run this
make install
Run unit tests
make test
Clean everything and start from scratch
make clean
Metadata
Release files for openmetadata-simplescheduler 0.2.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| openmetadata-simplescheduler-0.2.1.tar.gz | 329.8 kB | Without provenance | Details for openmetadata-simplescheduler-0.2.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| openmetadata_simplescheduler-0.2.1-py2.py3-none-any.whl | Python 3, Python 2 | none | any | Details for openmetadata_simplescheduler-0.2.1-py2.py3-none-any.whl |
Total release size: 697.6 kB
Release files / openmetadata-simplescheduler-0.2.1.tar.gz
| Download URL | openmetadata-simplescheduler-0.2.1.tar.gz |
|---|---|
| Size | 329.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ac7880137128457c46c3d141e4c51445812dd84e3eb8a981a4b2714293eac149
|
|
BLAKE2b-256 checksum How to use checksums |
976875e74f465bfe7c8c236e2ec09e59ed24134c2aaf37c9052f106d9d54bccc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.8.10
|
Release files / openmetadata_simplescheduler-0.2.1-py2.py3-none-any.whl
| Download URL | openmetadata_simplescheduler-0.2.1-py2.py3-none-any.whl |
|---|---|
| Size | 367.8 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
2fa873f9eba6ef3b54703ffdd533ed70107a113d7a77672bbd2771fe9b84da28
|
|
BLAKE2b-256 checksum How to use checksums |
13e0cf95f714597494e5d94175f0d82114ad98dbd9295cc638e9aab3c73b4a6e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.8.10
|
Security for openmetadata-simplescheduler 0.2.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.