Skip to main content

pydolphinscheduler is Apache DolphinScheduler Python API.

Project description

PyDolphinScheduler

PyPi Version PyPi Python Versions PyPi License PyPi Status Downloads Coverage Status Code style: black Imports: isort CI Twitter Follow Slack Status

PyDolphinScheduler is python API for Apache DolphinScheduler, which allow you definition your workflow by python code, aka workflow-as-codes.

Quick Start

Version Compatibility

At Nov 7, 2022 we seperated PyDolphinScheduler from DolphinScheduler, and the version of PyDolphinScheduler 4.0.0 can match multiple versions of DolphinScheduler, for more details, please refer to version

Installation

# Install
python -m pip install apache-dolphinscheduler

# Verify installation is successful, it will show the version of apache-dolphinscheduler, here we use 0.1.0 as example
pydolphinscheduler version
# 0.1.0

NOTE: package apache-dolphinscheduler not work on above Python version 3.10(including itself) in Window operating system due to dependence py4j not work on those environments.

Here we show you how to install and run a simple example of PyDolphinScheduler

Start DolphinScheduler

There are many ways to start DolphinScheduler, here we use docker to start and run it as a standalone server.

# Change the version of dolphinscheduler to the version you want to use, here we use 3.1.1 as example
DOLPHINSCHEDULER_VERSION=3.1.1
docker run --name dolphinscheduler-standalone-server -p 12345:12345 -p 25333:25333 -e API_PYTHON_GATEWAY_ENABLED="true" -d apache/dolphinscheduler-standalone-server:"${DOLPHINSCHEDULER_VERSION}"

After the container is started, you can access the DolphinScheduler UI via http://localhost:12345/dolphinscheduler. For more way to start DolphinScheduler and the more detail about DolphinScheduler, please refer to DolphinScheduler

Run a simple example

We have many examples in examples directory, we here pick up a typical one to show how to run it.

# Get the latest code of example from github 
curl https://raw.githubusercontent.com/apache/dolphinscheduler-sdk-python/main/src/pydolphinscheduler/examples/tutorial.py -o ./tutorial.py

# Change tenant to real exists tenant in the host your DolphinScheduler running, by any editor you like 

# Run the example
python ./tutorial.py

NOTICE: Since Apache DolphinScheduler's tenant is requests while running command, you have to change tenant value in file tutorial.py. The default value is tenant_exists, change it to username exists your host.

After that, a new workflow will be created by PyDolphinScheduler, and you can see it in DolphinScheduler web UI's Project Management page. It will trigger the workflow automatically, so you can see the workflow running in DolphinScheduler web UI's Workflow Instance page too. For more detail about any function about DolphinScheduler Project Management, please refer to DolphinScheduler Workflow

Documentation

For full documentation visit document. This documentation is generated from this repository so please raise issues or pull requests for any additions, corrections, or clarifications.

Contributing

If you would like to contribute, check out the open issues on GitHub. You can also see the guide to contributing.

Release

Follow the release guide to release a new version of PyDolphinScheduler.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

apache_dolphinscheduler-4.1.0.tar.gz (82.7 kB view details)

Uploaded Source

Built Distribution

apache_dolphinscheduler-4.1.0-py3-none-any.whl (134.3 kB view details)

Uploaded Python 3

File details

Details for the file apache_dolphinscheduler-4.1.0.tar.gz.

File metadata

  • Download URL: apache_dolphinscheduler-4.1.0.tar.gz
  • Upload date:
  • Size: 82.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.0.1 CPython/3.10.4

File hashes

Hashes for apache_dolphinscheduler-4.1.0.tar.gz
Algorithm Hash digest
SHA256 8fb8add653439c7a9ecd06f2e46612295e1ed5fb6ed8ba3bdfa6c590452e9af6
MD5 5971971f4b39e9ef9bd36e8313ce6adc
BLAKE2b-256 bf5ce8a41e35ed3e59c38d4bed1955455dc416bddb3ae8c8e2b88197d3348d0f

See more details on using hashes here.

File details

Details for the file apache_dolphinscheduler-4.1.0-py3-none-any.whl.

File metadata

File hashes

Hashes for apache_dolphinscheduler-4.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 835f47b6995f56ced9b4386922229faf1ca20ff936f7c90591d384a9ede1b7cc
MD5 15d0fe0afa7e16e87ab8c0db536b92de
BLAKE2b-256 52461f7f8b99ef9c0fd16514145a07917d10dbd89566e9761ca299d4efe058eb

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page