Skip to main content

Workflow orchestration and management.

Project description

Prefect 2.0

Prefect 2.0 is the second-generation dataflow coordination and orchestration platform from Prefect.

Prefect 2.0 has been designed from the ground up to handle the dynamic, scalable workloads that the modern data stack demands. Powered by Prefect Orion, a brand-new, asynchronous rules engine, it represents an enormous amount of research, development, and dedication to a simple idea:

You should love your workflows again.

Read the docs, run the code, or host the UI. Join thousands of community members in our Slack community to share your thoughts and feedback. Thanks for being part of the mission to build a new kind of workflow system and, of course, happy engineering!

"Don't Panic"

Still using Prefect 1.0 Core and Server? Find the legacy Prefect 1.0 docs at https://docs-v1.prefect.io/.


Hello, Orion!

Powered by the Orion engine, Prefect is the easiest way to transform any function into a unit of work that can be observed and governed by orchestration rules.

Add workflow features like retries, distributed execution, scheduling, caching, and much more, with minimal changes to your code. Every activity is tracked and becomes visible in the Orion Dashboard.

from prefect import flow, task
from typing import List
import httpx


@task(retries=3)
def get_stars(repo: str):
    url = f"https://api.github.com/repos/{repo}"
    count = httpx.get(url).json()["stargazers_count"]
    print(f"{repo} has {count} stars!")


@flow(name="GitHub Stars")
def github_stars(repos: List[str]):
    for repo in repos:
        get_stars(repo)


# run the flow!
github_stars(["PrefectHQ/Prefect"])

After running any running some flows, fire up the UI to gain insight into their execution:

prefect orion start

From here, you can continue to use Prefect interactively or set up automated deployments.

Next steps

Prefect 2.0 was designed to be incrementally adopted into your workflows, and our documentation is organized to support your exploration as much as possible. It is organized into four main sections whose applicability will depend on your objectives and comfort level.

Getting started

Begin by installing Prefect on your machine, then follow one of our friendly tutorials to learn by example. See the Getting Started overview for more.

Concepts

Learn more about Prefect's features and design by reading our in-depth concept docs. These are intended to introduce the building blocks of Prefect, build up to orchestration and deployment, and finally cover some of the advanced use cases that Prefect makes possible.

Frequently asked questions

Prefect 2.0 represents a fundamentally new way of building and orchestrating data workflows. Learn more about the project by reading our FAQ.

API reference

Prefect provides a number of programmatic workflow interfaces, each of which is documented in the API Reference. This is where you can learn how a specific function works, or see the expected payload for a REST endpoint.

Contributing

See our documentation on contributing to Prefect 2.0.

Join the community

Prefect 2.0 was made possible by the fastest-growing community of data engineers. The Prefect Slack community is a fantastic place to learn more, ask questions, or get help with workflow design. The Prefect Discourse is an additional community-driven knowledge base to find answers to your Prefect-related questions. Join us and thousands of friendly data engineers to help build a new kind of workflow system.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

prefect-2.0.0.tar.gz (852.7 kB view details)

Uploaded Source

Built Distribution

prefect-2.0.0-py3-none-any.whl (960.1 kB view details)

Uploaded Python 3

File details

Details for the file prefect-2.0.0.tar.gz.

File metadata

  • Download URL: prefect-2.0.0.tar.gz
  • Upload date:
  • Size: 852.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.9.13

File hashes

Hashes for prefect-2.0.0.tar.gz
Algorithm Hash digest
SHA256 98f8191b24d8a9463217518ec7ce0870671c35950eaa0444b5d7a101787ced8d
MD5 496a2052d66f7976f5025cc66e28906d
BLAKE2b-256 dc0ae5614b6c7e62f687686c6f5edcd657e504b4fa3e9f750c9541a86659fa43

See more details on using hashes here.

File details

Details for the file prefect-2.0.0-py3-none-any.whl.

File metadata

  • Download URL: prefect-2.0.0-py3-none-any.whl
  • Upload date:
  • Size: 960.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.9.13

File hashes

Hashes for prefect-2.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 b9bf2261d5c7f8209d496e2f523b696f3fd58f241bb3c19bec3bae7df4229165
MD5 446f7f7c37cb97e57671d45a2d36db7f
BLAKE2b-256 7c78e9df8be153ee28ff982dbbc6dd8edd0fb387d967739da8b66643cf9e3c6c

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page