prefect-collection

Welcome to your new Prefect collection! Feel free to modify this README to fit the needs of your collection!

Getting Started

Python setup

Requires an installation of Python 3.7+

We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.

Verify collection bootstrap

To ensure that your collection bootstrap was successful you can perform the following steps:

1. Create an editable install of your project

   pip install -e ".[dev]
   

2. Run the tests for the example tasks and flow in the bootstrapped collection

   pytest tests
   

3. Serve the bootstrapped example docs with mkdocs

   mkdocs serve
   

You are now ready to start creating your Prefect collection!

Developing tasks and flows

For information about the use and development of tasks and flow, check out the flows and tasks concepts docs in the Prefect docs.

Writing documentation

This collection has been setup to with mkdocs for automatically generated documentation. The signatures and docstrings of your tasks and flow will be used to generate documentation for the users of this collection. You can make changes to the structure of the generated documentation by editing the mkdocs.yml file in this project.

Development lifecycle

CI Pipeline

This collection comes with GitHub Actions for testing and linting. To add additional actions, you can add jobs in the .github/workflows folder. On PR, the pipeline will run linting via black and flake8 and unit tests via pytest.

Package and Publish

GitHub actions will also handle packaging and publishing of your collection to PyPI so other Prefect users can your collection in their flows.

In order to publish to PyPI, you'll need a PyPI account and generate an API token to authenticate with PyPI when publishing new versions of your collection. The PyPI documentation outlines the steps needed to get an API token.

Once you've obtained a PyPI API token, create a GitHub secret named PYPI_API_TOKEN.

There is also a GitHub Action that performs a test publish to test PyPI to allow for a test deployment without affecting PyPI. You will need a separate token for test PyPI which can be saved in a GitHub secret names TEST_PYPI_API_TOKEN

Further guidance

If you run into any issues during the bootstrapping process, feel free to open an issue in the prefect-collection-template repository.

If you have any questions or issues while developing your collection, you can find help in either the Prefect Discourse forum or the Prefect Slack community