ez-parallel 0.1.3

Easy Parallel Multiprocessing

ez-parallel

Easy Parallel Multiprocessing

Very first steps

Initial

1. Initialize git inside your repo:

git init

2. If you don't have Poetry installed run:

make download-poetry

3. Initialize poetry and install pre-commit hooks:

make install

4. Upload initial code to GitHub (ensure you've run make install to use pre-commit):

git add .
git commit -m ":tada: Initial commit"
git branch -M main
git remote add origin https://github.com/j-rossi-nl/ez-parallel.git
git push -u origin main

Initial setting up

• Set up Dependabot to ensure you have the latest dependencies.
• Set up Stale bot for automatic issue closing.

Poetry

All manipulations with dependencies are executed through Poetry. If you're new to it, look through the documentation.

Notes about Poetry

Poetry's commands are very intuitive and easy to learn, like:

• poetry add numpy
• poetry run pytest
• poetry build
• etc

Building your package

Building a new version of the application contains steps:

• Bump the version of your package poetry version <version>. You can pass the new version explicitly, or a rule such as major, minor, or patch. For more details, refer to the Semantic Versions standard.
• Make a commit to GitHub.
• Create a GitHub release.
• And... publish 🙂 poetry publish --build

What's next

Well, that's up to you. I can only recommend the packages and articles that helped me.

Packages:

• Typer is great for creating CLI applications.
• Rich makes it easy to add beautiful formatting in the terminal.
• FastAPI is a type-driven asynchronous web framework.
• IceCream is a little library for sweet and creamy debugging

Articles:

• Open Source Guides
• GitHub Actions Documentation
• Maybe you would like to add gitmoji to commit names. This is really funny. 😄

🚀 Features

For your development we've prepared:

• Supports for Python 3.7 and higher.
• Poetry as the dependencies manager. See configuration in pyproject.toml and setup.cfg.
• Power of black, isort and pyupgrade formatters.
• Ready-to-use pre-commit hooks with formatters above.
• Type checks with the configured mypy.
• Testing with pytest.
• Docstring checks with darglint.
• Security checks with safety and bandit.
• Well-made .editorconfig, .dockerignore, and .gitignore. You don't have to worry about those things.

For building and deployment:

• GitHub integration.
• Makefile for building routines. Everything is already set up for security checks, codestyle checks, code formatting, testing, linting, docker builds, etc. More details at Makefile summary).
• Dockerfile for your package.
• Github Actions with predefined build workflow as the default CI/CD.
• Always up-to-date dependencies with @dependabot (You will only enable it).
• Automatic drafts of new releases with Release Drafter. It creates a list of changes based on labels in merged Pull Requests. You can see labels (aka categories) in release-drafter.yml. Works perfectly with Semantic Versions specification.

For creating your open source community:

• Ready-to-use Pull Requests templates and several Issue templates.
• Files such as: LICENSE, CONTRIBUTING.md, CODE_OF_CONDUCT.md, and SECURITY.md are generated automatically.
• Stale bot that closes abandoned issues after a period of inactivity. (You will only need to setup free plan). Configuration is here.
• Semantic Versions specification with Release Drafter.

Installation

pip install -U ez-parallel

or install with Poetry

poetry add ez-parallel

Then you can run

ez-parallel --help

ez-parallel --name Roman

or if installed with Poetry:

poetry run ez-parallel --help

poetry run ez-parallel --name Roman

Makefile usage

Makefile contains many functions for fast assembling and convenient work.

1. Download Poetry

make download-poetry

2. Install all dependencies and pre-commit hooks

make install

If you do not want to install pre-commit hooks, run the command with the NO_PRE_COMMIT flag:

make install NO_PRE_COMMIT=1

3. Check the security of your code

make check-safety

This command launches a Poetry and Pip integrity check as well as identifies security issues with Safety and Bandit. By default, the build will not crash if any of the items fail. But you can set STRICT=1 for the entire build, or you can configure strictness for each item separately.

make check-safety STRICT=1

or only for safety:

make check-safety SAFETY_STRICT=1

multiple

make check-safety PIP_STRICT=1 SAFETY_STRICT=1

List of flags for check-safety (can be set to 1 or 0): STRICT, POETRY_STRICT, PIP_STRICT, SAFETY_STRICT, BANDIT_STRICT.

4. Check the codestyle

The command is similar to check-safety but to check the code style, obviously. It uses Black, Darglint, Isort, and Mypy inside.

make check-style

It may also contain the STRICT flag.

make check-style STRICT=1

List of flags for check-style (can be set to 1 or 0): STRICT, BLACK_STRICT, DARGLINT_STRICT, ISORT_STRICT, MYPY_STRICT.

5. Run all the codestyle formaters

Codestyle uses pre-commit hooks, so ensure you've run make install before.

make codestyle

6. Run tests

make test

7. Run all the linters

make lint

the same as:

make test && make check-safety && make check-style

List of flags for lint (can be set to 1 or 0): STRICT, POETRY_STRICT, PIP_STRICT, SAFETY_STRICT, BANDIT_STRICT, BLACK_STRICT, DARGLINT_STRICT, ISORT_STRICT, MYPY_STRICT.

8. Build docker

make docker

which is equivalent to:

make docker VERSION=latest

More information here.

9. Cleanup docker

make clean_docker

or to remove all build

make clean

More information here.

📈 Releases

You can see the list of available releases on the GitHub Releases page.

We follow Semantic Versions specification.

We use Release Drafter. As pull requests are merged, a draft release is kept up-to-date listing the changes, ready to publish when you’re ready. With the categories option, you can categorize pull requests in release notes using labels.

For Pull Request this labels are configured, by default:

Label	Title in Releases
enhancement, feature	🚀 Features
bug, refactoring, bugfix, fix	🔧 Fixes & Refactoring
build, ci, testing	📦 Build System & CI/CD
breaking	💥 Breaking Changes
documentation	📝 Documentation
dependencies	⬆️ Dependencies updates

You can update it in release-drafter.yml.

GitHub creates the bug, enhancement, and documentation labels for you. Dependabot creates the dependencies label. Create the remaining labels on the Issues tab of your GitHub repository, when you need them.

🛡 License

This project is licensed under the terms of the MIT license. See LICENSE for more details.

📃 Citation

@misc{ez-parallel,
  author = {Julien Rossi},
  title = {Easy Parallel Multiprocessing},
  year = {2021},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/j-rossi-nl/ez-parallel}}
}

Credits

This project was generated with python-package-template.