pipewine 0.2.0

Data workflow automation

Pipewine

Pipewine is a complete rewrite of the Pipelime library and intends to serve the same purpose: provide a set of tools to manipulate multi-modal small/medium-sized datasets, mainly for research purposes in the CV/ML domain.

🚀 Features

• Unified access pattern to datasets of various formats, origin and content.
• Underfolder, a quick and easy filesystem-based dataset format good for small/medium datasets.
• Common data encoding formats for images, text and metadata.
• Common operators to manipulate existing datasets.
• Workflows that transform data in complex DAGs (Directed Acyclic Graph) pipelines.
• CLI (Command Line Interface) to quickly run simple workflows without writing a full python script.
• Extendibility, allowing the user to easily extend the library in many aspects, adding components that seamlessly integrate with the built-in ones:
  • Add custom dataset formats
  • Add custom data encoding formats
  • Add custom operators on datasets
  • Register components to the CLI

Documentation

Full documentation is available at lucabonfiglioli.github.io/pipewine.

Installation

Pipewine is available on PyPI and can be installed via pip:

pip install pipewine

License

Pipewine is public domain software released under the Unlicense. You can do whatever you want with it.