Skip to main content

AI Flow, an extend operators library for airflow, which helps AI engineer to write less, reuse more, integrate easily.

Project description

AI Flow

Introduction

AI Flow, which offers various reusable operators & processing units in AI modeling, helps AI engineer to write less, reuse more, integrate easily.

Install

pip install aiflow

Concepts

Operators VS. Units

Ideally, we agree:

  • An Operator would contain lot of units, which will be integrated into airflow for building non-realtime processing workflow;
  • A Unit is a small calculation unit, which could be a function, or just a simple modeling logic, and it could be picked as bricks to build an operator. Besides, it could be reused anywhere for realtime calculation.

Classes

Operators

MongoToCSVOperator

Elastic2CSVOperator

RegExLabellingOperator

Units

Doc2VecUnit

Doc2MatUnit

Tests & Examples

Example: Use Units to Build Your Castle

Example: Working with Airflow

In tests/docker/ folder, we provide examples on how to use aiflow with airflow. It is a docker image, you could simply copy and start to use it!

In project root directory, run commands first:

docker-compose up --build aiflow

Then open localhost:8080 in your browser, you can see all the examples aiflow provided! Note: both the default username & password are admin

Enjoy!

Contribution

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

aiflow-1.0.0.tar.gz (8.1 kB view hashes)

Uploaded Source

Built Distribution

aiflow-1.0.0-py3-none-any.whl (16.0 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page