Skip to main content

AI Flow, an extend operators library for airflow, which helps AI engineer to write less, reuse more, integrate easily.

Project description

AI Flow

Introduction

AI Flow, which offers various reusable operators & processing units in AI modeling, helps AI engineer to write less, reuse more, integrate easily.

Install

pip install aiflow

Concepts

Operators VS. Units

Ideally, we agree:

  • An Operator would contain lot of units, which will be integrated into airflow for building non-realtime processing workflow;
  • A Unit is a small calculation unit, which could be a function, or just a simple modeling logic, and it could be picked as bricks to build an operator. Besides, it could be reused anywhere for realtime calculation.

Classes

Operators

MongoToCSVOperator

Elastic2CSVOperator

RegExLabellingOperator

Units

Doc2VecUnit

Doc2MatUnit

Tests & Examples

Example: Use Units to Build Your Castle

Example: Working with Airflow

In tests/docker/ folder, we provide examples on how to use aiflow with airflow. It is a docker image, you could simply copy and start to use it!

In project root directory, run commands first:

docker-compose up --build aiflow

Then open localhost:8080 in your browser, you can see all the examples aiflow provided! Note: both the default username & password are admin

Enjoy!

Contribution

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for aiflow, version 1.0.0
Filename, size File type Python version Upload date Hashes
Filename, size aiflow-1.0.0-py3-none-any.whl (16.0 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size aiflow-1.0.0.tar.gz (8.1 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page