AI Flow, an extend operators library for airflow, which helps AI engineer to write less, reuse more, integrate easily.
Project description
AI Flow
Introduction
AI Flow, which offers various reusable operators & processing units in AI modeling, helps AI engineer to write less, reuse more, integrate easily.
Install
pip install aiflow
Concepts
Operators VS. Units
Ideally, we agree:
- An Operator would contain lot of units, which will be integrated into
airflow
for building non-realtime processing workflow; - A Unit is a small calculation unit, which could be a function, or just a simple modeling logic, and it could be picked as bricks to build an operator. Besides, it could be reused anywhere for realtime calculation.
Classes
Operators
MongoToCSVOperator
Elastic2CSVOperator
RegExLabellingOperator
Units
Doc2VecUnit
Doc2MatUnit
Tests & Examples
Example: Use Units to Build Your Castle
Example: Working with Airflow
In tests/docker/
folder, we provide examples on how to use aiflow
with airflow
.
It is a docker image, you could simply copy and start to use it!
In project root directory, run commands first:
docker-compose up --build aiflow
Then open localhost:8080
in your browser, you can see all the examples aiflow
provided!
Note: both the default username & password are admin
Enjoy!
Contribution
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
aiflow-1.0.0.tar.gz
(8.1 kB
view hashes)
Built Distribution
aiflow-1.0.0-py3-none-any.whl
(16.0 kB
view hashes)