Skip to main content

AI Flow, an extend operators library for airflow, which helps AI engineer to write less, reuse more, integrate easily.

Project description

AI Flow

Introduction

AI Flow, which offers various reusable operators & processing units in AI modeling, helps AI engineer to write less, reuse more, integrate easily.

Install

pip install aiflow

Concepts

Operators VS. Units

Ideally, we agree:

  • An Operator would contain lot of units, which will be integrated into airflow for building non-realtime processing workflow;
  • A Unit is a small calculation unit, which could be a function, or just a simple modeling logic, and it could be picked as bricks to build an operator. Besides, it could be reused anywhere for realtime calculation.

Classes

Operators

MongoToCSVOperator

Elastic2CSVOperator

RegExLabellingOperator

Units

Doc2VecUnit

Doc2MatUnit

Tests & Examples

Example: Use Units to Build Your Castle

Example: Working with Airflow

In tests/docker/ folder, we provide examples on how to use aiflow with airflow. It is a docker image, you could simply copy and start to use it!

In project root directory, run commands first:

docker-compose up --build aiflow

Then open localhost:8080 in your browser, you can see all the examples aiflow provided! Note: both the default username & password are admin

Enjoy!

Contribution

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

aiflow-1.0.0.tar.gz (8.1 kB view details)

Uploaded Source

Built Distribution

aiflow-1.0.0-py3-none-any.whl (16.0 kB view details)

Uploaded Python 3

File details

Details for the file aiflow-1.0.0.tar.gz.

File metadata

  • Download URL: aiflow-1.0.0.tar.gz
  • Upload date:
  • Size: 8.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/2.0.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.6.0.post20191030 requests-toolbelt/0.9.1 tqdm/4.38.0 CPython/3.6.9

File hashes

Hashes for aiflow-1.0.0.tar.gz
Algorithm Hash digest
SHA256 1f11099ffe8a7a13822cd838b1f7a630088e58eeb40b134fc76963c15c18b35d
MD5 f4cd7f51c71d5f0a5727a01b353ddf79
BLAKE2b-256 a38d5b4db2abac3791e5ca26402a24154e93a00749b15a29a18b88ffe5b8fb7a

See more details on using hashes here.

File details

Details for the file aiflow-1.0.0-py3-none-any.whl.

File metadata

  • Download URL: aiflow-1.0.0-py3-none-any.whl
  • Upload date:
  • Size: 16.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/2.0.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.6.0.post20191030 requests-toolbelt/0.9.1 tqdm/4.38.0 CPython/3.6.9

File hashes

Hashes for aiflow-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 1fc3d0db04036fa3bb288760b035257fac9fc4324b930ae403108d6a7200dc45
MD5 50de71e09714f8dfa0b03d9ab6b6b69e
BLAKE2b-256 5f7593b8ca20b80525e71d862e76ea66949fc03e2447e67d10b674a8d0d8c174

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page