Skip to main content

HEflow: A platform for the privacy-preserving machine learning lifecycle

Project description

HEflow: A Privacy-Preserving Machine Learning Lifecycle Platform

HEflow is a platform to streamline privacy-preserving machine learning development, including tracking experiments, packaging code into reproducible runs, and sharing and deploying encrypted models. Built on top of MLflow, Seldon MLServer and OpenMined TenSEAL, HEflow offers a set of lightweight homomorphic encryption APIs that can be used with any existing machine learning application or library (scikit-learn, Keras, TensorFlow, PyTorch, etc), wherever you currently run ML code (e.g. in notebooks, standalone applications, or the cloud).

Binder PyPI version

Homomorphic Encryption (HE)

Homomorphic encryption differs from typical encryption methods in that it allows computation to be performed directly on encrypted data without requiring access to a secret key. The result of such a computation remains in encrypted form, and can at a later point be revealed by the owner of the secret key. This ground-breaking technology has enabled industry and government to provide never-before enabled capabilities for outsourced computation securely.

Homomorphic encryption workflows, for privacy-preserving machine learning, involve three entities:

  1. an ML model owner,

  2. a cloud server that performs model inference on HE encrypted data using the pre-computed ML model, and

  3. a user who sends confidential data to the cloud for model inference.

In all cases, the cloud should learn nothing about the underlying encrypted data.

Privacy-Preserving Machine Learning Operations (PPMLOps)

PPMLOps is a set of processes and automated steps to manage code, data, and encrypted models. This section describes a typical PPMLOps workflow.

HEflow PPMLOps PPMLOps using HEflow

  1. Experiments

    Data scientists develop, train and tune the model on the production data, then they encrypt ① and register it with the Encrypted Model Registry. Model quality is evaluated by testing on held-out production data. This pipeline can be triggered by code changes or by automated retraining jobs.

  2. Encrypted Model Registry

    Autologging saves a record of the training and evaluation process, which includes model metrics, parameters, tags, and the encrypted model itself. When training and hyperparameter tuning are complete, the data scientist registers the final encrypted model artifact in the Encrypted Model Registry for the production environment. This records a link between the encrypted model and the code used to generate it.

  3. Encrypted Model Serving

    A continuous deployment (CD) process takes new encrypted models and deploys ② them for low-latency online serving (APIs). Options include cloud provider serving endpoints, or custom serving applications.

  4. Invocations

    The serving system loads the Production encrypted model version from the Encrypted Model Registry. For each request, it scores the encrypted data ③, and returns encrypted predictions ④.

What is HEflow?

HEflow is an open source platform developed by InAccel to help manage the complete privacy-preserving machine learning lifecycle with enterprise reliability, security and scale. It tackles four primary functions:

:hammer_and_wrench: Encrypted Model development

Accelerate and simplify privacy-preserving machine learning lifecycle management with a standardized framework for developing production-ready PPML models. With HEflow, you can bootstrap PPML projects, perform rapid iteration with ease and ship high-quality encrypted models to production at scale.

:clipboard: Experiment tracking

Run experiments with any ML library, framework or language, and automatically keep track of parameters, metrics, code and encrypted models from each experiment. By using HEflow, you can securely share, manage and compare experiment results along with corresponding artifacts and code versions.

:jigsaw: Encrypted Model management

Use one central place to discover and share PPML models, collaborate on moving them from experimentation to online testing and production, integrate with approval and governance workflows and CI/CD pipelines, and monitor PPML deployments and their performance. HEflow facilitates sharing of expertise and knowledge, and helps you stay in control.

:package: Encrypted Model deployment

Quickly deploy production encrypted models for batch inference or as gRPC homomorphic encryption APIs using built-in integration with Docker containers or KServe. With HEflow, you can operationalize and monitor production encrypted models to scale based on the business needs.

Installing

Install HEflow from PyPI via pip install heflow

Official HEflow Docker Image

The official HEflow Docker image is available on Docker Hub at https://hub.docker.com/r/inaccel/heflow.

# Pull the latest version
docker pull inaccel/heflow

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

heflow-1.2.2.tar.gz (241.3 kB view details)

Uploaded Source

Built Distribution

heflow-1.2.2-py3-none-any.whl (15.9 kB view details)

Uploaded Python 3

File details

Details for the file heflow-1.2.2.tar.gz.

File metadata

  • Download URL: heflow-1.2.2.tar.gz
  • Upload date:
  • Size: 241.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.11.2

File hashes

Hashes for heflow-1.2.2.tar.gz
Algorithm Hash digest
SHA256 0cf28fb94b875b802127e1c14cf411b5ed02b6299087dc3437c3c5e9ee565240
MD5 b0047491aedab3d6d02543749ec7e13e
BLAKE2b-256 60ee65d22d5a2acd512f8cd846fda104329ce88a185762be33f83a38bdf7b131

See more details on using hashes here.

File details

Details for the file heflow-1.2.2-py3-none-any.whl.

File metadata

  • Download URL: heflow-1.2.2-py3-none-any.whl
  • Upload date:
  • Size: 15.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.11.2

File hashes

Hashes for heflow-1.2.2-py3-none-any.whl
Algorithm Hash digest
SHA256 9aad189ce8ef363616a322ff162fe569e4d9f36913c3d69728862fd3e8e02638
MD5 7b0035b16980f0611cb44c4bce6b3b6e
BLAKE2b-256 c1e96919a2ffed4fe339ee1fef6b6b8ff2c1bea774c371a0afe6183143ec0256

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page