veloce 0.0.1rc2

Veloce: An instant distributed computing library based on Ray stack

The project is currently under heavy development, and focusing on PyTorch and the recommendation scenario.

About

Veloce is an instant distributed computing library based on the Ray Train and Ray Data, which is scalable, efficient, and easy-to-use. It accelerates the development of any ML/DL training workload, on any cloud or local, at any parallelism size. Previously named Enscale.

Goals

• Launch any interactive ML/DL workloads instantly on your laptop or to any cloud
• Scale your own single-machine neural network modules to a native distributed manner
• Apply heterogeneous architecture
• Data scientist-friendly API
• Sparse and dense feature definitions

Non-Goals

• Not a neural network library, there are only some benchmark modules provided.

Getting Started

Prerequisites

• Python version >= 3.7.1
• Packages
  • requests >= 2.26.0
  • ray >= 1.9.2 and <= 1.10
  • torch >= 1.9.1
  • pandas >= 1.3.5
  • pyarrow >= 6.0.1

Installation

Using Pre-compiled Wheels

# CPU version
pip install veloce

From Source

git clone https://github.com/ryantd/veloce
cd veloce
pip install -e .

Runtime environment

The library can launch locally or on any cloud provider with Ray set up.

• If you want to launch on the cloud, go through this doc to set up your Ray Cluster. And then you can use environ_validate(n_cpus=N, cluster_endpoint="ray://<head_node_host>:<port>") to connect your cluster.
• Or just use environ_validate(n_cpus=N) to have a local experience.

You can add more native ray.init arguments, just put them into environ_validate call. Like environ_validate(n_cpus=N, ignore_reinit_error=True) to make Ray suppresses errors from calling ray.init() a second time.

Lightning example

See more hands-on and advanced examples here, like heterogeneous support and sparsity definition.

The following example requires sklearn to be installed. And tqdm is optional, which enables progress reporting.

import torch
import torch.nn as nn
from sklearn.metrics import roc_auc_score
from veloce.util import pprint_results, load_benchmark_dataset
from veloce.model.ctr import DeepFM
from veloce import NeuralNetTrainer, environ_validate

N_WORKERS = 2
N_DATA_PROCESSOR = 1

# ray environment setup
environ_validate(n_cpus=N_DATA_PROCESSOR + N_WORKERS)
# load dataset and sparsity definition pre-defined
datasets, feature_defs, dataset_options = load_benchmark_dataset(
    # set your own dataset by `data_path="criteo_mini.txt"`
    separate_valid_dataset=False
)
# trainer setup
trainer = NeuralNetTrainer(
    # module and dataset configs
    module=DeepFM, # your own nn.Module or built in modules
    module_params={
        "dense_feature_defs": feature_defs["dense"],
        "sparse_feature_defs": feature_defs["sparse"],
    },
    dataset=datasets,
    dataset_options=dataset_options,
    # trainer configs
    epochs=5,
    batch_size=512,
    loss_fn=nn.BCELoss(),
    optimizer=torch.optim.Adam,
    metric_fns=[roc_auc_score],
    # logger callbacks
    callbacks=["json"],
    # computation abstract on distributed
    num_workers=N_WORKERS,
)
# run and print results
results = trainer.run()
pprint_results(results)

Architecture

Roadmap

• Heterogeneous Strategy on Distributed Training
  • Sync Parameter Server
  • Aync Parameter Server
  • Hybird Phase 1: use sync or async for the dense or sparse component as you like, under homogeneous architecture
  • Hybird Phase 2: you can choose async PS for the sparse component, and sync Ring Allreduce (like PyTorch's DDP) for the dense component
• Framework Support
  • PyTorch: no specific plan to support other frameworks
• Advanced Parallel Mechanism
  • Heavy integrated torchrec
• Accelerator Support
  • GPU: complete inspection required

Reference

• Ray and Ray Train: Ray Train is a lightweight library for distributed deep learning, allowing you to scale up and speed up training for your deep learning models. Docs here.
• DeepCTR-Torch: Easy-to-use, modular and extendible package of deep-learning based CTR models.

License

Veloce is MIT licensed, as found in the LICENSE file.