Skip to main content

No project description provided

Project description

centml-python-client

Installation

First, ensure you meet the requirements for Hidet, namely:

  • CUDA Toolkit 11.6+
  • Python 3.8+

To install without cloning, run the following command:

pip install git+https://github.com/CentML/centml-python-client.git@main

Alternatively to build from source, clone this repo then inside the project's base directory, run the following command:

pip install . 

CLI

Once installed, use the centml CLI tool with the following command:

centml 

If you want tab completion, run

source scripts/completions/completion.<shell language>

Shell language can be: bash, zsh, fish (Hint: add source /path/to/completions/completion.<shell language> to your ~/.bashrc, ~/.zshrc or ~/.config/fish/completions/centml.fish)

Compilation

centml-python-client's compiler feature allows you to compile your ML model remotely using the hidet backend.
Thus, use the compilation feature, make sure to run:

pip install hidet

To run the server locally, you can use the following CLI command:

centml server

By default, the server will run at the URL http://0.0.0.0:8090.
You can change this by setting the environment variable CENTML_SERVER_URL

Then, within your python script include the following:

import torch
# This will import the "centml" torch.compile backend
import centml.compiler  

# Define these yourself
model = ...
inputs = ...

# Pass the "centml" backend
compiled_model = torch.compile(model, backend="centml")
# Since torch.compile is JIT, compilation is only triggered when you first call the model
output = compiled_model(inputs)

Note that the centml backend compiler is non-blocking. This means it that until the server returns the compiled model, your python script will use the uncompiled model to generate the output.

Again, make sure your script's environment sets CENTML_SERVER_URL to communicate with the desired server.

To see logs, add this to your script before triggering compilation:

logging.basicConfig(level=logging.INFO)

Tests

To run tests, first install required packages:

pip install -r requirements-dev.txt
cd tests

When running on a local machine, it is recommended to run tests with the following command. This skips tests that require a GPU.

pytest --sanity

To run all the tests, use:

pytest

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

centml-0.3.0.tar.gz (25.2 kB view details)

Uploaded Source

Built Distribution

centml-0.3.0-py3-none-any.whl (30.7 kB view details)

Uploaded Python 3

File details

Details for the file centml-0.3.0.tar.gz.

File metadata

  • Download URL: centml-0.3.0.tar.gz
  • Upload date:
  • Size: 25.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for centml-0.3.0.tar.gz
Algorithm Hash digest
SHA256 9669c7163336b63aafdf231db0cf112caa9961fc6d4865bb91cb4e6306fd5974
MD5 1d6798bd6aceb42ba1e291713230f1b8
BLAKE2b-256 87f5f7edf61ed6f9c90123e36088716c3d3639f3b193df1399dc5152e183665d

See more details on using hashes here.

Provenance

The following attestation bundles were made for centml-0.3.0.tar.gz:

Publisher: publish.yml on CentML/centml-python-client

Attestations:

File details

Details for the file centml-0.3.0-py3-none-any.whl.

File metadata

  • Download URL: centml-0.3.0-py3-none-any.whl
  • Upload date:
  • Size: 30.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for centml-0.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 e77e55f6c6c49c3b3979ed12950148b105b82d207b3b30d52d1d2f5b3ae6cb59
MD5 151edd84de5c6bfc8ea08cedbbaf95f3
BLAKE2b-256 0bbeb0d72c3003029ec5d3f2a07c9fd78e50ce70177cc19442705a1e5c1a7b2b

See more details on using hashes here.

Provenance

The following attestation bundles were made for centml-0.3.0-py3-none-any.whl:

Publisher: publish.yml on CentML/centml-python-client

Attestations:

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page