No project description provided
Project description
centml-python-client
Installation
First, ensure you meet the requirements for Hidet, namely:
- CUDA Toolkit 11.6+
- Python 3.8+
To install without cloning, run the following command:
pip install git+https://github.com/CentML/centml-python-client.git@main
Alternatively to build from source, clone this repo then inside the project's base directory, run the following command:
pip install .
CLI
Once installed, use the centml CLI tool with the following command:
centml
If you want tab completion, run
source scripts/completions/completion.<shell language>
Shell language can be: bash, zsh, fish
(Hint: add source /path/to/completions/completion.<shell language>
to your ~/.bashrc
, ~/.zshrc
or ~/.config/fish/completions/centml.fish
)
Compilation
centml-python-client's compiler feature allows you to compile your ML model remotely using the hidet backend.
Thus, use the compilation feature, make sure to run:
pip install hidet
To run the server locally, you can use the following CLI command:
centml server
By default, the server will run at the URL http://0.0.0.0:8090
.
You can change this by setting the environment variable CENTML_SERVER_URL
Then, within your python script include the following:
import torch
# This will import the "centml" torch.compile backend
import centml.compiler
# Define these yourself
model = ...
inputs = ...
# Pass the "centml" backend
compiled_model = torch.compile(model, backend="centml")
# Since torch.compile is JIT, compilation is only triggered when you first call the model
output = compiled_model(inputs)
Note that the centml backend compiler is non-blocking. This means it that until the server returns the compiled model, your python script will use the uncompiled model to generate the output.
Again, make sure your script's environment sets CENTML_SERVER_URL
to communicate with the desired server.
To see logs, add this to your script before triggering compilation:
logging.basicConfig(level=logging.INFO)
Tests
To run tests, first install required packages:
pip install -r requirements-dev.txt
cd tests
When running on a local machine, it is recommended to run tests with the following command. This skips tests that require a GPU.
pytest --sanity
To run all the tests, use:
pytest
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file centml-0.3.0.tar.gz
.
File metadata
- Download URL: centml-0.3.0.tar.gz
- Upload date:
- Size: 25.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/5.1.1 CPython/3.12.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9669c7163336b63aafdf231db0cf112caa9961fc6d4865bb91cb4e6306fd5974 |
|
MD5 | 1d6798bd6aceb42ba1e291713230f1b8 |
|
BLAKE2b-256 | 87f5f7edf61ed6f9c90123e36088716c3d3639f3b193df1399dc5152e183665d |
Provenance
The following attestation bundles were made for centml-0.3.0.tar.gz
:
Publisher:
publish.yml
on CentML/centml-python-client
-
Statement type:
https://in-toto.io/Statement/v1
- Predicate type:
https://docs.pypi.org/attestations/publish/v1
- Subject name:
centml-0.3.0.tar.gz
- Subject digest:
9669c7163336b63aafdf231db0cf112caa9961fc6d4865bb91cb4e6306fd5974
- Sigstore transparency entry: 148936662
- Sigstore integration time:
- Predicate type:
File details
Details for the file centml-0.3.0-py3-none-any.whl
.
File metadata
- Download URL: centml-0.3.0-py3-none-any.whl
- Upload date:
- Size: 30.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/5.1.1 CPython/3.12.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e77e55f6c6c49c3b3979ed12950148b105b82d207b3b30d52d1d2f5b3ae6cb59 |
|
MD5 | 151edd84de5c6bfc8ea08cedbbaf95f3 |
|
BLAKE2b-256 | 0bbeb0d72c3003029ec5d3f2a07c9fd78e50ce70177cc19442705a1e5c1a7b2b |
Provenance
The following attestation bundles were made for centml-0.3.0-py3-none-any.whl
:
Publisher:
publish.yml
on CentML/centml-python-client
-
Statement type:
https://in-toto.io/Statement/v1
- Predicate type:
https://docs.pypi.org/attestations/publish/v1
- Subject name:
centml-0.3.0-py3-none-any.whl
- Subject digest:
e77e55f6c6c49c3b3979ed12950148b105b82d207b3b30d52d1d2f5b3ae6cb59
- Sigstore transparency entry: 148936665
- Sigstore integration time:
- Predicate type: