LatencyAI - AI Performance Engineer
Introduction
LatencyAI is an AI agent that optimizes any Python code for best performance using reasoning LLMs. It iteratively profiles, optimizes, and benchmarks the code. The goal is to optimize code by GPU offloading, using data/task parallel, latency hiding and other techniques.
Note: this is an experimental library. Subscribe for updates at latency.ai.
Installation
- (Optional) Deploy a CUDA-enabled GPU instance
pip install --upgrade latencyai
Usage
- Set the OPEANAI_API_KEY environment variable
- Run
python -m latencyai --runs=3 script-to-optimize.py. Optionally set--runs, which is the number of optimization attempts, i.e. optimize-benchmark-profile iterations. The default is 2.
The provided script should have a main function. The benchmark runner calls it multiple times, depending on it's execution time.
If optimization is successful, a file named <original-script>_optimized.py is be written to original script directory.
Tracking optimizations
After integrating optimized code into your application, you can verify and track end-to-end performance improvements in deployed applications using Graphsignal.
1 maintainer
Metadata
1 maintainer
Release files for latencyai 0.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| latencyai-0.1.1.tar.gz | 6.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| latencyai-0.1.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 14.0 kB
Release files / latencyai-0.1.1.tar.gz
| Download URL | latencyai-0.1.1.tar.gz |
|---|---|
| Size | 6.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
dc57a2e2956854c7a5bd98e75be585e441015f462116c03538af605733150f35
|
|
BLAKE2b-256 checksum How to use checksums |
cd00db6fa7c8c7122d4bced214435daecd51dc3884745c256f79396e74fd1fa9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.1.1 CPython/3.10.16 Linux/6.8.0-1021-azure
|
Release files / latencyai-0.1.1-py3-none-any.whl
| Download URL | latencyai-0.1.1-py3-none-any.whl |
|---|---|
| Size | 8.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
28d50388e3958720c526adb3c5d683289c481ce5f0698aa5dd64c20ccf1927ca
|
|
BLAKE2b-256 checksum How to use checksums |
352fb48911373bce9fb83b8f4ba78dd86ca0f054b2e924c3a66adb8a5fede3f6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.1.1 CPython/3.10.16 Linux/6.8.0-1021-azure
|
