This release is a pre-release and may not be stable for production use.
LLM Engine
The LLM Engine Python library provides a convenient way of interfacing with a
llmengine endpoint running on
LLM Engine or on your own infrastructure.
Get Started
Install
pip install scale-llm-engine
Usage
If you are using LLM Engine, you can get your API key from
https://spellbook.scale.com/settings.
Set the SCALE_API_KEY environment variable to your API key.
If you are using your own infrastructure, you can set the
LLM_ENGINE_BASE_PATH environment variable to the base URL of your
self-hosted llmengine endpoint.
from llmengine import Completion
response = Completion.create(
model="llama-2-7b",
prompt="Hello, my name is",
max_new_tokens=10,
temperature=0.2,
)
print(response.outputs[0].text)
Documentation
Documentation is available at https://scaleapi.github.io/llm-engine/.
1 maintainer
Metadata
1 maintainer
Release files for scale-llm-engine 0.0.0b45
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| scale_llm_engine-0.0.0b45.tar.gz | 64.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| scale_llm_engine-0.0.0b45-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 138.2 kB
Release files / scale_llm_engine-0.0.0b45.tar.gz
| Download URL | scale_llm_engine-0.0.0b45.tar.gz |
|---|---|
| Size | 64.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c197de005152eef46c8867401c365c0a9c3221808e6fe048083c53b3ff065639
|
|
BLAKE2b-256 checksum How to use checksums |
e8dc901c7517c68c1de1f8a2b3b9584175f852048888ad6e7c6a07ae53aeb2fb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.0.1 CPython/3.10.15
|
Release files / scale_llm_engine-0.0.0b45-py3-none-any.whl
| Download URL | scale_llm_engine-0.0.0b45-py3-none-any.whl |
|---|---|
| Size | 73.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
e3f0148e37dabd9665b49e15eab9a4798c0e3c7809fbc805c2fe1bfdc92319e9
|
|
BLAKE2b-256 checksum How to use checksums |
4786297013a4a3ae7a54d572d294c2cd47b27937dc5006ce6c1c4d11c6ba7f3e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.0.1 CPython/3.10.15
|
