Drop-in OpenAI-compatible that can call LLMs from other providers
Project description
OpenLM
Drop-in OpenAI-compatible library that can call LLMs from other providers (e.g., HuggingFace, Cohere, and more).
1c1
< import openai
---
> import openlm as openai
completion = openai.Completion.create(
model=["bloom-560m", "cohere.ai/command"],
prompt=["Hello world", "second prompt and then"]
)
print(completion)
Features
- Takes in the same parameters as OpenAI's Completion API and returns a similarly structured response.
- Call models from HuggingFace's inference endpoint API, Cohere.ai, OpenAI, or your custom implementation.
- Complete multiple prompts on multiple models in the same request.
- Very small footprint: OpenLM calls the inference APIs directly rather than using multiple SDKs.
Installation
pip install openlm
Examples
- Import as OpenAI
- Set up API keys via environment variables or pass a dict
- Add a custom model or provider
- Complete multiple prompts on multiple models
OpenLM currently supports the Completion endpoint, but over time will support more standardized endpoints that make sense.
Other Languages
r2d4/llm.ts is a TypeScript library that has a similar API that sits on top of multiple language models.
Roadmap
- Streaming API
- Embeddings API
Contributing
Contributions are welcome! Please open an issue or submit a PR.
License
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
openlm-0.0.2.tar.gz
(7.4 kB
view details)
Built Distribution
File details
Details for the file openlm-0.0.2.tar.gz
.
File metadata
- Download URL: openlm-0.0.2.tar.gz
- Upload date:
- Size: 7.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.2 Darwin/22.4.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8f75c478a0553b92fa958957d0a296327ff03e573e5b542e74bf28f35921dd2f |
|
MD5 | 79a54b4076ad2906dbd529487943c7ae |
|
BLAKE2b-256 | 2d688be9c96b15e3925770dc85e7d9163c946832aa645ab19f636858045a7698 |
File details
Details for the file openlm-0.0.2-py3-none-any.whl
.
File metadata
- Download URL: openlm-0.0.2-py3-none-any.whl
- Upload date:
- Size: 9.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.2 Darwin/22.4.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | f3c3de08623563f87c8739c653f2434a479f284d1000801f4b4dd02a0c2177c9 |
|
MD5 | a28231930b48870c6b1eef8df1a964d5 |
|
BLAKE2b-256 | c54194e1d86d236c5ab4557395d7459f839ce2682b7571d47a6594ed89af4a71 |