Skip to main content

Drop-in OpenAI-compatible that can call LLMs from other providers

Project description

OpenLM

Drop-in OpenAI-compatible library that can call LLMs from other providers (e.g., HuggingFace, Cohere, and more).

1c1
< import openai
---
> import openlm as openai

completion = openai.Completion.create(
    model=["bloom-560m", "cohere.ai/command"], 
    prompt=["Hello world", "second prompt and then"]
)
print(completion)

Features

  • Takes in the same parameters as OpenAI's Completion API and returns a similarly structured response.
  • Call models from HuggingFace's inference endpoint API, Cohere.ai, OpenAI, or your custom implementation.
  • Complete multiple prompts on multiple models in the same request.
  • Very small footprint: OpenLM calls the inference APIs directly rather than using multiple SDKs.

Installation

pip install openlm

Examples

OpenLM currently supports the Completion endpoint, but over time will support more standardized endpoints that make sense.

Other Languages

r2d4/llm.ts is a TypeScript library that has a similar API that sits on top of multiple language models.

Roadmap

  • Streaming API
  • Embeddings API

Contributing

Contributions are welcome! Please open an issue or submit a PR.

License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

openlm-0.0.2.tar.gz (7.4 kB view details)

Uploaded Source

Built Distribution

openlm-0.0.2-py3-none-any.whl (9.4 kB view details)

Uploaded Python 3

File details

Details for the file openlm-0.0.2.tar.gz.

File metadata

  • Download URL: openlm-0.0.2.tar.gz
  • Upload date:
  • Size: 7.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.3.2 CPython/3.11.2 Darwin/22.4.0

File hashes

Hashes for openlm-0.0.2.tar.gz
Algorithm Hash digest
SHA256 8f75c478a0553b92fa958957d0a296327ff03e573e5b542e74bf28f35921dd2f
MD5 79a54b4076ad2906dbd529487943c7ae
BLAKE2b-256 2d688be9c96b15e3925770dc85e7d9163c946832aa645ab19f636858045a7698

See more details on using hashes here.

File details

Details for the file openlm-0.0.2-py3-none-any.whl.

File metadata

  • Download URL: openlm-0.0.2-py3-none-any.whl
  • Upload date:
  • Size: 9.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.3.2 CPython/3.11.2 Darwin/22.4.0

File hashes

Hashes for openlm-0.0.2-py3-none-any.whl
Algorithm Hash digest
SHA256 f3c3de08623563f87c8739c653f2434a479f284d1000801f4b4dd02a0c2177c9
MD5 a28231930b48870c6b1eef8df1a964d5
BLAKE2b-256 c54194e1d86d236c5ab4557395d7459f839ce2682b7571d47a6594ed89af4a71

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page