HTTP API for LLM with OpenAI compatibility
Project description
llm-http-api
HTTP API for LLM with OpenAI compatibility
Usage
> llm http-api --help
Usage: llm http-api [OPTIONS]
Run a FastAPI HTTP server with OpenAI compatibility
Options:
-h, --host TEXT [default: 0.0.0.0]
-p, --port INTEGER [default: 8080]
-l, --log-level TEXT [default: info]
--help Show this message and exit.
> curl http://localhost:8080/v1/embeddings -X POST -H "Content-Type: application/json" -d '{
"input": "Hello world",
"model": "jina-embeddings-v2-small-en"
}'
{"object":"embedding","embedding":[-0.47561466693878174,-0.4471365511417389,...],"index":0}
OpenAI Endpoints
Embeddings
Unimplemented
A detailed list of unimplemented OpenAI endpoints can be found here
Development
This repository manages the dev environment as a Nix flake and requires Nix to be installed
nix develop -c $SHELL
make deps.install
make deps.install/test
make test
make coverage
make lint
make format
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
llm-http-api-0.0.1.tar.gz
(7.9 kB
view hashes)
Built Distribution
Close
Hashes for llm_http_api-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 208f18fa7f7f08c5ed5b243b2f64938d4d4d17de777a75e7d8db29470d75fd64 |
|
MD5 | d6ac8b9a27ea94eacd700fe0d80ccfd3 |
|
BLAKE2b-256 | bbb636e17e0113f921bc1c983ed83beba13d1548118df62283392063609abb8a |