Project description

llama-api-server

This project is under active deployment. Breaking changes could be made any time.

Llama as a Service! This project try to build a REST-ful API server compatible to OpenAI API using open source backends like llama.

Tested with

openai-python
- OPENAI_API_TYPE=default
- OPENAI_API_TYPE=azure

Supported APIs

Completions
- set temperature, top\_p, and top\_k
- set max\_tokens
- set stop
- set stream
- set n
- set presence\_penalty and frequency\_penalty
- set logit\_bias
Embeddings
- batch process
Chat

Supported backed

llama.cpp via llamacpp-python

Others

Documents
Token auth
Intergration tests
Performance parameters like n_batch and n_thread
A tool to download/prepare pretrain model

Get start

Project details

These details have not been verified by PyPI

Project links

Release history Release notifications | RSS feed

0.3.5

Aug 24, 2023

0.3.4

Aug 23, 2023

0.3.3

Aug 23, 2023

0.2.3

Apr 20, 2023

0.2.2

Apr 16, 2023

0.2.1

Apr 15, 2023

0.2.0

Apr 14, 2023

0.1.3

Apr 9, 2023

0.1.2

Apr 8, 2023

This version

0.1.0

Apr 8, 2023

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

llama_api_server-0.1.0.tar.gz (5.5 kB view hashes)

Uploaded Apr 8, 2023 Source

Built Distribution

llama_api_server-0.1.0-py3-none-any.whl (5.4 kB view hashes)

Uploaded Apr 8, 2023 Python 3

Hashes for llama_api_server-0.1.0.tar.gz

Hashes for llama_api_server-0.1.0.tar.gz
Algorithm	Hash digest
SHA256	`2ce516c53dde497d957e4fd4ded2c7c4cf669c0f12d685504613f635740ebe98`
MD5	`9fe7a79b00307eda668e5c9322077324`
BLAKE2b-256	`05167b18998752cd1d275f8a02c97fbbac6c6a0dfd5c9207abfd92857be6597f`

Hashes for llama_api_server-0.1.0-py3-none-any.whl

Hashes for llama_api_server-0.1.0-py3-none-any.whl
Algorithm	Hash digest
SHA256	`f153b50a8aa2ef8d6923c1924cb4aa3d135baa23c63570adcc5b9d617fc93a55`
MD5	`dff66d60e25fed297b5ca91ab8fceb4f`
BLAKE2b-256	`de05bd552bba50082e6297efc84ee8ca8226eb41676430ca1006d1465dcc07a4`