Implementation of the OpenAI API for local LLM use
Implementation of the OpenAI API. Currently supports two end points (ChatCompletion and Embeddings) with the following models:
ChatCompletion models:
- 'mistral-7b': 7B Instruct model trained by MistralAI.
- 'llama-13b': 13B Llama 2 model trained by Meta.
- 'llama-7b': 7B Instruct Llama 2 model trained by Together AI (32K context).
- 'codellama-13b': 13B CodeLlama model trained by Meta.
Embeddings models:
- 'mistral-7b' : Encodes documents into 4096-dimension vectors.
- 'bert' : Encodes documents into 386-dimension vectors.
Getting Started
Can be installed directly with pip (a setup.py file is provided, if needed).
Once installed, the FastAPI server (Uvicorn) can be started with:
run_server
Prompts
Currently, two kinds of prompts are supported.
- Open ended questions to engage in conversation with the model.
- Instruct prompts for vector-based retrieval over some test collections.
Some considerations
This project is by no means production ready. The ChatCompletion endpoint in particular has been tailored to communicate with the QOPA-LLM demo. A lot of work is needed security-wise to avoid data leaking.
1 maintainer
Metadata
1 maintainer
Release files for qualtop-llmapi 0.47
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| qualtop-llmapi-0.47.tar.gz | 17.5 kB | Without provenance | Details for qualtop-llmapi-0.47.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| qualtop_llmapi-0.47-py3-none-any.whl | Python 3 | none | any | Details for qualtop_llmapi-0.47-py3-none-any.whl |
Total release size: 39.3 kB
Release files / qualtop-llmapi-0.47.tar.gz
| Download URL | qualtop-llmapi-0.47.tar.gz |
|---|---|
| Size | 17.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
eb485c94be09ece4b51999df183e94064f00ede3356ff37ef172fc93e3c41717
|
|
BLAKE2b-256 checksum How to use checksums |
94b04172fc69d7526de6fe24d0fb2533a1762e65dc0985ec59cc97838a68ccd1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.11.5
|
Release files / qualtop_llmapi-0.47-py3-none-any.whl
| Download URL | qualtop_llmapi-0.47-py3-none-any.whl |
|---|---|
| Size | 21.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
82ec49b933c6f68ffb8ed725cef6d931689061f0f060db26ea791ce470be52d0
|
|
BLAKE2b-256 checksum How to use checksums |
2ae547c3e1e33e71a6d7d45b887cd7c4f9081027a1fce6886a1db8b5ba90f56d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.11.5
|
Security for qualtop-llmapi 0.47
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
