Lanarky
Lanarky is an open-source framework to deploy LLM applications in production. It is built on top of FastAPI and comes with batteries included.
🚀 Features
- supports LangChain
- simple gradio chatbot UI for fast prototyping
See Roadmap for upcoming features.
❓ Why?
There are great low-code/no-code solutions in the open source to deploy your LLM projects. However, most of them are opinionated in terms of cloud or deployment code. This project aims to provide users with a cloud-agnostic and deployment-agnostic solution which can be easily integrated into existing backend infrastructures.
💾 Installation
The library is available on PyPI and can be installed via pip.
pip install lanarky
You can find the full documentation at https://lanarky.readthedocs.io/en/latest/.
🔥 Deploy a simple Langchain application in under 20 lines of code
from dotenv import load_dotenv
from fastapi import FastAPI
from langchain import ConversationChain
from langchain.chat_models import ChatOpenAI
from pydantic import BaseModel
from lanarky.responses import StreamingResponse
load_dotenv()
app = FastAPI()
class Request(BaseModel):
query: str
@app.post("/chat")
async def chat(request: Request) -> StreamingResponse:
chain = ConversationChain(llm=ChatOpenAI(temperature=0, streaming=True), verbose=True)
return StreamingResponse.from_chain(chain, request.query, media_type="text/event-stream")
See examples/ for list of available demo examples.
Create a .env file using .env.sample and add your OpenAI API key to it
before running the examples.
📍 Roadmap
- Add support for LangChain
- Add Gradio UI for fast prototyping
- Add SQL database integration
- Add support for Guardrails
- Add support for LlamaIndex
- Add GPTCache integration
🤝 Contributing
Contributions are more than welcome! If you have an idea for a new feature or want to help improve lanarky, please create an issue or submit a pull request on GitHub.
See CONTRIBUTING.md for more information.
⚖️ License
The library is released under the MIT License.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file lanarky-0.6.6.tar.gz.
File metadata
- Download URL: lanarky-0.6.6.tar.gz
- Upload date:
- Size: 11.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
poetry/1.5.0 CPython/3.9.16 Linux/5.15.0-1037-azure
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
03f38846eb56ebf90ccf8658205d7ab983bfd58f343557e05f403cd5314ee8e7
|
|
| MD5 |
b8106e84eba8357d9be5fea23db20fb5
|
|
| BLAKE2b-256 |
8203727dac938261478fb61c8a957e5dfef53ca5731eef87e91261b177f48923
|
File details
Details for the file lanarky-0.6.6-py3-none-any.whl.
File metadata
- Download URL: lanarky-0.6.6-py3-none-any.whl
- Upload date:
- Size: 14.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
poetry/1.5.0 CPython/3.9.16 Linux/5.15.0-1037-azure
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
c37e0de75f074ea44f526f97ac986949ec1b388c5499677452b1cef5daf66c51
|
|
| MD5 |
5e2aaab259f3f38d9cecab0baf0c8b92
|
|
| BLAKE2b-256 |
b87c19425c1502bae147c4444f3a1345abde67a2eb8527d753760a04ac5792e7
|
