Core Neon LLM service
Project description
NeonAI Core LLM
Core module for Neon LLM's
Request Format
API requests should include history
, a list of tuples of strings, and the current
query
Example Request:
{ "history": [["user", "hello"], ["llm", "hi"]], "query": "how are you?" }
Response Format
Responses will be returned as dictionaries. Responses should contain the following:
response
- String LLM response to the query
Connection Configuration
When running this as a docker container, the XDG_CONFIG_HOME
envvar is set to /config
.
A configuration file at /config/neon/diana.yaml
is required and should look like:
MQ:
port: <MQ Port>
server: <MQ Hostname or IP>
users:
<LLM MQ service_name>:
user: <MQ user>
password: <MQ user's password>
LLM_<LLM NAME uppercase>:
num_parallel_processes: <integer > 0>
Enabling Chatbot personas
An LLM may be configured to connect to a /chatbots
vhost and participate in
discussions as described in the chatbots project.
One LLM may define multiple personas to participate as:
llm_bots:
<LLM Name>:
- name: Assistant
description: You are a personal assistant who responds in 40 words or less
- name: Author
description: You are an author and expert in literary history
- name: Student
description: You are a graduate student working in the field of artificial intelligence
enabled: False
LLM Name
is defined in the propertyNeonLLMMQConnector.name
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for neon_llm_core-0.1.1a3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c0e5076a5d0e38022638c7543e57d9c8f7a8feeaaad1a8a070db0d3c8dd9672f |
|
MD5 | 51cbde2159a4f9e75c82c529804f5b44 |
|
BLAKE2b-256 | 99e4a798656118ee2113f834267cdf74adfeb8186684f28709627fb2f28de9aa |