Skip to main content

AI Smart Human Assistant Library

Project description

AISHA Lib: A High-Level Abstraction for Building AI Assistants

In the evolving landscape of artificial intelligence, the development of smart assistants has become increasingly prevalent. To streamline this process, the AISHA (AI Smart Human Assistant) Lib offers a high-level abstraction designed for creating AI assistants. This versatile library supports various large language models (LLMs) and different LLM backends, providing developers with a powerful and flexible toolset.

Environment

To create a Python virtual environment, use the command:

conda env create -f environment.yml

Installation

pip install aishalib

Supported Models

The following LLM models are supported:

  • microsoft/Phi-3-medium-128k-instruct
  • CohereForAI/c4ai-command-r-v01
  • google/gemma-2-27b-it
  • Qwen/Qwen2-72B-Instruct

LLM backends

The following LLM backends are supported:

  • Llama.cpp Server API

Telegram bot example

import os

from aishalib.aishalib import Aisha
from aishalib.llmbackend import LlamaCppBackend
from aishalib.tools import parseToolResponse
from aishalib.utils import get_time_string
from aishalib.memory import SimpleMemory
from telegram import Update
from telegram.ext import Application, MessageHandler, ContextTypes, filters


BOT_NAME = os.environ['BOT_NAME']
TG_TOKEN = os.environ['TG_TOKEN']

PERSISTENCE_DIR = BOT_NAME + "/"

if not os.path.exists(PERSISTENCE_DIR):
    os.makedirs(PERSISTENCE_DIR)

memory = SimpleMemory(PERSISTENCE_DIR + "memory.json")


def get_aisha(aisha_context_key, tg_context):
    if aisha_context_key not in tg_context.user_data:
        backend = LlamaCppBackend("http://127.0.0.1:8088/completion", max_predict=256)
        aisha = Aisha(backend, "google/gemma-2-27b-it", prompt_file="system_prompt_example.txt", max_context=8192)
        tg_context.user_data[aisha_context_key] = aisha
    aisha = tg_context.user_data[aisha_context_key]
    aisha.load_context(aisha_context_key)
    return aisha


async def process_message(update: Update, context: ContextTypes.DEFAULT_TYPE):
    chat_id = update.effective_chat.id
    user_id = str(update.message.from_user.id)
    user_name = memory.get_memory_value("names:" + user_id, "")
    computed_name = user_name if user_name else f"id_{user_id}"
    message = update.message.text

    aisha = get_aisha(PERSISTENCE_DIR + str(chat_id), context)
    aisha.add_user_request(f"{computed_name}: {message}", meta_info=get_time_string())
    tools_response = aisha.completion(temp=0.7, top_p=0.9)
    aisha.save_context(PERSISTENCE_DIR + str(chat_id))

    tools = parseToolResponse(tools_response, ["directly_answer", "save_human_name", "pass"])

    if "save_human_name" in tools:
        user_name = tools["save_human_name"]
        memory.save_memory_value("names:" + user_name.split(":")[0].replace("id_", ""), user_name.split(":")[1])

    if "pass" not in tools:
        await context.bot.send_message(chat_id=chat_id,
                                       text=tools["directly_answer"],
                                       reply_to_message_id=update.message.message_id)


application = Application.builder().token(TG_TOKEN).build()
application.add_handler(MessageHandler(filters.TEXT & ~filters.COMMAND, process_message))
application.run_polling()

Run Llama.CPP Server backend

llama.cpp/build/bin/llama-server -m model_q5_k_m.gguf -ngl 99 -fa -c 4096 --host 0.0.0.0 --port 8000

Install CUDA toolkit for Llama.cpp compilation

Please note that the toolkit version must match the driver version. The driver version can be found using the nvidia-smi command. To install toolkit for CUDA 12.5 you need to run the following commands:

CUDA_TOOLKIT_VERSION=12-5
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update
sudo apt -y install cuda-toolkit-${CUDA_TOOLKIT_VERSION}
echo -e '
export CUDA_HOME=/usr/local/cuda
export PATH=${CUDA_HOME}/bin:${PATH}
export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:$LD_LIBRARY_PATH
' >> ~/.bashrc

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

aishalib-0.0.22.tar.gz (8.8 kB view details)

Uploaded Source

Built Distribution

aishalib-0.0.22-py3-none-any.whl (8.0 kB view details)

Uploaded Python 3

File details

Details for the file aishalib-0.0.22.tar.gz.

File metadata

  • Download URL: aishalib-0.0.22.tar.gz
  • Upload date:
  • Size: 8.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.0 CPython/3.10.12

File hashes

Hashes for aishalib-0.0.22.tar.gz
Algorithm Hash digest
SHA256 c029807cb138b922c11d95775837f342d9644678fdaf15c9147d1df9b1db0fbc
MD5 2610d8866664e8347ba79e4fc65c65bc
BLAKE2b-256 4c1a3dafd8b7a992fe457e99b1351180556fca47e9ed3dc5c54e43f0613a35ca

See more details on using hashes here.

File details

Details for the file aishalib-0.0.22-py3-none-any.whl.

File metadata

  • Download URL: aishalib-0.0.22-py3-none-any.whl
  • Upload date:
  • Size: 8.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.0 CPython/3.10.12

File hashes

Hashes for aishalib-0.0.22-py3-none-any.whl
Algorithm Hash digest
SHA256 41c2d9920302853c85676c14ff678bb9ba8e3ec93ea0ca12610557e9789904d1
MD5 03babdf219f73adb561a91945c3e5927
BLAKE2b-256 3542f3796ef6471574b682ab0217a5671109fd14e7412fb4ef7b06f36dab9aa8

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page