Archived
This project has been archived by its maintainers, and is no longer receiving any updates.
mlipy
Pure Python-based Machine Learning Interface for multiple engines with multi-modal support.
Python HTTP Server/Client (including WebSocket streaming support) for:
Prerequisites
Debian/Ubuntu
sudo apt update -y
sudo apt install build-essential git curl libssl-dev libffi-dev pkg-config
Python
- Install Python using internal repository:
sudo apt install python3.11 python3.11-dev python3.11-venv
- Install Python using external repository:
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update -y
sudo apt install python3.11 python3.11-dev python3.11-venv
llama.cpp
cd ~
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
make -j
Run Development Server
Setup virtualenv and install requirements:
git clone https://github.com/mtasic85/mlipy.git
cd mlipy
python3.11 -m venv venv
source venv/bin/activate
pip install poetry
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
poetry install
Run server:
python -B -m mli.server --llama-cpp-path='~/llama.cpp'
Run Examples
Using GPU:
NGL=99 python -B examples/sync_demo.py
Using CPU:
python -B examples/sync_demo.py
python -B examples/async_demo.py
python -B examples/langchain_sync_demo.py
python -B examples/langchain_async_demo.py
Run Production Server
Generate self-signed SSL certificates
openssl req -x509 -nodes -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365
Run
python3.11 -m venv venv
source venv/bin/activate
pip install -U mlipy
python -B -m mli.server
2 maintainers
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
mlipy-0.1.57.tar.gz
(15.1 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
mlipy-0.1.57-py3-none-any.whl
(15.6 kB
view details)
File details
Details for the file mlipy-0.1.57.tar.gz.
File metadata
- Download URL: mlipy-0.1.57.tar.gz
- Upload date:
- Size: 15.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.8.3 CPython/3.12.4 Linux/6.9.10-arch1-1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
7fabc2e10c73db757ea7dcc343f43aaf2a31fe92f64fbe856797a69c8ab7d817
|
|
| MD5 |
7bb3211f22923acd7b937ed2a46baa38
|
|
| BLAKE2b-256 |
56a7e9786571bb3cebe5d7d6b846812f195a4c9117c53d14dc31b4fbca722c8f
|
File details
Details for the file mlipy-0.1.57-py3-none-any.whl.
File metadata
- Download URL: mlipy-0.1.57-py3-none-any.whl
- Upload date:
- Size: 15.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.8.3 CPython/3.12.4 Linux/6.9.10-arch1-1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
d2f0acd51fc00cdea92304f214f3b974b5fd268c5c8a5cdfb385d756a59ef234
|
|
| MD5 |
ec2f6e2efff47a4a1fc0cb06344648e2
|
|
| BLAKE2b-256 |
ca00ab408b095247f9501d0ecc5fd9ecf05c9a2d887aadc33bc17529145864ba
|
