Nexa SDK
The Nexa SDK is a comprehensive toolkit for supporting ONNX and GGML models. It supports text generation, image generation, vision-language models (VLM), and text-to-speech (TTS) capabilities. Additionally, it offers an OpenAI-compatible API server with JSON schema mode for function calling and streaming support, and a user-friendly Streamlit UI.
Features
-
Model Support:
- ONNX & GGML models
- Conversion Engine
- Inference Engine:
- Text Generation
- Image Generation
- Vision-Language Models (VLM)
- Text-to-Speech (TTS)
Detailed API documentation is available here.
- Server:
- OpenAI-compatible API
- JSON schema mode for function calling
- Streaming support
- Streamlit UI for interactive model deployment and testing
Installation
GPU version(optional)
check if you have GPU acceleration (torch required)
CUDA:
import torch
torch.cuda.is_available()
if True
CMAKE_ARGS="-DGGML_CUDA=on -DSD_CUBLAS=ON" pip install nexaai-gpu
Apple M Chip:
Apple icon -> about this mac -> Graphicsif True:
CMAKE_ARGS="-DGGML_METAL=on -DSD_METAL=ON" pip install nexaai-gpu
AMD graphics card:
CMAKE_ARGS="-DGGML_HIPBLAS=on" pip install nexaai-gpu
CPU version
Mac with Intel chips
CMAKE_ARGS="-DCMAKE_CXX_FLAGS=-fopenmp" pip install nexaai
Mac with M chips or other Operating systems:
pip install nexaai
Nexa CLI commands
Model Commands
NLP Models
| Model | Type | Format | Command |
|---|---|---|---|
| octopus-v2 | NLP | GGUF | nexa gen-text octopus-v2 |
| octopus-v4 | NLP | GGUF | nexa gen-text octopus-v4 |
| tinyllama | NLP | GGUF | nexa gen-text tinyllama |
| llama2 | NLP | GGUF/ONNX | nexa gen-text llama2 |
| llama3 | NLP | GGUF/ONNX | nexa gen-text llama3 |
| llama3.1 | NLP | GGUF/ONNX | nexa gen-text llama3.1 |
| gemma | NLP | GGUF/ONNX | nexa gen-text gemma |
| gemma2 | NLP | GGUF | nexa gen-text gemma2 |
| qwen1.5 | NLP | GGUF | nexa gen-text qwen1.5 |
| qwen2 | NLP | GGUF/ONNX | nexa gen-text qwen2 |
| mistral | NLP | GGUF/ONNX | nexa gen-text mistral |
| codegemma | NLP | GGUF | nexa gen-text codegemma |
| codellama | NLP | GGUF | nexa gen-text codellama |
| codeqwen | NLP | GGUF | nexa gen-text codeqwen |
| deepseek-coder | NLP | GGUF | nexa gen-text deepseek-coder |
| dolphin-mistral | NLP | GGUF | nexa gen-text dolphin-mistral |
| nomic-embed-text | NLP | GGUF | nexa gen-text nomic-embed-text |
| phi2 | NLP | GGUF | nexa gen-text phi2 |
| phi3 | NLP | GGUF/ONNX | nexa gen-text phi3 |
Multimodal Models
| Model | Type | Format | Command |
|---|---|---|---|
| nanollava | Multimodal | GGUF | nexa vlm nanollava |
| llava-phi3 | Multimodal | GGUF | nexa vlm llava-phi3 |
| llava-llama3 | Multimodal | GGUF | nexa vlm llava-llama3 |
| llava1.6-mistral | Multimodal | GGUF | nexa vlm llava1.6-mistral |
| llava1.6-vicuna | Multimodal | GGUF | nexa vlm llava1.6-vicuna |
Computer Vision Models
| Model | Type | Format | Command |
|---|---|---|---|
| stable-diffusion-v1-4 | Computer Vision | GGUF | nexa gen-image sd1-4 |
| stable-diffusion-v1-5 | Computer Vision | GGUF/ONNX | nexa gen-image sd1-5 |
| lcm-dreamshaper | Computer Vision | GGUF/ONNX | nexa gen-image lcm-dreamshaper |
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
nexaai_metal-0.0.3.tar.gz
(38.1 MB
view details)
File details
Details for the file nexaai_metal-0.0.3.tar.gz.
File metadata
- Download URL: nexaai_metal-0.0.3.tar.gz
- Upload date:
- Size: 38.1 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/5.1.1 CPython/3.10.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
d38347496ec5e3904a9b3e7ef054d913afba442ecf374fb03451aa0de53a7b26
|
|
| MD5 |
edd5ac47486991a6258a485ef9ab5092
|
|
| BLAKE2b-256 |
2433c0b4cd421bc77e56bc0b3e17f22df61a23cb53f6d313fdebce6f5cd700ab
|
