Nexa SDK

The Nexa SDK is a comprehensive toolkit for supporting ONNX and GGML models. It supports text generation, image generation, vision-language models (VLM), and text-to-speech (TTS) capabilities. Additionally, it offers an OpenAI-compatible API server with JSON schema mode for function calling and streaming support, and a user-friendly Streamlit UI.

Features

Model Support:
- ONNX & GGML models
- Conversion Engine
- Inference Engine:
  - Text Generation
  - Image Generation
  - Vision-Language Models (VLM)
  - Text-to-Speech (TTS)

Detailed API documentation is available here.

Server:
- OpenAI-compatible API
- JSON schema mode for function calling
- Streaming support
Streamlit UI for interactive model deployment and testing

Installation

GPU version(optional)

check if you have GPU acceleration (torch required)

CUDA:

import torch
torch.cuda.is_available()

if True

CMAKE_ARGS="-DGGML_CUDA=on -DSD_CUBLAS=ON" pip install nexaai-gpu

Apple M Chip:

Apple icon -> about this mac -> Graphics

if True:

CMAKE_ARGS="-DGGML_METAL=on -DSD_METAL=ON" pip install nexaai-gpu

AMD graphics card:

CMAKE_ARGS="-DGGML_HIPBLAS=on" pip install nexaai-gpu

CPU version

Mac with Intel chips

CMAKE_ARGS="-DCMAKE_CXX_FLAGS=-fopenmp" pip install nexaai

Mac with M chips or other Operating systems:

pip install nexaai

Nexa CLI commands

Model Commands

NLP Models

Model	Type	Format	Command
octopus-v2	NLP	GGUF	`nexa gen-text octopus-v2`
octopus-v4	NLP	GGUF	`nexa gen-text octopus-v4`
tinyllama	NLP	GGUF	`nexa gen-text tinyllama`
llama2	NLP	GGUF/ONNX	`nexa gen-text llama2`
llama3	NLP	GGUF/ONNX	`nexa gen-text llama3`
llama3.1	NLP	GGUF/ONNX	`nexa gen-text llama3.1`
gemma	NLP	GGUF/ONNX	`nexa gen-text gemma`
gemma2	NLP	GGUF	`nexa gen-text gemma2`
qwen1.5	NLP	GGUF	`nexa gen-text qwen1.5`
qwen2	NLP	GGUF/ONNX	`nexa gen-text qwen2`
mistral	NLP	GGUF/ONNX	`nexa gen-text mistral`
codegemma	NLP	GGUF	`nexa gen-text codegemma`
codellama	NLP	GGUF	`nexa gen-text codellama`
codeqwen	NLP	GGUF	`nexa gen-text codeqwen`
deepseek-coder	NLP	GGUF	`nexa gen-text deepseek-coder`
dolphin-mistral	NLP	GGUF	`nexa gen-text dolphin-mistral`
nomic-embed-text	NLP	GGUF	`nexa gen-text nomic-embed-text`
phi2	NLP	GGUF	`nexa gen-text phi2`
phi3	NLP	GGUF/ONNX	`nexa gen-text phi3`

Multimodal Models

Model	Type	Format	Command
nanollava	Multimodal	GGUF	`nexa vlm nanollava`
llava-phi3	Multimodal	GGUF	`nexa vlm llava-phi3`
llava-llama3	Multimodal	GGUF	`nexa vlm llava-llama3`
llava1.6-mistral	Multimodal	GGUF	`nexa vlm llava1.6-mistral`
llava1.6-vicuna	Multimodal	GGUF	`nexa vlm llava1.6-vicuna`

Computer Vision Models

Model	Type	Format	Command
stable-diffusion-v1-4	Computer Vision	GGUF	`nexa gen-image sd1-4`
stable-diffusion-v1-5	Computer Vision	GGUF/ONNX	`nexa gen-image sd1-5`
lcm-dreamshaper	Computer Vision	GGUF/ONNX	`nexa gen-image lcm-dreamshaper`

Metadata

Release files for nexaai-metal 0.0.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for nexaai-metal 0.0.3
File	Size	Uploaded
nexaai_metal-0.0.3.tar.gz	38.1 MB	Aug 22, 2024	Details

Release files / nexaai_metal-0.0.3.tar.gz

Download URL	nexaai_metal-0.0.3.tar.gz
Size	38.1 MB
Tags	Source
SHA-256 checksum How to use checksums	`d38347496ec5e3904a9b3e7ef054d913afba442ecf374fb03451aa0de53a7b26`
BLAKE2b-256 checksum How to use checksums	`2433c0b4cd421bc77e56bc0b3e17f22df61a23cb53f6d313fdebce6f5cd700ab`
Upload date	Aug 22, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/5.1.1 CPython/3.10.0`