nexaai-metal 0.0.3

Nexa AI SDK

Nexa SDK

The Nexa SDK is a comprehensive toolkit for supporting ONNX and GGML models. It supports text generation, image generation, vision-language models (VLM), and text-to-speech (TTS) capabilities. Additionally, it offers an OpenAI-compatible API server with JSON schema mode for function calling and streaming support, and a user-friendly Streamlit UI.

Features

• Model Support:

  • ONNX & GGML models
  • Conversion Engine
  • Inference Engine:
    • Text Generation
    • Image Generation
    • Vision-Language Models (VLM)
    • Text-to-Speech (TTS)

Detailed API documentation is available here.

• Server:
  • OpenAI-compatible API
  • JSON schema mode for function calling
  • Streaming support
• Streamlit UI for interactive model deployment and testing

Installation

GPU version(optional)

check if you have GPU acceleration (torch required)

CUDA:

import torch
torch.cuda.is_available()

if True

CMAKE_ARGS="-DGGML_CUDA=on -DSD_CUBLAS=ON" pip install nexaai-gpu

Apple M Chip:

Apple icon -> about this mac -> Graphics

if True:

CMAKE_ARGS="-DGGML_METAL=on -DSD_METAL=ON" pip install nexaai-gpu

AMD graphics card:

CMAKE_ARGS="-DGGML_HIPBLAS=on" pip install nexaai-gpu

CPU version

Mac with Intel chips

CMAKE_ARGS="-DCMAKE_CXX_FLAGS=-fopenmp" pip install nexaai

Mac with M chips or other Operating systems:

pip install nexaai

Nexa CLI commands

Model Commands

NLP Models

Model	Type	Format	Command
octopus-v2	NLP	GGUF	nexa gen-text octopus-v2
octopus-v4	NLP	GGUF	nexa gen-text octopus-v4
tinyllama	NLP	GGUF	nexa gen-text tinyllama
llama2	NLP	GGUF/ONNX	nexa gen-text llama2
llama3	NLP	GGUF/ONNX	nexa gen-text llama3
llama3.1	NLP	GGUF/ONNX	nexa gen-text llama3.1
gemma	NLP	GGUF/ONNX	nexa gen-text gemma
gemma2	NLP	GGUF	nexa gen-text gemma2
qwen1.5	NLP	GGUF	nexa gen-text qwen1.5
qwen2	NLP	GGUF/ONNX	nexa gen-text qwen2
mistral	NLP	GGUF/ONNX	nexa gen-text mistral
codegemma	NLP	GGUF	nexa gen-text codegemma
codellama	NLP	GGUF	nexa gen-text codellama
codeqwen	NLP	GGUF	nexa gen-text codeqwen
deepseek-coder	NLP	GGUF	nexa gen-text deepseek-coder
dolphin-mistral	NLP	GGUF	nexa gen-text dolphin-mistral
nomic-embed-text	NLP	GGUF	nexa gen-text nomic-embed-text
phi2	NLP	GGUF	nexa gen-text phi2
phi3	NLP	GGUF/ONNX	nexa gen-text phi3

Multimodal Models

Model	Type	Format	Command
nanollava	Multimodal	GGUF	nexa vlm nanollava
llava-phi3	Multimodal	GGUF	nexa vlm llava-phi3
llava-llama3	Multimodal	GGUF	nexa vlm llava-llama3
llava1.6-mistral	Multimodal	GGUF	nexa vlm llava1.6-mistral
llava1.6-vicuna	Multimodal	GGUF	nexa vlm llava1.6-vicuna

Computer Vision Models

Model	Type	Format	Command
stable-diffusion-v1-4	Computer Vision	GGUF	nexa gen-image sd1-4
stable-diffusion-v1-5	Computer Vision	GGUF/ONNX	nexa gen-image sd1-5
lcm-dreamshaper	Computer Vision	GGUF/ONNX	nexa gen-image lcm-dreamshaper