npuserver 1.0.2

No project description provided

npuserver 🚀

A lightweight, efficient utility library for compiling and preparing Generative AI LLM models for the Intel NPU (Neural Processing Unit) using OpenVINO™ GenAI.

---

Features

• Intel NPU Optimization: Fast, local LLM compilation designed for Intel NPU architectures.
• Robust Model Fallbacks: Automated properties configuration with retry logic for prompt lengths.
• Hugging Face Hub Integration: Seamless resolution and down-caching of models.
• Clean API Design: Import and use directly in any Python environment.

---

Installation

From PyPI

pip install npuserver

From Source

1. Clone the repository:

   git clone https://github.com/yourusername/npuserver.git
   cd npuserver
   

2. Set up a virtual environment and install:

   python -m venv .venv
   # On Windows:
   .venv\Scripts\activate
   # On macOS/Linux:
   source .venv/bin/activate
   
   pip install -e .
   

---

Quick Start

Compile your favorite Hugging Face LLM model for the Intel NPU:

from pathlib import Path
from npuserver import compile_model

# Path to store compiled model caches
cache_dir = Path("./npu_cache")
cache_dir.mkdir(exist_ok=True)

# Compile a Hugging Face LLM (e.g., Qwen or Phi)
compile_model(
    repo_id="Qwen/Qwen2.5-0.5B-Instruct",
    cache_dir=cache_dir,
    prompt_len=8192
)

---

Development

Running with Poetry

This library uses Poetry as its package manager:

poetry install
poetry run python -c "import npuserver; print(npuserver.__all__)"

Directory Structure

npuserver/
├── .github/workflows/   # CI/CD & Automated Publishing
├── src/
│   └── npuserver/
│       ├── __init__.py  # Package entry point
│       └── compile.py   # Core compilation functions
├── tests/               # Test suites
├── pyproject.toml       # Modern packaging configuration
└── requirements.txt     # Standard pip requirements

---

PyPI Automatic Publishing

The project includes an automated GitHub Actions CI/CD pipeline (.github/workflows/publish.yml) that builds and publishes releases securely using OIDC Trusted Publishing:

1. Tag your release:

   git tag v0.1.0
   git push origin v0.1.0
   

2. The GitHub Action will trigger, build source/wheel distributions, and push to PyPI.