fastapi-struct 0.3.0

Production-ready FastAPI architecture generator tailor-made for AI, GenAI, LLM, and Machine Learning services.

---

FastAPI Documentation: https://fastapi.tiangolo.com
PyPI Package: https://pypi.org/project/fastapi-struct/

---

⚡ Why fastapi-struct for AI & ML Engineers?

Traditional web generator templates force relational database CRUD patterns (repositories/, models/, users/, email.py), cluttering modern AI microservices.

fastapi-struct defaults to a lean, enterprise-grade AI & Machine Learning architecture:

• 🤖 GenAI Token Streaming: Real-time token-by-token streaming via StreamingResponse (Server-Sent Events) for LLM chat completions.
• 📚 RAG & Vector Search: Modular vector retrieval layer ready for Pinecone, Qdrant, ChromaDB, or pgvector.
• 🧠 Lifespan Model Caching: Heavy model weights (PyTorch, ONNX, HuggingFace) load into GPU/CPU memory once at startup via FastAPI's @asynccontextmanager lifespan.
• 🩺 GPU & Readiness Probes: Built-in /health endpoint reporting GPU/device readiness (cpu or cuda).
• 🧹 Zero __pycache__ Clutter: Automatically excludes byte-compiled files and stray caches during project scaffolding.
• 🧪 Preconfigured Pytest Suite: 4 automated integration tests for health, prediction, chat, and RAG.

---

📦 Quick Start & CLI Usage

Generate in the Current Directory (.)

fastapi-struct .

Or Create a Named Project Folder

fastapi-struct my_ai_service

(You can also run it instantly with uvx without installing: uvx fastapi-struct .)

---

📂 The AI & ML Architecture

my_ai_service/
│
├── app/
│   ├── __init__.py
│   ├── main.py                 # FastAPI initialization + Lifespan AI/ML model loader
│   │
│   ├── api/
│   │   ├── __init__.py
│   │   └── v1/
│   │       ├── __init__.py
│   │       ├── api.py          # Central router aggregating AI, RAG, and ML routes
│   │       └── routes/
│   │           ├── __init__.py
│   │           ├── health.py   # /health probe (GPU / device readiness check)
│   │           ├── chat.py     # /chat & /stream (GenAI real-time token streaming)
│   │           ├── rag.py      # /rag/query (Retrieval-Augmented Generation)
│   │           └── predict.py  # /predict (Classical ML / Deep Learning inference)
│   │
│   ├── core/
│   │   ├── __init__.py
│   │   └── config.py           # LLM_MODEL, EMBEDDING_MODEL, DEVICE, API_KEYS
│   │
│   ├── ai/                     # GenAI, LLMs & Retrieval Engine
│   │   ├── __init__.py
│   │   ├── llm.py              # LLM client & async token streaming generator
│   │   ├── prompt.py           # System prompts & prompt engineering templates
│   │   └── rag.py              # Vector retriever & RAG context builder
│   │
│   ├── ml/                     # Machine Learning & Deep Learning Engine
│   │   ├── __init__.py
│   │   ├── model.py            # Model wrapper (PyTorch / ONNX / Transformers)
│   │   └── pipeline.py         # Tensor preprocessing & postprocessing
│   │
│   ├── schemas/                # Pydantic request & response models
│   │   ├── __init__.py
│   │   ├── chat.py             # ChatRequest, ChatResponse, ChatMessage
│   │   ├── rag.py              # RAGRequest, RAGResponse, DocumentChunk
│   │   └── predict.py          # PredictionRequest, PredictionResponse, HealthResponse
│   │
│   └── services/               # Business logic & latency tracking
│       ├── __init__.py
│       ├── ai_service.py       # GenAI orchestration (Chat, Stream, RAG)
│       └── inference.py        # ML inference orchestration & latency benchmarking
│
├── tests/                      # Automated test suite
│   ├── __init__.py
│   ├── conftest.py             # TestClient fixtures + mock AI/ML providers
│   ├── test_chat.py            # Pytest test cases for chat & RAG
│   └── test_predict.py         # Pytest test cases for /health & /predict
│
├── .gitignore                  # Ignores __pycache__, .env, and large model weights (*.pt, *.bin)
├── .env                        # Local model configuration
├── .env.example
├── requirements.txt            # Ready for pip install -r requirements.txt
├── pyproject.toml              # Build & pytest configuration
└── README.md

---

🚀 Running Your Service

1. Create and activate a virtual environment

Windows PowerShell:

python -m venv .venv
.\.venv\Scripts\Activate.ps1

Windows CMD:

python -m venv .venv
call .venv\Scripts\activate.bat

macOS / Linux:

python3 -m venv .venv
source .venv/bin/activate

2. Install dependencies

pip install -r requirements.txt

3. Start the development server

python -m uvicorn app.main:app --reload

---

🔍 Interactive Endpoints

Visit http://127.0.0.1:8000/docs for interactive Swagger UI:

Method	Endpoint	Description
GET	/	API status and capabilities overview
GET	/api/v1/health	Health & device readiness probe (cpu or cuda)
POST	/api/v1/chat/	Standard GenAI chat completion
POST	/api/v1/chat/stream	Real-time token-by-token streaming (SSE)
POST	/api/v1/rag/query	Retrieval-Augmented Generation (Vector Search + LLM)
POST	/api/v1/predict/	ML model inference with execution latency tracking

---

🧪 Running Automated Tests

Run the included Pytest suite:

python -m pytest

Output:

tests\test_chat.py ..                                                    [ 50%]
tests\test_predict.py ..                                                 [100%]
============================== 4 passed in 0.15s ==============================

---

🛠 Customizing Your AI & ML Models

Connecting Real LLMs (OpenAI, Anthropic, Gemini, Ollama)

In app/ai/llm.py, plug your provider call directly into generate_response() and stream_tokens():

from openai import AsyncOpenAI
client = AsyncOpenAI(api_key=settings.OPENAI_API_KEY)

async def stream_tokens(self, messages):
    response = await client.chat.completions.create(
        model=self.model_name,
        messages=[m.model_dump() for m in messages],
        stream=True,
    )
    async for chunk in response:
        content = chunk.choices[0].delta.content or ""
        yield f"data: {content}\n\n"

Loading PyTorch / ONNX Weights

In app/ml/model.py, initialize your local weights inside load():

import torch

def load(self):
    self.model = torch.load(settings.ML_MODEL_PATH, map_location=settings.DEVICE)
    self.model.eval()
    self.is_loaded = True

---

📜 License

This project is open-source software licensed under the MIT License.