marziel 0.9.1

Private AI with real-time maritime intelligence — runs entirely on your hardware

---

language:

• en license: mit tags:
• llama
• mlx
• vllm
• 4-bit
• local-ai
• private
• maritime
• vessel-tracking
• osint library_name: mlx pipeline_tag: text-generation base_model: mistralai/Ministral-8B-Instruct-2410

---

🧠 Marziel AI — v0.9.0

Private Local Intelligence — runs entirely on your hardware.

One command to run your own AI — with Hybrid RAG and TurboQuant KV cache.

Marziel AI is a locally-powered AI assistant with hybrid RAG document retrieval, TurboQuant 3-bit KV cache compression, real-time maritime intelligence, web search, browser automation, GitHub analysis, URL reading, code generation, and autonomous agent capabilities — all without sending a single byte to the cloud.

Quick Install

pip install marziel
marziel serve

That's it. The right engine is auto-installed for your hardware. The first run downloads the model from Hugging Face. Everything is automatic.

Changelog

v0.9.0

• Hybrid RAG System — upload PDFs/docs, get answers with citations using dual retrieval:
  • PageIndex — reasoning-based tree search (LLM navigates document structure)
  • Vector Search — fastembed BGE-small cosine similarity (zero-config SQLite store)
  • Score fusion: 60% reasoning + 40% vector for best accuracy
• TurboQuant KV Cache — 3-bit KV cache quantization with PolarQuant + QJL error correction:
  • 3x memory reduction with 0.91+ cosine similarity
  • Works on both GGUF (NVIDIA/CPU) and MLX (Apple Silicon)
  • SSD offloading for cold KV cache layers
• oMLX Engine — Apple Silicon inference upgraded to oMLX with tiered KV cache (RAM + SSD), continuous batching, and prefix sharing
• RAG API — POST /api/rag/upload, POST /api/rag/query, GET /api/rag/documents
• TurboQuant API — GET /api/turboquant/status, POST /api/turboquant/benchmark
• Auto-context injection — uploaded documents automatically augment chat responses
• Removed Ollama fallback — Marziel uses its own engines exclusively
• Added numpy, fastembed, PyMuPDF to core dependencies

v0.8.0

• v6 Model — upgraded to Ministral-8B base with full fine-tune + LoRA merge
• Professional code quality — typed Python, async patterns, SQL builders, Result types
• Web scraping — Playwright, BeautifulSoup, rate limiting, PostgreSQL upserts, robots.txt
• Agentic tool use — multi-step reasoning, file analysis, autonomous task execution
• GGUF Q4_K_M (4.8GB) + MLX 4-bit (4.5GB) on HuggingFace
• HuggingFace model card with usage for GGUF, MLX, and safetensors
• Install auto-detects OS and installs the correct backend

v0.7.0

• v5 Model — fine-tuned on 2,061 samples with new capabilities
• Tool use — web search, URL reading, code execution patterns
• Advanced code — decorators, REST APIs, TypeScript, CSS Grid/Flexbox
• Math & reasoning — calculus, combinatorics, Big O, statistics
• Maritime expertise — EEXI/CII, ballast water, P&I, charter types, chokepoints
• Training: eval loss 0.4363, token accuracy 88.2%

v0.6.0

• Auto-install semantic router — semantic-router + fastembed installed automatically on first marziel serve
• No more pip install marziel[router] — everything works out of the box

v0.5.9

• Semantic routing — intent detection via vector similarity (~5ms), replaces brittle keyword matching
• 8 intent routes — chat, web_search, agent, browser, maritime, code, github, url
• fastembed BGE-small — local embeddings, no API calls, 95% routing accuracy
• Graceful degradation — falls back to keywords if semantic-router not installed
• Install: pip install marziel[router] for semantic routing

v0.5.8

• 3-tier inference — Apple Silicon → MLX, NVIDIA GPU → vLLM, CPU → llama.cpp
• llama.cpp for CPU — uses same 4.9GB GGUF file as GPU, no more vLLM CPU format issues
• Auto-install engines — mlx-lm, vllm, or llama-cpp-python[server] detected + installed automatically
• Chat-first UI — opens directly to chat interface (no landing page)
• Python 3.14 support — tested on 3.10–3.14
• Works on any Linux — Arch, Fedora, RHEL, Ubuntu, Debian + macOS + WSL

v0.5.7

• CPU uses safetensors — vLLM CPU loads model.safetensors from HF (GGUF not supported on CPU)
• GPU uses GGUF — faster local loading with quantized model
• Fixed tokenizer — PreTrainedTokenizerFast in HF tokenizer_config.json

v0.5.6

• Clean install logic — only installs vLLM if missing; detects GPU → CUDA, no GPU → CPU wheel
• No forced uninstalls — respects existing user environment

v0.5.5

• Tokenizer fix — --trust-remote-code for custom tokenizer loading
• Clean CPU install — auto-uninstalls CUDA vllm/torch before installing CPU wheel

v0.5.4

• vLLM CPU pre-built wheel — auto-installs vllm+cpu from GitHub releases on CPU-only machines
• GPU auto-detects — installs standard CUDA vLLM when NVIDIA GPU present
• CPU optimized — bfloat16 dtype, VLLM_CPU_KVCACHE_SPACE, TCMalloc, Intel OpenMP
• 3-minute CPU timeout — 180s for CPU model loading

v0.5.3

• Fixed CPU device detection — set VLLM_TARGET_DEVICE=cpu for machines without GPU
• vLLM debug logging — errors now saved to ~/.marziel/vllm.log

v0.5.2

• Prefix caching — system prompt KV cache reused across requests (faster responses)
• 120s CPU timeout — longer startup window for CPU-only machines loading 4.6GB model
• All NVIDIA GPUs supported (GTX, RTX, Tesla, A100) — auto-detected via nvidia-smi

v0.5.1

• vLLM auto-installs — no more manual pip install vllm, it's a declared dependency
• 8GB RAM optimized — reduced context to 2048 tokens, eager execution, swap-space fallback
• Engine routing — Apple Silicon → MLX, everything else → vLLM (CPU & GPU)

v0.5.0

• Unified architecture: MLX (Apple Silicon) + vLLM (all other platforms)
• GGUF Q4_K_M model for vLLM (~4.6 GB)
• Auto-download model from Hugging Face
• Zero-config: pip install marziel && marziel serve

What It Does

Feature	Description
🧠 AI Chat	Conversational AI with personality, code generation, data analysis
📡 Maritime Intelligence	Real-time GDACS, GDELT, NASA EONET, AIS, market data, bunker prices
💰 Market Data	Live Brent crude, BDI, EU Carbon, freight rates, forex, 12-port bunker prices
🚢 Vessel Integration	AIS tracking, fleet management, voyage data, maintenance records
🔌 Third-Party Connectors	Plug in MarineTraffic, VesselFinder, Spire, Datalastic, or any REST API
📊 Live Reports	Generate comprehensive maritime intelligence reports on demand
🔍 Web Search	Live internet results via DuckDuckGo — always up-to-date answers
🌐 Browser Control	Headless Chromium automation — navigate, click, type, screenshot
🔗 URL Analysis	Share any link — reads, extracts, and analyzes content
🐙 GitHub Analysis	Paste any repo URL — get architecture, languages, README analysis
⚡ Autonomous Agent	Multi-step reasoning with bash, file, web, browser, and maritime tools
📑 Hybrid RAG	Upload documents → reasoning + vector search → answers with citations
⚡ TurboQuant	3-bit KV cache quantization — 3x memory reduction, 0.91+ cosine similarity
🔒 100% Private	Everything runs locally. Zero cloud dependency

Maritime Intelligence (Enterprise)

Marziel integrates real-time maritime data from multiple global sources:

┌───────────────────────────────────────────────────────────┐
│                   MARZIEL MARITIME INTEL                   │
├───────────────────────────────────────────────────────────┤
│  OSINT (Free APIs)         │  Platform Integration        │
│  ─────────────────         │  ────────────────────        │
│  📡 GDACS — Disasters      │  🚢 Fleet Vessels            │
│  📰 GDELT — Geopolitics    │  🗺️  Active Voyages          │
│  🌍 NASA EONET — Climate   │  ⚠️  Risk Zones              │
│  💰 Yahoo Finance — Market │  🏗️  Maintenance             │
│  💱 ExchangeRate — Forex   │  📋 Incidents                │
│  🚢 AISStream — Tracking   │  ⛽ Fuel Records             │
│                            │  🏭 Port Directory           │
├───────────────────────────────────────────────────────────┤
│  Third-Party Connectors (Enterprise)                      │
│  ─────────────────────────────────                        │
│  MarineTraffic · VesselFinder · Spire Maritime            │
│  Datalastic · OpenWeather · Custom REST API               │
└───────────────────────────────────────────────────────────┘

Market Data Sources

Data	Source	Update
Brent Crude Oil	Yahoo Finance (BZ=F)	Real-time
Bunker VLSFO/MGO/HSFO	Brent-derived (12 ports)	Real-time
Baltic Dry Index	BDRY ETF × 200	Real-time
EU Carbon (EUA)	KRBN ETF × 2.1	Real-time
Freight Rates	DSX/GNK shipping stocks	Real-time
Exchange Rates	ExchangeRate API (9 currencies)	Real-time

Third-Party Integration

Connect Marziel to any maritime service via ~/.marziel/connectors.json:

{
  "connectors": [
    {"type": "marinetraffic", "api_key": "YOUR_KEY"},
    {"type": "openweather", "api_key": "YOUR_KEY"},
    {
      "type": "custom",
      "name": "my_fleet",
      "base_url": "https://api.myfleet.com/v1",
      "api_key": "secret",
      "endpoints": {
        "vessels": "/ships",
        "voyages": "/trips",
        "weather": "/weather?lat={lat}&lon={lon}"
      }
    }
  ]
}

Maritime API Endpoints

GET  /api/maritime/report           — Full intelligence report
GET  /api/maritime/data/market      — Live market data
GET  /api/maritime/data/bunker      — 12-port bunker prices
GET  /api/maritime/data/fx          — Exchange rates
GET  /api/maritime/data/gdacs       — Disaster events
GET  /api/maritime/data/gdelt       — Geopolitical news
GET  /api/maritime/data/nasa        — Natural events
GET  /api/maritime/data/ais         — Vessel tracking
GET  /api/maritime/data/vessels     — Fleet data
GET  /api/maritime/data/voyages     — Voyage data
GET  /api/maritime/data/risk-zones  — Active risk zones
GET  /api/maritime/data/ports       — Port directory
GET  /api/maritime/connectors       — List connectors
POST /api/maritime/connectors/query — Query third-party services

Agent Tools

The LLM has access to 20+ tools:

Tool	Purpose
bash	Shell command execution
file_read / file_write	File I/O
web_search	DuckDuckGo search
browser	Headless Chromium automation
maritime_report	Full intelligence report
market_data	Brent, BDI, Carbon, bunker, freight
gdacs	Active disasters & cyclones
gdelt	Geopolitical conflict news
nasa	NASA EONET natural events
ais	Live AIS vessel tracking
fleet	Fleet vessel data
voyages	Active voyage data
risk_zones	Active risk zones
ports	Global port directory
exchange_rates	Live forex rates
maintenance	Vessel maintenance records
incidents	Maritime incident reports
fuel	Fuel consumption data
port_calls	Port call history

Requirements

• RAM: 8GB minimum (16GB recommended)
• Disk: ~5GB for the model
• Python: 3.10–3.14
• Supported platforms:

Platform	Engine	Model	Auto-installed
Apple Silicon (M1–M4)	MLX	MLX 4-bit (4.5GB)	mlx-lm
NVIDIA GPU (any)	vLLM	GGUF Q4_K_M (4.8GB)	vllm
CPU (any x86/ARM)	llama.cpp	GGUF Q4_K_M (4.8GB)	llama-cpp-python[server]

• Linux: Ubuntu, Debian, Arch, Fedora, RHEL, Alpine — any distro with glibc
• macOS: Intel (llama.cpp) or Apple Silicon (MLX)
• Windows: WSL2 recommended

Usage

# Install and start (basic)
pip install marziel
marziel serve

# With browser automation
pip install "marziel[browser]"
python -m playwright install chromium
marziel serve

# Check status
marziel status

# Show version
marziel version

Then open http://localhost:8001 — your private AI dashboard.

Architecture

┌──────────────────────────────────────────────┐
│  Browser (localhost:8001)                     │
│  ┌────────────────────────────────────────┐   │
│  │  Marziel Dashboard (React)            │   │
│  │  Dashboard · Chat · Settings          │   │
│  └──────────────┬─────────────────────────┘   │
│                 │ API calls                    │
│  ┌──────────────▼─────────────────────────┐   │
│  │  Marziel Backend (Flask+Gunicorn)     │   │
│  │  Agent · Search · Browser · Maritime  │   │
│  │  Billing · Connectors · Reports       │   │
│  └──────┬───────┬───────┬─────────────────┘   │
│         │       │       │                      │
│  ┌──────▼──┐ ┌──▼────┐ ┌▼──────────────────┐  │
│  │ LLM     │ │ OSINT │ │ Third-Party APIs   │  │
│  │ MLX     │ │ GDACS │ │ MarineTraffic      │  │
│  │ vLLM    │ │ GDELT │ │ VesselFinder       │  │
│  │ llama   │ │ NASA  │ │ Spire · Custom     │  │
│  │ .cpp    │ │ Yahoo │ │                    │  │
│  └─────────┘ └───────┘ └────────────────────┘  │
│                YOUR MACHINE                     │
└──────────────────────────────────────────────┘

Pricing

Plan	Price	Features
Free	€0/mo	AI Chat, Web Search, URL Analysis, GitHub Analysis
Pro	€20/mo	+ Agent Mode, Browser Control, Priority Inference
Enterprise	€199/mo	+ Maritime Intelligence, Connectors, Reports, Custom Fine-tuning

Links

• 🌐 Website: marziel.com
• 💻 GitHub: github.com/efops/marziel
• 🤗 Model: huggingface.co/efops/marziel-8b-custom
• 📦 PyPI: pypi.org/project/marziel

Contact

📧 [e-mail]

License

MIT License

---

Built with ❤️ by Efe (Efkan Isazade)