meridian-oss 1.2.2

Heroku for ML Features

Meridian

The Context Store for LLMs & ML Features

Define RAG pipelines and ML features in Python. Get production retrieval, vector search, and training data for free.

Stop fighting the infrastructure tax. Meridian takes you from "Notebook Prototype" to "Production RAG" in 30 seconds.

🆕 v1.2.0: Context Store for LLMs with RAG, vector search (pgvector), and token budgets.

📚 Documentation | 🤖 Context Store | 🛠️ Testing Guide

⚡ The 30-Second Quickstart

Option A: The "I just want to see it work" (Clone & Run)

git clone https://github.com/davidahmann/meridian.git
cd meridian
pip install -e ".[ui]"
meridian serve examples/basic_features.py

Option B: The "Builder" (Pip Install)

1. Install Meridian

pip install "meridian-oss[ui]"

2. Create a file named my_features.py:

from meridian.core import FeatureStore, entity, feature
from datetime import timedelta
import random

store = FeatureStore()

@entity(store)
class User:
    user_id: str

@feature(entity=User, refresh=timedelta(minutes=5), materialize=True)
def user_click_count(user_id: str) -> int:
    return len(user_id) + random.randint(0, 100)

3. Serve it immediately:

meridian serve my_features.py
# 🚀 Meridian server running on http://localhost:8000

4. Query it:

curl -X POST http://localhost:8000/features \
  -H "Content-Type: application/json" \
  -d '{"entity_name": "User", "entity_id": "u1", "features": ["user_click_count"]}'
# Output: {"user_click_count": 42}

---

🚀 Why Meridian?

Most feature stores are built for the 1% of companies (Uber, DoorDash) with platform teams. They require Kubernetes, Spark, and complex microservices.

Meridian is built for the rest of us.

Feature	The "Old Way"	The Meridian Way
Config	500 lines of YAML	Python Decorators (@feature)
Infra	Kubernetes + Spark	Runs on your Laptop (DuckDB)
Serving	Complex API Gateway	meridian serve file.py
Philosophy	"Google Scale"	"Get it Shipped"

Key Features

• Context Store for LLMs (v1.2.0): Full RAG infrastructure with vector search (pgvector), token budgets, and priority-based context assembly.
• Local-First, Cloud-Ready: Runs on your laptop with zero dependencies (DuckDB + In-Memory). Scales to production with boring technology (Postgres + Redis).
• No Magic: Your code is your config. Explicit caching (materialize=True) and explicit refresh logic.
• Production Reliability: Built-in circuit breakers, fallback chains (Cache -> Compute -> Default), and Prometheus metrics (meridian_feature_requests_total).
• Self-Healing: Run meridian doctor to instantly diagnose config and connectivity issues.
• Rich UI & TUI: Includes a Streamlit dashboard with Visual Dependency Graphs and a production-grade Terminal UI for live monitoring.
• Hybrid Features (v1.1.0): Mix Python logic (for complex math) and SQL (for heavy joins) in the same API.
• Point-in-Time Correctness (v1.1.0): Zero data leakage using ASOF JOIN (DuckDB) and LATERAL JOIN (Postgres).

---

🏗️ Architecture

Meridian is designed to grow with you.

graph TD
    subgraph Dev [Tier 1: Local Development]
        A[Laptop] -->|Uses| B(DuckDB)
        A -->|Uses| C(In-Memory Dict)
        style Dev fill:#e1f5fe,stroke:#01579b
    end

    subgraph Prod [Tier 2: Production]
        D[API Pods] -->|Async| E[(Postgres)]
        D -->|Async| F[(Redis)]
        style Prod fill:#fff3e0,stroke:#ff6f00
    end

    Switch{MERIDIAN_ENV} -->|development| Dev
    Switch -->|production| Prod

Tier 1: Local Development (The "Wedge")

• Perfect for prototyping and single-developer projects.
• Offline Store: DuckDB (Embedded)
• Online Store: Python Dictionary (In-Memory)
• Infra: None (Just pip install)

Tier 2: Production (The "Standard")

• Robust, scalable, and boring.
• Offline Store: Postgres / Snowflake / BigQuery
• Online Store: Redis
• Vector Store: pgvector (Postgres extension)
• Infra: 1x Postgres (with pgvector), 1x Redis, Nx API Pods

🔧 Configuration

Meridian uses 12-factor app configuration via Environment Variables.

Variable	Description	Default
MERIDIAN_ENV	Environment mode (development or production)	development
MERIDIAN_REDIS_URL	Redis Connection String	redis://localhost:6379/0
MERIDIAN_POSTGRES_URL	Postgres Connection String	postgresql+asyncpg://...
MERIDIAN_API_KEY	Master API Key for Server Authentication	None (Public in Dev)
OPENAI_API_KEY	Required for Vector Embeddings & Context Assembly	None
COHERE_API_KEY	Alternative embedding provider (Cohere)	None
MERIDIAN_EMBEDDING_MODEL	Embedding model for vector search	text-embedding-3-small
MERIDIAN_EMBEDDING_CONCURRENCY	Max concurrent embedding requests	10
MERIDIAN_PG_POOL_SIZE	Postgres Connection Pool Size	5
MERIDIAN_PG_MAX_OVERFLOW	Postgres Connection Pool Overflow	10

---

📚 Advanced Usage

1. Context Store & RAG (v1.2.0)

Meridian isn't just for numbers. It's a full Context Infrastructure for LLMs.

from meridian.retrieval import retriever
from meridian.context import context, Context, ContextItem

# Index documents (auto-chunks and embeds)
await store.index(index_name="docs", entity_id="doc_1", text="Meridian is awesome...")

# Define retriever for semantic search
@retriever(store, index="docs", top_k=3)
async def search_docs(query: str) -> list[str]:
    pass  # Meridian handles vector search via pgvector

# Assemble context with token budget
@context(store, max_tokens=4000)
async def chat_context(user_id: str, query: str) -> Context:
    docs = await search_docs(query)
    user_prefs = await store.get_feature("user_preferences", user_id)
    return Context(items=[
        ContextItem("You are a helpful assistant.", priority=0, required=True),
        ContextItem(docs, priority=1, required=True),
        ContextItem(f"User prefs: {user_prefs}", priority=2),  # Truncated first
```python
    ])

Debug Context Assembly:

# View exactly why documents were included/dropped
meridian context <id>

📖 Full Context Store Documentation →

2. Time Travel

Debug production issues by querying the state of the world as it was yesterday.

# Create a context that travels back in time
with time_travel(datetime.now() - timedelta(days=1)):
    # All feature queries in this block return historical values from Postgres
    print(user_click_count("u1"))

3. Event-Driven Updates

Meridian listens to your event bus (Redis Streams) to push fresh features instantly.

@feature(entity=User, trigger="transaction_event")
def last_transaction_amount(user_id: str, event: AxiomEvent) -> float:
    return event.payload["amount"]

Monitor Events via CLI:

meridian events listen --stream transaction_event

4. Manage Indexes

Create and inspect vector indexes directly from the CLI.

# Create a new index with 1536 dimensions
meridian index create my_docs --dimension 1536

# Check status (row count)
meridian index status my_docs

---

🏭 Production Configuration

Copy this to your .env or .env.production file:

# Security
MERIDIAN_API_KEY=change_me_to_something_secure

# Data Stores (Required for Prod)
MERIDIAN_REDIS_URL=redis://localhost:6379
MERIDIAN_POSTGRES_URL=postgresql://user:password@localhost:5432/meridian  # pragma: allowlist secret

# LLM Providers (Required for Context Store)
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
COHERE_API_KEY=...

# Tuning
MERIDIAN_EMBEDDING_CONCURRENCY=10
MERIDIAN_POSTGRES_POOL_SIZE=20

---

🔍 Request Lifecycle

sequenceDiagram
    participant C as Client
    participant S as Server
    participant G as Resolver
    participant O as Online (Redis)
    participant D as Offline (Postgres)

    C->>S: POST /ingest/event
    S->>O: Publish Event
    Note over O: Async Worker picks up Event

    C->>S: GET /features
    S->>G: Resolve Dependencies
    G->>O: Get Cached Values
    alt Cache Miss
        G->>D: Get Historical Data (if needed)
        G->>G: Compute Derived Features
        G->>O: Update Cache
    end
    G-->>S: Return Features
    S-->>C: JSON Response

---

🗺️ Roadmap

• ✅ Phase 1: Core API, DuckDB/Postgres support, Redis caching, FastAPI serving, PIT Correctness, Async I/O.
• ✅ Phase 2 (v1.2.0): Context Store, RAG infrastructure, pgvector, Event-Driven features, Time Travel.
• 🚧 Phase 3: Drift detection, RBAC, and multi-region support.

🤝 Contributing

We love contributions! This is a community-driven project. Please read our CONTRIBUTING.md to get started.

📄 License

Apache 2.0