surrealengine 1.0.1

A powerful, asynchronous Object-Graph Mapper (OGM) and utility library for SurrealDB.

SurrealEngine

Replace Your Multi-Database Stack with One Pythonic ORM

Now generally available as v1.0.0

Documents, graphs, vectors, real-time, and analytics—without the operational nightmare. Powered by SurrealDB. Built for Python developers.

Vector Search, Graph Queries, Live Updates, and Zero-Copy Analytics — All in One ORM.

SurrealEngine is the comprehensive Python ORM for SurrealDB, delivering capabilities that previously required multiple databases and complex distributed systems. You get familiar object mapping plus native support for graphs, vectors, real-time queries, and advanced analytics.

Whether you're building a Real-Time Recommendation Engine, an AI-Powered Search Service, or a High-Frequency Data Pipeline, SurrealEngine provides a unified Pythonic API without the operational complexity of managing multiple databases.

---

🚀 The "Magic" Example

Why choose SurrealEngine? Because you can do this in a single query:

# The "Why SurrealEngine?" Example:
# Vector Search + Graph Traversal — in one query.
# Try doing THIS with PostgreSQL + Neo4j + Pinecone.

similar_users = await User.objects \
    .semantic_search(field="embedding", vector=user_vector, k=10) \
    .out("friends") \
    .out(Person) \
    .all()
# Result: Top 10 similar users' friends (Deep traversal)

---

Vector Query Migration

Preferred style (new):

results = await Article.objects \
    .semantic_search(field="embedding", vector=query_embedding, k=10) \
    .filter(published=True) \
    .all()

Legacy-compatible style (still supported):

results = await Article.objects \
    .filter(embedding__knn=(query_embedding, 10)) \
    .filter(published=True) \
    .all()

Use metric="COSINE" (or other supported metrics) when your model index metadata does not uniquely define a dist value.

---

Upgrade to 1.0

• Preferred vector query API is now semantic_search(...) for readability and fluent chaining.
• Existing embedding__knn=(vector, k) remains supported for backward compatibility.
• If metric inference is ambiguous or missing from Meta.indexes, pass metric="COSINE" (or another supported metric) explicitly.

# Preferred in v1.0
results = await Article.objects.semantic_search(
    field="embedding",
    vector=query_embedding,
    k=10,
    metric="COSINE",
).filter(published=True).all()

Release-note migration example (raw SurrealQL -> SurrealEngine fluent API):

# Before (raw SurrealQL)
rows = await conn.client.query(
    "SELECT id, title, content FROM articles WHERE embedding <|8,COSINE|> $vec",
    {"vec": query_embedding},
)

# After (v1.0 preferred)
rows = await Article.objects.semantic_search(
    field="embedding",
    vector=query_embedding,
    k=8,
    metric="COSINE",
).all()

---

🤔 Why SurrealEngine?

Before SurrealEngine:

from pymongo import MongoClient        # Documents
from neo4j import GraphDatabase        # Graphs
from pinecone import Index             # Vectors
from redis import Redis                # Real-time
import pandas as pd                    # Analytics

With SurrealEngine:

from surrealengine import Document, create_connection

# Everything you need

One database. One connection. One API. Zero operational complexity.

---

🆚 How It Compares

Need	Traditional Approach	With SurrealEngine
Documents	MongoDB + PyMongo	✅ Built-in
Graphs	Neo4j + Driver	✅ Built-in
Vectors	Pinecone/Weaviate	✅ Built-in
Real-Time	Redis/Firebase	✅ Built-in
Analytics	ClickHouse/Pandas	✅ Built-in
Databases to Manage	3-5	1
APIs to Learn	3-5	1
Connection Pools	3-5	1
Failure Modes	Many	Few

---

📦 Installation

We strongly recommend using uv for 10-100x faster package installation and resolution but pip and poetry work too.

Using uv (Recommended)

uv add surrealengine
# Optional extras:
# uv add "surrealengine[signals]"  # For pre/post save hooks
# uv add "surrealengine[data]"     # For PyArrow/Polars support
# uv add "surrealengine[jupyter]"  # For Jupyter Notebook support

Using pip

pip install surrealengine
# Optional: pip install "surrealengine[signals, data]"

---

⚡ Quick Start

1. Connect (Sync or Async)

SurrealEngine auto-detects your context. Use async_mode=True for async apps (FastAPI), or defaults to sync for scripts.

from surrealengine import create_connection

# Async connection (e.g. FastAPI)
await create_connection(
    url="ws://localhost:8000/rpc",
    namespace="test", database="test",
    username="root", password="root",
    async_mode=True
).connect()

# OR Sync connection (e.g. Scripts)
create_connection(
    url="ws://localhost:8000/rpc",
    namespace="test", database="test",
    username="root", password="root",
    async_mode=False
)

2. Define Your Model

from surrealengine import Document, StringField, IntField

class Person(Document):
    name = StringField(required=True)
    age = IntField()

    class Meta:
        collection = "person"
        indexes = [
            # HNSW Vector Index
            {"name": "idx_vector", "fields": ["embedding"], "dimension": 1536, "dist": "COSINE", "m": 16},
        ]

3. Polyglot Usage (Same API!)

# Create
# In async function: await Person(name="Jane", age=30).save()
# In sync script:    Person(name="Jane", age=30).save()
jane = await Person(name="Jane", age=30).save()

# Query with Pythonic Syntax (Overloaded Operators)
# Or use Django-style: Person.objects.filter(age__gt=25)
people = await Person.objects.filter(Person.age > 25).all()

# Graph Relations & Traversal
await jane.relate_to("knows", other_person)

# Traversal - Two Ways:

# 1. Edge Only (Lazy/Dict Result) -> returns list of dicts {out: ..., in: ...}
# Equivalent to: SELECT ->knows->? FROM person:jane
relations = await Person.objects.filter(id=jane.id).out("knows").all()

# 2. Edge + Node (Hydrated Documents) -> returns list of Person objects
# Equivalent to: SELECT ->knows->person.* FROM person:jane
friends = await Person.objects.filter(id=jane.id).out("knows").out(Person).all()

# Chain traversals freely:
# Friends of friends (Hydrated)
fof = await Person.objects.filter(id=jane.id).out("knows").out("knows").out(Person).all()

# 3. Magic Accessor (.rel)
# Simple access to relationships from a document instance
friends = await jane.rel.knows(Person).all()

4. Advanced Performance

# Zero-Copy Data Export (10-50x faster)
# Export directly to Arrow/Polars without Python object overhead
df = await Person.objects.all().to_polars()
arrow_table = await Person.objects.all().to_arrow()

# Advanced Aggregation Pipeline
# "Find high-revenue categories with VIP activity"
from surrealengine import Sum, Mean, CountIf, DistinctCount

stats = await Transaction.objects.aggregate() \
    .match(status="success", created_at__gt="2024-01-01") \
    .group(
        by_fields=["category", "region"],
        total_revenue=Sum("amount"),
        avg_ticket=Mean("amount"),
        vip_transactions=CountIf("amount > 1000"),
        unique_customers=DistinctCount("user_id")
    ) \
    .having(total_revenue__gt=50000, vip_transactions__gte=10) \
    .sort(total_revenue="DESC") \
    .limit(5) \
    .execute()

---

🎬 See It In Action

Real-World Examples

Recommendation Engine (Vector + Graph)

# Find products similar to what user liked (vector)
# that were purchased by friends (graph)
friend_ids = [u.id for u in await user.rel.friends(User)]

recommendations = await Product.objects \
    .semantic_search(field="embedding", vector=user_preference_vector, k=20) \
    .in_("purchased") \
    .filter(id__inside=friend_ids) \
    .all()

Analytics Dashboard

# MongoDB-style aggregation pipeline
stats = await Order.objects.aggregate() \
    .group("category", revenue=Sum("amount")) \
    .sort(revenue="DESC") \
    .limit(5) \
    .execute()

AI-Powered Search

# Semantic search + full-text filters
# using HNSW vector index and BM25 full-text search

results = await Article.objects \
    .semantic_search(field="embedding", vector=query_embedding, k=10) \
    .filter(content__search="machine learning") \
    .filter(published=True) \
    .all()

# FTS ergonomic aliases (both compile to @@)
docs = await Article.objects.filter(title__search="surrealdb").all()
docs = await Article.objects.filter(title__match="surrealdb").all()

👉 See 20+ more examples in the docs

---

📚 Documentation

Full documentation is available at https://iristech-systems.github.io/SurrealEngine-Docs/

👉 Read the State of SurrealEngine announcement!

---

✨ Features

Feature	Status	Notes
Polyglot API	✅ Supported	(New in v0.7.0) Write code once, run in both Sync (WSGI/Scripts) and Async (ASGI/FastAPI) contexts naturally.
Aggregations	✅ Supported	MongoDB-style aggregation pipelines (.aggregate().group(...).execute()) for complex analytics.
Materialized Views	✅ Supported	Define pre-computed views using Document.create_materialized_view() for high-performance analytics.
Connection Pooling	✅ Supported	Full support for async pooling (auto-reconnect, health checks). Sync pooling available via Queue.
Live Queries	⚠️ Partial	Supported on WebSocket connections (ws://, wss://). NOT supported on embedded (mem://, file://).
Graph Traversal	✅ Supported	Fluent API (.out().in_()) and Magic .rel accessor (user.rel.friends(User)).
Change Tracking	✅ Supported	Objects track dirty fields (.is_dirty, .get_changes()) to optimize UPDATE queries.
Schema Generation	✅ Supported	Can generate DEFINE TABLE/FIELD statements from Python classes.
Embedded Docs	✅ Supported	Define structured nested objects with EmbeddedDocument. Generates schema automatically.
Stored Functions	✅ Supported	Define SurrealDB functions directly in Python using @surreal_func.
Vector Search	✅ Supported	Support for HNSW indexes via Meta.indexes (dimension, dist, etc.).
Full Text Search	✅ Supported	Support for BM25 and Highlights via Meta.indexes.
Events	✅ Supported	Define triggers via Meta.events using the Event class.
Data Science	✅ Supported	Zero-copy export to PyArrow (.to_arrow()) and Polars (.to_polars()). Requires surrealengine[data].
Pydantic	✅ Compatible	RecordID objects (SDK v1.0.8+) are Pydantic-compatible.

---

⚠️ Sharp Edges & Limitations

SurrealEngine is designed to be a robust, high-level abstraction. However, be aware of these known limitations:

1. Embedded Connections & Live Queries: Attempting to use .live() on a mem:// or file:// connection will raise a NotImplementedError. The underlying SDK's embedded connector does not currently support the event loop mechanism required for live subscriptions.

2. RecordID Fields: The RecordID object from the SDK has table_name and id attributes. Gotcha: Do not assume it has a .table attribute (it does not). Always use .table_name. Gotcha: When parsing strings manually, prefer the SDK's RecordID.parse("table:id") over manual string splitting to handle escaped characters correctly.

3. Strict Mode: By default, Document classes have strict=True. Initializing a document with unknown keyword arguments will raise an AttributeError. Set strict=False in Meta if you need to handle dynamic unstructured data.

4. Auto-Connect: When using create_connection(..., auto_connect=True), the connection is established immediately. For async connections, ensure this is called within a running event loop.

5. Transactions: SurrealDB supports transactions (BEGIN, COMMIT, CANCEL). SurrealEngine provides a helper await connection.transaction([coro1, coro2]) that ensures atomicity via connection pinning when using pools.

---

🎯 Get Started

uv add surrealengine

Next Steps:

• 📖 Read the Docs
• 💬 Join Discussions
• 🐛 Report Issues
• ⭐ Star on GitHub (if you find it useful!)

---

Built with ❤️ by Iristech Systems
Powered by SurrealDB