sibyl-core 0.9.0

Core library for Sibyl - domain models, graph operations, and knowledge retrieval

sibyl-core

Core library for Sibyl. Domain models, graph operations, retrieval algorithms, and tool implementations. Shared foundation for the API server and CLI.

Quick Reference

# Install
uv add sibyl-core

# Development
moon run core:lint        # Ruff check
moon run core:typecheck   # ty
moon run core:test        # Pytest

What's Here

• models/: Domain entities (Task, Project, Epic, Source, etc.)
• graph/: SurrealDB graph managers plus Graphiti compatibility adapters
• backends/surreal/: SurrealDB driver, schema, and per-table ops
• retrieval/: Native context retrieval, compatibility search, fusion, deduplication
• tools/: MCP tool implementations (search, explore, add, manage)
• tasks/: Workflow engine, dependency resolution
• auth/: JWT primitives, password hashing

Structure

src/sibyl_core/
├── models/
│   ├── entities.py       # Entity, EntityType, base classes
│   ├── tasks.py          # Task, Project, Epic, Milestone
│   ├── sources.py        # Source, Document
│   └── responses.py      # API response models
├── graph/
│   ├── client.py         # GraphClient (connection, write lock)
│   ├── entities.py       # EntityManager (CRUD, search)
│   └── relationships.py  # RelationshipManager
├── retrieval/
│   ├── native.py         # Surreal-native context-pack retrieval
│   ├── hybrid.py         # Compatibility hybrid search orchestration
│   └── fusion.py         # Score fusion (RRF)
├── tools/
│   ├── search.py         # Semantic search
│   ├── explore.py        # Graph navigation
│   ├── add.py            # Entity creation
│   └── manage.py         # Task workflow, admin
└── tasks/
    ├── workflow.py       # State machine, transitions
    └── manager.py        # Task operations

Usage

Models

from sibyl_core.models import (
    Entity, EntityType, Task, TaskStatus, Project, Epic,
)

task = Task(
    name="Implement OAuth",
    content="Add Google and GitHub OAuth",
    project_id="proj_abc",
    status=TaskStatus.TODO,
)

Graph Client

from sibyl_core.graph import GraphClient, EntityManager

client = GraphClient()
await client.connect()
manager = EntityManager(client, group_id=str(org_id))

# CRUD
await manager.create(entity)
# Retrieval uses search or list_by_type rather than direct ID lookup
results = await manager.search("authentication patterns", limit=20)

Write Concurrency

Write concurrency is handled by the active graph driver. The SurrealDB driver serializes WebSocket operations per client, and org-scoped graph access should use cloned drivers.

# Direct writes go through the active graph backend
await client.execute_write_org(query, org_id, **params)

# Or use EntityManager
await manager.create(entity)

Task Workflow

from sibyl_core.tasks import TaskManager

manager = TaskManager(entity_manager, relationship_manager)
await manager.create_task_with_knowledge_links(task)
await manager.find_similar_tasks(task)
await manager.estimate_task_effort(task)

Entity Types

Type	Description
pattern	Reusable coding patterns
episode	Temporal learnings
task	Work items with workflow
project	Container for tasks
epic	Feature-level grouping
source	Documentation sources
document	Crawled content

Relationship Types

from sibyl_core.models import RelationshipType

# Knowledge
RelationshipType.APPLIES_TO, REQUIRES, CONFLICTS_WITH, SUPERSEDES

# Task
RelationshipType.BELONGS_TO, DEPENDS_ON, BLOCKS, REFERENCES

Configuration

SIBYL_EMBEDDING_PROVIDER=openai     # openai | gemini
SIBYL_OPENAI_API_KEY=sk-...         # Required when provider=openai
SIBYL_GEMINI_API_KEY=...            # Required when provider=gemini
SIBYL_ANTHROPIC_API_KEY=...         # Entity extraction

SIBYL_EMBEDDING_MODEL=text-embedding-3-small
SIBYL_EMBEDDING_DIMENSIONS=1536
SIBYL_GRAPH_EMBEDDING_PROVIDER=openai
SIBYL_GRAPH_EMBEDDING_MODEL=text-embedding-3-small
SIBYL_GRAPH_EMBEDDING_DIMENSIONS=1024

Gemini embeddings default to gemini-embedding-2; Gemini keys can also come from GEMINI_API_KEY or GOOGLE_API_KEY. Changing embedding provider, model, or dimensions requires re-embedding existing graph and document vectors before comparing old and new search results.

Key Patterns

Multi-tenancy: Every operation requires org context

manager = EntityManager(client, group_id=str(org.id))

Node shapes: Native retrieval queries direct Surreal records and projectable legacy Episodic/Entity records

WHERE (n:Episodic OR n:Entity) AND n.entity_type = $type

Creation paths: direct native writes first, compatibility extraction when explicitly needed

await manager.create_direct(entity)  # Native write path, no LLM
await manager.create(entity)         # Compatibility extraction path

Testing

# With mock LLM (fast, deterministic)
SIBYL_MOCK_LLM=true uv run pytest tests/

# Live model tests (costs money)
uv run pytest tests/live --live-models

# Retrieval benchmark suite
moon run core:bench-retrieval

# Live read-only search benchmark against a running stack
moon run core:bench-live

# Live context-pack smoke benchmark
moon run core:bench-context

# Save labeled artifacts for store-to-store comparison
moon run core:bench-live -- --label surreal --metadata store=surreal

core:bench-live probes the real /api/search path with CLI auth. core:bench-context probes /api/context/pack; pass a JSON case file to turn smoke checks into dogfood acceptance fixtures for coding handoffs, Haven recall, or other memory spaces. Both benchmarks are read-only. Saved reports can be compared with uv run python benchmarks/compare_eval_reports.py <baseline.json> <candidate.json>.