Skip to main content

OmniNode document ingestion and semantic retrieval

Project description

OmniMemory

Python 3.12+ ONEX 4.0 Linting: ruff Type checked: mypy Pre-commit

Memory persistence, recall, and semantic retrieval for the OmniNode platform. OmniMemory provides ONEX-compliant nodes and handlers for storing agent context, indexing embeddings, querying intent graphs, and managing the full memory lifecycle across distributed omni agents.

Four-Node Architecture

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│     EFFECT      │───▶│     COMPUTE     │───▶│     REDUCER     │───▶│  ORCHESTRATOR   │
│  (store/fetch)  │    │ (embed/analyze) │    │  (consolidate)  │    │  (coordinate)   │
└─────────────────┘    └─────────────────┘    └─────────────────┘    └─────────────────┘
  • EFFECT: Memory storage, retrieval, and intent query against external backends
  • COMPUTE: Semantic analysis, similarity scoring, embedding generation
  • REDUCER: Memory consolidation, statistics aggregation, lifecycle state management
  • ORCHESTRATOR: Agent coordination, multi-step memory lifecycle workflows

What This Repo Provides

  • Memory nodesmemory_storage_effect, memory_retrieval_effect, intent_storage_effect, intent_query_effect, intent_event_consumer_effect
  • Compute nodessemantic_analyzer_compute, similarity_compute
  • Reducer nodesmemory_consolidator_reducer, statistics_reducer
  • Orchestrator nodesmemory_lifecycle_orchestrator, agent_coordinator_orchestrator
  • Intent handlershandler_intent, handler_subscription with protocol-driven adapters
  • Protocol interfaces — embedding provider, intent graph adapter, secrets provider
  • Audit layer — I/O audit logging via audit/
  • Runtime plugin — registered as onex.domain_plugins entry point (PluginMemory)

Infrastructure Ownership

OmniMemory's docker-compose.yml owns the memory-layer data services. These are the services you need to run omnimemory locally:

Service Container Default Port Purpose
Qdrant omnimemory-qdrant 6333 (HTTP), 6334 (gRPC) Vector database for semantic memory
Memgraph omnimemory-memgraph 7687 (Bolt), 7444 (HTTP) Graph database for relationship/intent queries
Valkey omnimemory-valkey 6379 In-memory cache and session storage
Kreuzberg omnimemory-kreuzberg-parser 8090 Document text extraction service

Not owned here — these services are managed by other repositories:

Service Owner Repository Why
Kafka / Redpanda omnibase_infra Platform-wide event bus, shared by all services
PostgreSQL omnibase_infra Platform-wide relational database, shared by all services

If you need Kafka or Postgres, start the omnibase_infra stack first:

docker compose -f /path/to/omnibase_infra/docker/docker-compose.infra.yml up -d

Quick Start

Memory services only

To run just the omnimemory data services (Qdrant, Memgraph, Valkey, Kreuzberg):

git clone https://github.com/OmniNode-ai/omnimemory.git
cd omnimemory

# Start memory data services
docker compose up -d

# Verify all services are healthy
docker compose ps

Default service ports (all configurable via .env):

  • Qdrant REST: localhost:6333
  • Memgraph Bolt: localhost:7687
  • Valkey: localhost:6379
  • Kreuzberg parser: localhost:8090

Install and run tests

uv sync
uv run pytest tests/ -m unit

For configuration options see docs/environment_variables.md.

Minimal example using the intent handler:

import asyncio
from uuid import uuid4

from omnibase_core.container import ModelONEXContainer
from omnimemory.handlers.adapters.models import ModelIntentClassificationOutput
from omnimemory.handlers.handler_intent import HandlerIntent


async def main() -> None:
    container = ModelONEXContainer()
    handler = HandlerIntent(container)

    await handler.initialize(connection_uri="bolt://localhost:7687")

    # Store an intent
    result = await handler.store_intent(
        session_id="session_123",
        intent_data=ModelIntentClassificationOutput(
            intent_category="debugging",
            confidence=0.92,
            keywords=["error", "traceback"],
        ),
        correlation_id=str(uuid4()),
    )

    # Query session intents
    query_result = await handler.query_session(
        session_id="session_123",
        min_confidence=0.5,
    )

    await handler.shutdown()


asyncio.run(main())

Directory Structure

src/omnimemory/
├── audit/              # I/O audit logging
├── enums/              # Domain enumerations (memory types, operation types, lifecycle states)
├── errors/             # Structured error types
├── handlers/           # HandlerIntent, HandlerSubscription + adapters
├── models/             # Pydantic models (core, memory, intelligence, service, container, contracts)
├── nodes/              # EFFECT, COMPUTE, REDUCER, ORCHESTRATOR node implementations
├── protocols/          # Protocol interfaces (embedding, intent graph, secrets)
├── runtime/            # Plugin registration, wiring, dispatch, introspection
├── tools/              # Contract linter and stubs
└── utils/              # Shared utilities (audit logger, PII detection, retry, health)

Development

Uses uv for package management.

uv sync
uv run pytest tests/ -m unit
uv run mypy src/omnimemory/ --strict
uv run ruff check src/ tests/
uv run ruff format src/ tests/

Documentation

Reference: docs/

Open an issue or email contact@omninode.ai.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

omninode_memory-0.6.1.tar.gz (1.1 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

omninode_memory-0.6.1-py3-none-any.whl (646.3 kB view details)

Uploaded Python 3

File details

Details for the file omninode_memory-0.6.1.tar.gz.

File metadata

  • Download URL: omninode_memory-0.6.1.tar.gz
  • Upload date:
  • Size: 1.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.10.7 {"installer":{"name":"uv","version":"0.10.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for omninode_memory-0.6.1.tar.gz
Algorithm Hash digest
SHA256 1e4f1117a7486294eecfc27577176ef1f4e1b254d0f7cd1d29de87d3f72313de
MD5 4b2190a0605077e745c1522441f91554
BLAKE2b-256 fcf67ea2698f87888fd93098d9d05cd4606b2cd3f20c72d5cbfa86ddb2aae0bc

See more details on using hashes here.

File details

Details for the file omninode_memory-0.6.1-py3-none-any.whl.

File metadata

  • Download URL: omninode_memory-0.6.1-py3-none-any.whl
  • Upload date:
  • Size: 646.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.10.7 {"installer":{"name":"uv","version":"0.10.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for omninode_memory-0.6.1-py3-none-any.whl
Algorithm Hash digest
SHA256 dc132079aee824eda0e19eeedfc581b51465408c29eb07f3b236f8c1a08edcec
MD5 fba18da1e2cb1457206f08f5c1bdb55a
BLAKE2b-256 dcdf71f0a8c46268fb50b9c19fca0d7d29ee900c869bc4e66b91388c6d107ab9

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page