substrate-haystack

SUBSTRATE memory components for Haystack pipelines. Retriever and writer that connect Haystack to SUBSTRATE's causal memory, emotional state, and identity continuity.

What SUBSTRATE adds to Haystack

• Causal memory retrieval -- episodes linked by cause-effect rules, not just vector similarity
• Emotional context output -- retriever exposes the entity's emotional state alongside documents
• Persistent storage -- writer stores documents through the entity's full cognitive pipeline
• Hybrid search -- semantic + keyword retrieval across the entity's full knowledge store
• Identity continuity -- cryptographically verified entity state persists across sessions

Installation

pip install substrate-haystack

Quick start: Retrieval pipeline

import os
from haystack import Pipeline
from haystack.components.generators import OpenAIGenerator
from haystack.components.builders import PromptBuilder
from substrate_haystack import SubstrateMemoryRetriever

retriever = SubstrateMemoryRetriever(
    api_key=os.environ["SUBSTRATE_API_KEY"],
)

prompt_builder = PromptBuilder(
    template="""
    Context from SUBSTRATE memory:
    {% for doc in documents %}
    - {{ doc.content }}
    {% endfor %}

    Entity emotional state: {{ entity_state }}

    Question: {{ query }}
    Answer:
    """
)

generator = OpenAIGenerator(model="gpt-4o")

pipe = Pipeline()
pipe.add_component("retriever", retriever)
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", generator)

pipe.connect("retriever.documents", "prompt_builder.documents")
pipe.connect("retriever.entity_state", "prompt_builder.entity_state")
pipe.connect("prompt_builder", "llm")

result = pipe.run({
    "retriever": {"query": "What patterns have we identified?"},
    "prompt_builder": {"query": "What patterns have we identified?"},
})

print(result["llm"]["replies"][0])

Quick start: Writing to memory

from haystack import Document, Pipeline
from substrate_haystack import SubstrateMemoryWriter

writer = SubstrateMemoryWriter(
    api_key=os.environ["SUBSTRATE_API_KEY"],
)

pipe = Pipeline()
pipe.add_component("writer", writer)

result = pipe.run({
    "writer": {
        "documents": [
            Document(content="The team decided to use event sourcing for the audit trail."),
            Document(content="Performance testing showed 99th percentile at 45ms."),
        ]
    }
})

print(f"Stored {result['writer']['documents_written']} documents")

Components

SubstrateMemoryRetriever

Retrieves documents from SUBSTRATE memory using hybrid search.

Inputs:

Outputs:

Parameters:

SubstrateMemoryWriter

Stores documents into SUBSTRATE memory via the respond tool.

Inputs:

Outputs:

API key

Get your API key at garmolabs.com. The free tier includes memory_search and get_emotion_state. Upgrade to Pro for hybrid_search and get_trust_state.

License

MIT -- see LICENSE for details.

Built by Garmo Labs.