N-Seek 0.1.6

Explainable neuro-symbolic search for trustworthy document retrieval

NeuroSeek

Explainable neuro-symbolic search for Python and trustworthy RAG pipelines.

Package name: neuroseek
Project name: NeuroSeek
Install: pip install neuroseek
Import: from neuroseek import NeuroSymbolicIndex

NeuroSeek combines dense vectors and BM25 with lexical relevance, typed facts, symbolic rules, metadata constraints, hierarchy, source trust, evidence support, and contradiction penalties. It returns validated result models with separate relevance and confidence scores so a mathematically close chunk does not silently override the user's actual constraints.

• Hybrid dense and BM25 candidate generation with reciprocal-rank fusion.
• Deterministic local hash embeddings and exact NumPy search for tests and offline use.
• Optional Sentence Transformers, OpenAI embeddings, and FAISS indexes.
• Typed facts and constrained, explainable scoring rules.
• Metadata, relationship, regular-expression, and temporal operators.
• Indexed hard constraints, soft preferences, and query-specific symbolic goals.
• Optional CrossEncoder reranking after hard eligibility filtering.
• Contextual embeddings, versioned caches, batched indexing, and tombstone compaction.
• Independent confidence scoring and contradiction diagnostics.
• Complete score attribution and source provenance for every result.
• JSON snapshots and an optional SQLite record store.
• Python API, CLI, and FastAPI application factory.
• DocGun, LangChain, and LlamaIndex integrations.
• Dependency-light core with heavyweight backends kept optional.

---

Before / After

from neuroseek import NeuroSymbolicIndex

index = NeuroSymbolicIndex()
index.add(
    id="notice-period",
    text="Employees must provide four weeks written notice.",
    metadata={"document_type": "contract", "trust": 0.95},
)

result = index.search("How much notice must I give?")[0]

print(result.text)
print(result.score)
print(result.confidence)

Example output:

Employees must provide four weeks written notice.
0.864
0.835

With explain=True—the default—each result includes semantic, lexical, symbolic, hierarchy, trust, and evidence contributions alongside matched rules, rejected rules, provenance, and contradictions.

---

Why Neuro-Symbolic Search?

Dense search can return a semantically similar result that is obsolete, applies to the wrong user, comes from an untrusted source, or contradicts current policy. NeuroSeek makes those constraints part of retrieval rather than leaving them for the application to infer:

Query
  -> Normalize query and extract high-confidence constraints
  -> Generate dense, BM25, and metadata-filtered candidates
  -> Fuse rankings with reciprocal-rank fusion
  -> Enforce hard symbolic eligibility rules
  -> Optionally rerank a bounded window with a CrossEncoder
  -> Apply semantic-gated soft preferences
  -> Calibrate confidence, evidence, and contextual contradictions
  -> Return ranked results with a complete explanation

The default fused score is 50% semantic, 10% lexical, 15% symbolic, 8% hierarchy, 7% trust, and 10% evidence support, minus configurable contradiction and uncertainty penalties. Every weight can be replaced through NeuroSeekConfig.

Raw BM25 values are saturation-normalized before fusion, cosine scores are bounded, and symbolic contributions are semantic-gated. fit_fusion_weights() can learn non-negative normalized weights from labelled feature rows, while ConfidenceCalibrator provides dependency-free Platt scaling for held-out confidence labels.

Retrieval architecture

flowchart LR
    Q[Query] --> X[Constraint extraction]
    Q --> D[Dense retrieval]
    Q --> B[BM25 retrieval]
    X --> M[Metadata postings]
    D --> F[Reciprocal-rank fusion]
    B --> F
    M --> F
    F --> H[Hard symbolic rules]
    H --> R[Optional CrossEncoder]
    R --> S[Vectorized scoring and soft rules]
    S --> C[Confidence and contextual contradictions]
    C --> R[Ranked results]
    C --> X[Explanations and provenance]

Dense retrieval handles paraphrases while BM25 preserves exact terms, names, dates, and identifiers. Symbolic rules enforce eligibility after fused candidate generation; they refine relevant results and are gated so metadata cannot rescue an irrelevant match.

---

Install

pip install neuroseek

For accelerated FAISS indexes:

pip install "neuroseek[faiss]"

For neural embedding models or DocGun ingestion:

pip install "neuroseek[embeddings]"
pip install "neuroseek[docgun]"

For the complete optional toolset:

pip install "neuroseek[all]"

For development:

pip install -e ".[dev,all]"
pytest -q
python -m build

---

Quick Start

Build an Index

from neuroseek import NeuroSymbolicIndex

index = NeuroSymbolicIndex(
    embedding="hash",
    vector_backend="numpy-flat",
)

index.add(
    id="contract-12-termination",
    text="Employees must provide four weeks written notice.",
    metadata={
        "document_id": "contract-12",
        "document_type": "employment_contract",
        "section": "Termination",
        "is_current": True,
        "trust": 0.95,
    },
)

Add Symbolic Rules

from neuroseek import Condition, Rule

index.rules.add(Rule(
    name="prefer_current_policy",
    conditions=(
        Condition(field="metadata.document_type", operator="eq", value="employment_contract"),
        Condition(field="metadata.is_current", operator="eq", value=True),
    ),
    score_delta=0.15,
    reason="Current employment contracts are preferred.",
))

Required rules reject candidates that do not satisfy their conditions:

index.rules.add(Rule(
    name="current_only",
    conditions=(Condition(field="metadata.is_current", operator="eq", value=True),),
    required=True,
))

results = index.search(
    "What is my notice period?",
    goals=["current_only"],
    explain=True,
)

Add Facts and Relationships

index.facts.add(
    "contract-12",
    "applies_to",
    "employee-robert",
)

index.rules.add(Rule(
    name="user_relevant",
    conditions=(Condition(
        field="metadata.document_id",
        operator="related_to",
        value={"predicate": "applies_to", "object": "employee-robert"},
    ),),
    required=True,
))

Supported condition operators are eq, neq, gt, gte, lt, lte, in, contains, exists, before, after, matches, and related_to.

---

CLI

Create a local index:

neuroseek init .neuroseek

Index one or more text files:

neuroseek add .neuroseek ./documents --recursive --document-type policy

Search and print complete result models as JSON:

neuroseek search .neuroseek "What is the current policy?" --top-k 5 --explain

Add a fact:

neuroseek fact .neuroseek contract-12 applies_to employee-robert

Run a query benchmark from a JSON list:

neuroseek benchmark .neuroseek benchmark/queries.json

---

Backends

Capability	Core default	Optional backends
Embeddings	Deterministic signed feature hashing	Sentence Transformers, OpenAI
Vector index	NumPy exact cosine search	FAISS Flat, HNSW, IVF, IVF-PQ
Records	In-memory with JSON snapshots	SQLite record store
Rules	Constrained typed conditions	Relationship and temporal predicates
API	Python API and CLI	FastAPI application factory
Ecosystem	Native document records	DocGun, LangChain, LlamaIndex

Neural embeddings and FAISS

from neuroseek import NeuroSymbolicIndex

index = NeuroSymbolicIndex(
    embedding_backend="sentence-transformers",
    embedding_model="BAAI/bge-small-en-v1.5",
    vector_backend="faiss-hnsw",
)

FAISS vectors are L2-normalized and searched with inner product for cosine similarity. The core package never installs FAISS or Torch unless their extras are explicitly selected. Hash embeddings remain the deterministic development backend; BGE with FAISS HNSW is the recommended production profile. NeuroSeekConfig.production() supplies those defaults.

Retrieve and rerank

from neuroseek import NeuroSymbolicIndex
from neuroseek.retrieval.reranking import CrossEncoderReranker

index = NeuroSymbolicIndex(
    embedding_backend="sentence-transformers",
    embedding_model="BAAI/bge-small-en-v1.5",
    vector_backend="faiss-hnsw",
    reranker=CrossEncoderReranker(),
)

The CrossEncoder sees only candidates that satisfy hard filters and required rules. rerank_candidates bounds the expensive window and defaults to 30.

---

DocGun

DocGun handles quality-aware document ingestion; NeuroSeek preserves its document IDs, element types, hierarchy, page locations, extraction confidence, and parser provenance for retrieval:

from docgun import ingest
from neuroseek import NeuroSymbolicIndex
from neuroseek.integrations.docgun import index_document

state = ingest("policy.pdf")
index = NeuroSymbolicIndex()

index_document(index, state.document)
results = index.search("What is the current expenses policy?")

This keeps the responsibilities explicit:

DocGun      -> quality-aware document ingestion
NeuroSeek   -> constraint-aware, explainable retrieval
PaperTrail  -> application and interface

---

Persistence

Save records, facts, rules, configuration, and a versioned manifest:

index.save(".neuroseek")

Restore the index later:

from neuroseek import NeuroSymbolicIndex

index = NeuroSymbolicIndex.load(".neuroseek")

The generated manifest.json records the schema version, backend, embedding dimensions, embedding backend, and record count.

---

Architecture

src/neuroseek/
  __init__.py          # Small public API
  api.py               # NeuroSymbolicIndex facade
  config.py            # Validated configuration and score weights
  models/              # Records, facts, rules, results, and explanations
  embeddings/          # Hash, Sentence Transformers, and OpenAI backends
  indexes/             # NumPy and FAISS vector indexes
  symbolic/            # Fact store, rule store, and condition evaluation
  retrieval/           # Filtering and explainable score fusion
  inference/           # Lightweight evidence graph
  integrations/        # DocGun, FastAPI, LangChain, and LlamaIndex
  storage/             # SQLite record storage
  cli/                 # Command-line interface
tests/                 # Unit and integration tests
examples/              # Search, symbolic, and DocGun examples
CHANGELOG.md            # Release history
pyproject.toml          # Package metadata and dependency extras

---

FastAPI

The optional application factory exposes health and search routes around an existing index:

from neuroseek import NeuroSymbolicIndex
from neuroseek.integrations.fastapi import create_app

index = NeuroSymbolicIndex.load(".neuroseek")
app = create_app(index)

Run it with Uvicorn after installing neuroseek[api].

---

Development

pip install -e ".[dev,all]"
python -m pytest
ruff check .
python -m build

The test suite covers retrieval, symbolic constraints, explanations, persistence, batch indexing, deletion, and the duck-typed DocGun adapter.

Retrieval benchmark

The included synthetic policy benchmark compares dense-only retrieval with the same index using required rules and metadata constraints:

python -m benchmarks.retrieval_quality

It reports Recall@1/@3/@10, Precision@3, MRR, NDCG@10, constraint accuracy, forbidden-result rate, explanation fidelity, p50/p95 latency, indexing throughput, and peak memory per record. Separate suites cover semantic paraphrases, exact identifiers, constraint collisions, adversarial similarity, and keyword-heavy policies. Results compare dense-only, hybrid retrieval, and hybrid-plus-rules. Replace benchmarks/synthetic_policy.json or call run(path) to evaluate another corpus.

---

License

MIT

---

Contributing

Contributions are welcome. Please include a minimal corpus or generated fixture, the expected ranking and explanation, and a regression test for retrieval or reasoning changes.

---

Citation

@software{NeuroSeek2026,
  title={NeuroSeek: Explainable Neuro-Symbolic Search for Python},
  author={Arkay92},
  url={https://github.com/Arkay92/NeuroSeek},
  year={2026},
  version={0.1.6},
}

---

Acknowledgments

• NumPy for the dependency-light exact vector backend.
• FAISS for accelerated similarity search.
• Sentence Transformers for neural embeddings and reranking tools.
• Pydantic for validated public models and configuration.
• DocGun for quality-aware structured ingestion.