synap-er

Overview

synap-er is a B2B entity resolution library that extends advanced-text-processing (ner_lib) with scoped registries, a multi-stage resolution pipeline, and in-memory indexing for sub-millisecond lookups. It is designed to be imported as a library into a host application — it is not a standalone microservice.

Key capabilities:

• Multi-level scoped registries (World → Client → Customer → User) with confidence-weighted inheritance
• 6-stage resolution pipeline (exact → fuzzy → semantic → contextual → LLM-optional → manual review)
• Write-through consistency: PostgreSQL as durable store, in-memory indices for runtime reads
• Corpus adapters for domain-specific data (UNSPSC, UMLS, schema.org, etc.)
• Optional vendor integration (Senzing, Zingg) for clients with existing MDM infrastructure
• No GPU required — all components run on CPU

Installation

From source (development)

git clone git@github.com:gauravmaximem/synap-er.git
cd synap-er
pip install -e ".[dev]"

As a dependency in another project

Option A: Git SSH (for development machines)

# In your host project's requirements.txt
synap-er @ git+ssh://git@github.com/gauravmaximem/synap-er.git@v0.1.0

Option B: Git HTTPS with token (for CI/CD and deployment)

# In your host project's requirements.txt
synap-er @ git+https://${GITHUB_TOKEN}@github.com/gauravmaximem/synap-er.git@v0.1.0

Option C: Vendored wheel (zero auth required)

# Download the wheel from a GitHub Release
gh release download v0.1.0 --repo gauravmaximem/synap-er --pattern "*.whl" --dir vendor/

# In your host project's requirements.txt
./vendor/synap_er-0.1.0-py3-none-any.whl

CI/CD Setup (GitHub Actions)

For Option B, add this step before pip install:

- name: Configure git for private repo access
  run: git config --global url."https://${GITHUB_TOKEN}@github.com/".insteadOf "https://github.com/"
  env:
    GITHUB_TOKEN: ${{ secrets.SYNAP_ER_TOKEN }}

For Option C, no auth is needed — the wheel is committed to your repo.

The registry is yours

synap-er resolves names against a registry of canonical entities. It does not own one. It opens no database connection, creates no tables and writes no rows. Your application decides where entries live, how they are written and how a change reaches every worker. The library says what it needs, in synap_er.registry:

	What it is
RegistryEntryLike	The attributes the resolver reads from an entry: canonical_id, canonical_name, entity_type, scope, scope_id, aliases, metadata_, embedding. Any object that has them will do, including your own ORM rows.
RegistryEntry	A plain dataclass with those attributes, for hosts that do not need a model of their own.
Registry	The lookups the resolver makes.
ReviewSink	Where the threshold path hands a mention it could not place. Optional.
EntityType, Scope, Source	The enums an entry is described with.

synap_er.memory.RegistryIndexManager is an in-memory registry that satisfies Registry. The usual arrangement is:

1. Keep entries in your own database, in whatever shape you like.
2. At start-up, load them and call index.build(entries).
3. On every write, update the index too: add_entry, update_entry, remove_entry. If you run several workers, tell the others.

An entry's embedding is a 384-float sentence embedding of its name (synap_er.memory.compute_embedding). Entries without one are found by name only, not by semantic search.

Configuration

Resolution settings come from environment variables (prefix SYNAP_ER_) or from code:

from synap_er.config import PipelineSettings, Settings

settings = Settings(pipeline=PipelineSettings(confidence_threshold=0.85))

Usage

from synap_er.config import Settings
from synap_er.memory import RegistryIndexManager
from synap_er.pipeline.adjudication import MentionInput
from synap_er.pipeline.resolver import PipelineResolver
from synap_er.registry import EntityType, RegistryEntry

# Start-up: your entries, from your storage, into the index.
index = RegistryIndexManager()
index.build([
    RegistryEntry("Acme Corporation", EntityType.ORG, "CLIENT", "client-123",
                  aliases=["Acme Corp", "ACME"]),
])
resolver = PipelineResolver(index, Settings())

# One call for every name in a document. Narrowest scope first.
scope_chain = [("CLIENT", "client-123"), ("WORLD", None)]
[result] = await resolver.resolve_many(
    [MentionInput("Acme Corp", EntityType.ORG)], scope_chain,
    context="Acme Corp is renewing the contract.",
)
result.decision        # MATCH, NEW or AMBIGUOUS
result.canonical_name  # "Acme Corporation"

What you do with the answer is yours as well: on NEW, write an entry to your storage and index.add_entry(...); on AMBIGUOUS, queue it for a person. Pass adjudicator= to have doubtful pairs judged; without one they are decided by score.

Using the original ner_lib (world-data canonicalization)

The full ner_lib library is bundled and works unchanged:

from ner_lib import canonicalize_entity, canonicalize_relationship

result = canonicalize_entity("Apple Inc", mode="progressive")
print(result["canonical_name"])  # "Apple Inc."

Development & Testing

pytest tests/synap_er/ -v

The tests need no database: the library has no storage.

Architecture

See ARCHITECTURE.md for detailed design decisions.

Key points:

• Library, not a service. synap-er is imported into your application process. No separate deployment.
• Your database, not ours. synap-er adds tables to the host application's PostgreSQL instance.
• Write-through consistency. PostgreSQL is the durable store. In-memory structures (hash maps, BK-trees, FAISS, PhraseMatcher) serve all runtime reads. Updates are synchronous — a write is visible immediately.
• No GPU required. All components (spaCy, sentence-transformers, FAISS, RapidFuzz) run on CPU at the projected registry sizes.

Upstream Sync

synap-er is a private extension of the public advanced-text-processing library. To pull in updates from the public repo:

git fetch upstream
git merge upstream/main

The ner_lib/ directory is upstream's code. The synap_er/ directory is the B2B extension. They never conflict because synap_er imports from ner_lib but never modifies it.

License

Proprietary. All rights reserved.