Skip to main content

spaCy pipeline component for ANN Entity Linking using LanceDB

Project description

spacy-lancedb-linker

spaCy pipeline component for ANN Entity Linking using LanceDB

Installation

pip install spacy-lancedb-linker

Usage

import spacy

from spacy_lancedb_linker.kb import AnnKnowledgeBase
from spacy_lancedb_linker.linker import AnnLinker  # noqa
from spacy_lancedb_linker.types import Alias, Entity

nlp = spacy.load("en_core_web_md")
kb = AnnKnowledgeBase(uri="data/sample-lancedb")
kb.add_entities(
    [
        Entity(**entity)
        for entity in [
            {
                "entity_id": "a3",
                "name": "Natural language processing",
                "description": "Natural language processing (NLP) is a subfield of linguistics, computer science, information engineering, and artificial intelligence concerned with the interactions between computers and human (natural) languages, in particular how to program computers to process and analyze large amounts of natural language data.",
            },
            {
                "entity_id": "a4",
                "name": "Neuro-linguistic programming",
                "description": "Neuro-linguistic programming (NLP) is a pseudoscientific approach to communication, personal development, and psychotherapy created by Richard Bandler and John Grinder in California, United States in the 1970s.",
            },
        ]
    ]
)
kb.add_aliases(
    [
        Alias(**alias)
        for alias in [
            {"alias": "NLP", "entities": ["a3", "a4"], "probabilities": [0.5, 0.5]},
            {
                "alias": "Natural language processing",
                "entities": ["a3"],
                "probabilities": [1.0],
            },
            {
                "alias": "Neuro-linguistic programming",
                "entities": ["a4"],
                "probabilities": [1.0],
            },
        ]
    ]
)

ann_linker = nlp.add_pipe("ann_linker", last=True)
ann_linker.set_kb(kb)

doc = nlp("NLP is a subset of machine learn.")

print(doc.ents[0].kb_id_)
print(doc.ents[0]._.alias_candidates)
print(doc.ents[0]._.kb_candidates)

Test

poetry install
poetry shell
poetry run pytest

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

spacy_lancedb_linker-0.1.2.tar.gz (10.0 kB view details)

Uploaded Source

Built Distribution

spacy_lancedb_linker-0.1.2-py3-none-any.whl (10.7 kB view details)

Uploaded Python 3

File details

Details for the file spacy_lancedb_linker-0.1.2.tar.gz.

File metadata

  • Download URL: spacy_lancedb_linker-0.1.2.tar.gz
  • Upload date:
  • Size: 10.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.1 CPython/3.12.6

File hashes

Hashes for spacy_lancedb_linker-0.1.2.tar.gz
Algorithm Hash digest
SHA256 bd2a4a80d6e4a62102b61ea07252fcf9ac168075a6f8c14b18e11466b7518a77
MD5 6081fb8db4bc74d190d215d891357947
BLAKE2b-256 a4dca247c5a20b3a715124a4913527b1e5a4979c58a1b632b3cea25ae923d6d5

See more details on using hashes here.

File details

Details for the file spacy_lancedb_linker-0.1.2-py3-none-any.whl.

File metadata

File hashes

Hashes for spacy_lancedb_linker-0.1.2-py3-none-any.whl
Algorithm Hash digest
SHA256 dcc3dd3dc8fc3e60ad9e6a726f58083be09ec96bcaed4b5584b14fd9044de308
MD5 e20b57e054334a99513ecd2efbb70f5a
BLAKE2b-256 0ca295bb64536c19876ac99710cf911212fea4b629905f8918e3d29f3221b592

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page