spaCy extension for kenLM.
Project description
## spacy_kenlm: KenLM extension for spaCy 2.0
This package adds [kenLM](https://github.com/kpu/kenlm) support
as a [spaCy 2.0 extension](https://spacy.io/usage/processing-pipelines#extensions).
## Usage
Train a `kenLM` language model first (or use the test model from `test.arpa`).
Add the spaCyKenLM to the spaCy pipeline to return scores.
```
import spacy
from spacy_kenlm import spaCyKenLM
nlp = spacy.load('en_core_web_sm')
spacy_kenlm = spaCyKenLM() # default model from test.arpa
nlp.add_pipe(spacy_kenlm)
doc = nlp('How are you?')
# doc score
doc._.kenlm_score
# span score
doc[:2]._.kenlm_score
# token score
doc[2]._.kenlm_score
```
## Installation
Install from the pip package.
```
pip install spacy_kenlm
```
This package adds [kenLM](https://github.com/kpu/kenlm) support
as a [spaCy 2.0 extension](https://spacy.io/usage/processing-pipelines#extensions).
## Usage
Train a `kenLM` language model first (or use the test model from `test.arpa`).
Add the spaCyKenLM to the spaCy pipeline to return scores.
```
import spacy
from spacy_kenlm import spaCyKenLM
nlp = spacy.load('en_core_web_sm')
spacy_kenlm = spaCyKenLM() # default model from test.arpa
nlp.add_pipe(spacy_kenlm)
doc = nlp('How are you?')
# doc score
doc._.kenlm_score
# span score
doc[:2]._.kenlm_score
# token score
doc[2]._.kenlm_score
```
## Installation
Install from the pip package.
```
pip install spacy_kenlm
```
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
spacy_kenlm-0.1.1.tar.gz
(3.2 kB
view hashes)