Language detection using FastText and Spacy
Project description
spacy_fastlang
Install
Assuming you have a working python environment, you can simply install it using
pip install spacy_fastlang
Usage
The library exports a pipeline component called language_detector
that will set two spacy extensions
- doc._.language = ISO code of the detected language or
xx
as a fallback - doc._.language_score = confidence
import spacy_fastlang # noqa: F401 # pylint: disable=unused-import
nlp = spacy.load("...")
nlp.add_pipe("language_detector")
doc = nlp(en_text)
doc._.language == "..."
doc._.language_score >= ...
Options
Check the tests to see more examples and available options
License
Everythin is under MIT
except the default model which is distributed under Creative Commons Attribution-Share-Alike License 3.0 by facebook here
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
spacy_fastlang-2.1.0.tar.gz
(782.4 kB
view hashes)
Built Distribution
spacy_fastlang-2.1.0-py3-none-any.whl
(780.4 kB
view hashes)
Close
Hashes for spacy_fastlang-2.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 401a5838f7f483ba8efd93ab420317bbc16223fe07a28a9d89747e8bacdbb2d2 |
|
MD5 | cfedcdf5a76eaec549c8d623b00bdc02 |
|
BLAKE2b-256 | 63f5ae0e5d30979c4090f12178f31c8fb08f7d43458f48f55233e3cb2b0709a9 |