Skip to main content

Language detection using FastText and Spacy

Project description

spacy_fastlang

Install

Assuming you have a working python environment, you can simply install it using

pip install spacy_fastlang

Usage

The library exports a pipeline component called language_detector that will set two spacy extensions

  • doc._.language = ISO code of the detected language or xx as a fallback
  • doc._.language_score = confidence
import spacy_fastlang  # noqa: F401 # pylint: disable=unused-import
nlp = spacy.load("...")
nlp.add_pipe("language_detector")
doc = nlp(en_text)

doc._.language == "..."
doc._.language_score >= ...

Options

Check the tests to see more examples and available options

License

Everythin is under MIT except the default model which is distributed under Creative Commons Attribution-Share-Alike License 3.0 by facebook here

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

spacy_fastlang-2.1.0.tar.gz (782.4 kB view details)

Uploaded Source

Built Distribution

spacy_fastlang-2.1.0-py3-none-any.whl (780.4 kB view details)

Uploaded Python 3

File details

Details for the file spacy_fastlang-2.1.0.tar.gz.

File metadata

  • Download URL: spacy_fastlang-2.1.0.tar.gz
  • Upload date:
  • Size: 782.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.7.1 CPython/3.8.18 Linux/6.2.0-1018-azure

File hashes

Hashes for spacy_fastlang-2.1.0.tar.gz
Algorithm Hash digest
SHA256 e10e7056275926211e5453e2e4f7047fec83e1c3c84f50f0a1b9229d33b83801
MD5 d1449e4d2b53ba62249ea682f15667a5
BLAKE2b-256 d5b9845453dbaf261dd4623b30a79038718c554af719ee1afd639675bfbcee18

See more details on using hashes here.

File details

Details for the file spacy_fastlang-2.1.0-py3-none-any.whl.

File metadata

  • Download URL: spacy_fastlang-2.1.0-py3-none-any.whl
  • Upload date:
  • Size: 780.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.7.1 CPython/3.8.18 Linux/6.2.0-1018-azure

File hashes

Hashes for spacy_fastlang-2.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 401a5838f7f483ba8efd93ab420317bbc16223fe07a28a9d89747e8bacdbb2d2
MD5 cfedcdf5a76eaec549c8d623b00bdc02
BLAKE2b-256 63f5ae0e5d30979c4090f12178f31c8fb08f7d43458f48f55233e3cb2b0709a9

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page