Skip to main content

Lightning Fast Language Prediction powered by Fasttext.

Project description

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Description: whatlang
=========

Lightning Fast Language Prediction :rocket:

Dependencies
=============

The below dependencies must be installed.

```
$ pip install future==0.16.0
$ pip install Cython==0.27.2
$ pip install cysignals==1.6.5
$ pip install pyfasttext==0.4.3
```

Basic Usage
============

Predicting Language using ``whatlang``

```python
>>> from whatlang import WhatLang
>>> wl = WhatLang()
>>> wl.predict_lang("Mother")
'en'
>>> wl.predict_lang(u"தாய்")
'ta'
>>> wl.predict_lang(u"അമ്മ")
'ml'
```

Batch Prediction is also supported

```python
>>>wl.predict_lang(["English sentence",u"അമ്മ"])
['en','ml']
```


Supported Language
==================

Supports 176 languages . The ISO codes for the corresponding languages are as below.

```
af als am an ar arz as ast av az azb ba bar bcl be bg bh bn bo bpy br bs bxr ca cbk
ce ceb ckb co cs cv cy da de diq dsb dty dv el eml en eo es et eu fa fi fr frr fy ga
gd gl gn gom gu gv he hi hif hr hsb ht hu hy ia id ie ilo io is it ja jbo jv ka kk km
kn ko krc ku kv kw ky la lb lez li lmo lo lrc lt lv mai mg mhr min mk ml mn mr mrj ms
mt mwl my myv mzn nah nap nds ne new nl nn no oc or os pa pam pfl pl pms pnb ps pt qu
rm ro ru rue sa sah sc scn sco sd sh si sk sl so sq sr su sv sw ta te tg th tk tl tr
tt tyv ug uk ur uz vec vep vi vls vo wa war wuu xal xmf yi yo yue zh
```

Reference
==========

``whatlang`` is powered by ``FastText``

### Enriching Word Vectors with Subword Information

[1] P. Bojanowski\*, E. Grave\*, A. Joulin, T. Mikolov, [*Enriching Word Vectors with Subword Information*](https://arxiv.org/abs/1607.04606)

```
@article{bojanowski2016enriching,
title={Enriching Word Vectors with Subword Information},
author={Bojanowski, Piotr and Grave, Edouard and Joulin, Armand and Mikolov, Tomas},
journal={arXiv preprint arXiv:1607.04606},
year={2016}
}
```

### Bag of Tricks for Efficient Text Classification

[2] A. Joulin, E. Grave, P. Bojanowski, T. Mikolov, [*Bag of Tricks for Efficient Text Classification*](https://arxiv.org/abs/1607.01759)

```
@article{joulin2016bag,
title={Bag of Tricks for Efficient Text Classification},
author={Joulin, Armand and Grave, Edouard and Bojanowski, Piotr and Mikolov, Tomas},
journal={arXiv preprint arXiv:1607.01759},
year={2016}
}
```

### FastText.zip: Compressing text classification models

[3] A. Joulin, E. Grave, P. Bojanowski, M. Douze, H. Jégou, T. Mikolov, [*FastText.zip: Compressing text classification models*](https://arxiv.org/abs/1612.03651)

```
@article{joulin2016fasttext,
title={FastText.zip: Compressing text classification models},
author={Joulin, Armand and Grave, Edouard and Bojanowski, Piotr and Douze, Matthijs and J{\'e}gou, H{\'e}rve and Mikolov, Tomas},
journal={arXiv preprint arXiv:1612.03651},
year={2016}
}
```

Keywords: language detection library
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

whatlang-1.0.2.tar.gz (782.9 kB view details)

Uploaded Source

Built Distribution

whatlang-1.0.2-py2.py3-none-any.whl (783.6 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file whatlang-1.0.2.tar.gz.

File metadata

  • Download URL: whatlang-1.0.2.tar.gz
  • Upload date:
  • Size: 782.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for whatlang-1.0.2.tar.gz
Algorithm Hash digest
SHA256 3b2bcc8e3707f5ce05181eb70c808e6cf228c1f8c35034f351e36e7772471188
MD5 9d73f55a5e7cf9254b27fdebe6874f13
BLAKE2b-256 c7c46ccd045aa1556c67b2962132a637dd9af1e24c31b4e05f053d27271f778c

See more details on using hashes here.

File details

Details for the file whatlang-1.0.2-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for whatlang-1.0.2-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 435bc2637dfdc890b705fa6238a7c8d5c0ee3d0ffce05ac75caa7bb1ff3a4ee6
MD5 6c28e2ea6b658a138d83c2fb5232b59a
BLAKE2b-256 71535ff4573afe56e4bd6d5ec759a973f02f1f294b8037ccf2e3ffecc4030b16

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page