This release is a pre-release and may not be stable for production use.
Author: Rob Speer
## Installation
wordfreq requires Python 3 and depends on a few other Python modules (msgpack-python, langcodes, and ftfy). You can install it and its dependencies in the usual way, either by getting it from pip:
pip3 install wordfreq
or by getting the repository and running its setup.py:
python3 setup.py install
To handle word frequency lookups in Japanese, you need to additionally install mecab-python3, which itself depends on libmecab-dev. These commands will install them on Ubuntu:
sudo apt-get install mecab-ipadic-utf8 libmecab-dev pip3 install mecab-python3
## Unicode data
The tokenizers used to split non-Japanese phrases use regexes built using the unicodedata module from Python 3.4, which uses Unicode version 6.3.0. To update these regexes, run scripts/gen_regex.py.
## License
wordfreq is freely redistributable under the MIT license (see MIT-LICENSE.txt), and it includes data files that may be redistributed under a Creative Commons Attribution-ShareAlike 4.0 license (https://creativecommons.org/licenses/by-sa/4.0/).
wordfreq contains data extracted from Google Books Ngrams (http://books.google.com/ngrams) and Google Books Syntactic Ngrams (http://commondatastorage.googleapis.com/books/syntactic-ngrams/index.html). The terms of use of this data are:
Ngram Viewer graphs and data may be freely used for any purpose, although acknowledgement of Google Books Ngram Viewer as the source, and inclusion of a link to http://books.google.com/ngrams, would be appreciated.
It also contains data derived from the following Creative Commons-licensed sources:
The Leeds Internet Corpus, from the University of Leeds Centre for Translation Studies (http://corpus.leeds.ac.uk/list.html)
The OpenSubtitles Frequency Word Lists, by Invoke IT Limited (https://invokeit.wordpress.com/frequency-word-lists/)
Wikipedia, the free encyclopedia (http://www.wikipedia.org)
Some additional data was collected by a custom application that watches the streaming Twitter API, in accordance with Twitter’s Developer Agreement & Policy. This software only gives statistics about words that are very commonly used on Twitter; it does not display or republish any Twitter content.
Release files for wordfreq 1.0b4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| wordfreq-1.0b4.tar.gz | 5.7 MB | Without provenance | Details for wordfreq-1.0b4.tar.gz |
Release files / wordfreq-1.0b4.tar.gz
| Download URL | wordfreq-1.0b4.tar.gz |
|---|---|
| Size | 5.7 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e9709ee4fcb8bcdfdb158e0d8a14ce415bac64c105164b821201984d94a402e6
|
|
BLAKE2b-256 checksum How to use checksums |
f1447e4a9447917988e4955045b0275ab053acff2b74b722fe2f42a68aef273f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Security for wordfreq 1.0b4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
