Preprocess German texts for serious NLP.

Project description

German Preprocessing

Preprocess German texts to do some serious natural-language processing.

clean texts
remove stopwords (as defined by spaCy)
lemmatize
lower-case, and remove all punctions, digits are replaced with "0"

Installation

pip install german

Usage

from german import preprocess

preprocess(['Johannes war einer von vielen guten Schülern.', 'Julia trinkt gern Tee.'], remove_stop=True)
# ['johannes gut schüler', 'julia trinken tee']

License

MIT.

Sponsoring

This work was created as part of a project that was funded by the German Federal Ministry of Education and Research.

Project details

Release history Release notifications | RSS feed

This version

0.1.0

Nov 16, 2019

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

german-0.1.0.tar.gz (2.1 kB view hashes)

Uploaded Nov 16, 2019 source

Built Distributions

german-0.1.0-py3-none-any.whl (3.3 kB view hashes)

Uploaded Nov 16, 2019 py3

german-0.1.0-py2-none-any.whl (3.3 kB view hashes)

Uploaded Nov 16, 2019 py2

Hashes for german-0.1.0.tar.gz

Hashes for german-0.1.0.tar.gz
Algorithm	Hash digest
SHA256	`a078711a05b8207e22e3f8c1f58eefa0c155a07df1275a5b1c3e38029efaf14c`
MD5	`5fc34c140288de65ffd422e27956cdaa`
BLAKE2b-256	`3c01a7837bdbb47b59101d5468d9b1a6c6143a378df03b03144e963041c669fd`

Hashes for german-0.1.0-py3-none-any.whl

Hashes for german-0.1.0-py3-none-any.whl
Algorithm	Hash digest
SHA256	`547635261f1bf1a338052034f177c5d5f6dfffe9aad10ef152d60a4de8ff58ba`
MD5	`9a6ec5145e5f971a6ad9c0a2fedd6269`
BLAKE2b-256	`bedda5a6e235538d803fbe39468a897196b38dcaa8dc6aba8902e2f658ac2ddf`

Hashes for german-0.1.0-py2-none-any.whl

Hashes for german-0.1.0-py2-none-any.whl
Algorithm	Hash digest
SHA256	`028fb365a9e8b1c5f57f957f0fe1c39dc7df386081572f9fabe2a34c9929c823`
MD5	`f8f06cfffd112d121eb8c3296d1fd53f`
BLAKE2b-256	`904dafc3a979b4a395fc647ec98bda92a7f1843b35a3be9f94940c0b14aae37b`