Skip to main content

Python implementations of various stemming algorithms.

Project description

Python implementations of the Porter, Porter2, Paice-Husk, and Lovins stemming algorithms for English. These implementations are straightforward and efficient, unlike some Python versions of the same algorithms available on the Web. This package is an extraction of the stemming code included in the Whoosh search engine.

Note that these are pure Python implementations. Python wrappers for, e.g. the Snoball stemmers and the C implementation of the Porter stemmer are available on PyPI and will be faster if using compiled code is an option for you.

Stemming algorithms attempt to automatically remove suffixes (and in some cases prefixes) in order to find the “root word” or stem of a given word. This is useful in various natural language processing scenarios, such as search.

In general porter2 is the best overall stemming algorithm, but not necessarily the fastest or most aggressive.

The stemming package contains modules for each algorithm (lovins, paicehusk, porter, and porter2). Each module contains a stem() function:

>> from stemming.porter2 import stem
>> stem("factionally")
faction

(The Paice-Husk algorithm allows custom stemming rule sets, so the paicehusk module also includes a PaiceHuskStemmer class you can instantiate with custom rules.)

The source code for this package is available on BitBucket:

http://bitbucket.org/mchaput/stemming

Please use BitBucket to file bug reports or feature requests:

http://bitbucket.org/mchaput/stemming/issues/

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for stemming, version 1.0
Filename, size File type Python version Upload date Hashes
Filename, size stemming-1.0.zip (13.6 kB) File type Source Python version None Upload date Hashes View
Filename, size stemming-1.0.tar.gz (10.9 kB) File type Source Python version None Upload date Hashes View
Filename, size stemming-1.0-py2.5.egg (23.9 kB) File type Egg Python version 2.5 Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page