porter2 0.1.1

A python wrapper around surgebase's Go implimentation of the Porter2 stemmer

porter2

A python wrapper around surgebase's porter2 implimentation.

The surgebase library impliments a finate state machine and as such has very good performance. For more information see Credits

Usage

from porter2 import stem

print(stem("running"))
# "run"

What does the python library do?

In order to make this wrapper library I took the porter2.go file from Surgebase, made a few small modifications including making the stem function accept and return C strings and renaming the package to main. I then compiled that file to C, and provided a small wrapper to call the c version.

Becuase the C version of porter2.go is 2.4 MB, I went ahead and compressed it using Python's built in bz2 and added the zipper module to unzip it on first use. The bz2 version is about 0.8 MB. As a result of the decompression step, the very first time porter2 is imported it will take a fraction of a second longer than future imports.

Alternatives for Porter2 in Python

• pystemmer
• A Python version
• Another Python version from Whoosh

Credits and Info

I'm not clear on the relationship between surgebase and Jian Zhen but zhenjl appears to deserve authorship credit and surgebase is where I found the go implimentation.

• Martin Porter
• SurgeBase for go source
• Jian Zhen

Additional Info

• Jian Zhen's blog post
• Porter 2 Wikipedia Entry
• History of the Porter Stemmer