Skip to main content

Simple automated text preprocessor

Project description

## Natural Language Text Preprocessor (nltp)

A simplified package for automating text preprocessing activities such as lemmatization, tokenization, removal of stop words, removal of certain pattern from text using regular expression. Working under the hood, this package makes use of the NLTK library for its text cleaning activities.

Installation

Requirements:

  • Python 3.7 or higher
  • NLTK

Install latest release:

pip install nltp

Install from source:

git clone https://github.com/izzyx6/nltp.git
cd nltp
pip install .

Usage: the basics

Here's how to perform text cleaning with nltp

First, we pass text in a list to the instantiated Preprocessor object as it takes an argument text.

This lines of code returns a tokenized version of the text passed on instantiating the text Preprocessor .

from nltp import Preprocessor

text = ["I like eat delicious food", "That's I'm cooking food myself, case '10 Best Foods' helps lot, also 'Best Before (Shelf Life)'"]

output = Preprocessor(text)
output.token()

You can retrive the text with their index (default set to 0):

output.token(1)

Next, you can get the cleaned version of the text passed in a list with lemmatization, stop word and unwanted patterns in text removed.

Available parameters to modify are stop_words and patterns.

output = Preprocessor(text,stop_words = [USER DEFINED], pattern = [USER DEFINED])

Note: the purpose of having these parameters are to by pass the defualt parameters that remove non alphabets, repeted sequence of words, and users name (identified with the @User).

output = Preprocessor(text)
output.text_cleaner()

Note: Using the output. you can get the default stop word, patterns, and text passed

output = Preprocessor(text)
output.patterns
output.stop_words
output.text

Citation

BibTex entry:

@misc{omalley2019kerastuner,
	title        = {Natural Language Text Preprocessor {nltp}},
	author       = { Ufumaka Isreal},
	year         = 2020,
	howpublished = {\url{https://github.com/izzyx6/nltp}}
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for nltp, version 0.1.0
Filename, size File type Python version Upload date Hashes
Filename, size nltp-0.1.0.tar.gz (3.7 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page