textcleaner v0.4.26
Text-Cleaner is a utility library for text-data pre-processing. Use it before passing the text data to a model.
website - for more
Features!
main_cleaner to do all the below in one call ! or
remove unnecessary blank lines
stip out a perticular character or default one
transfer all characters to lowercase if needed
remove numbers, symblos and stop-words from the whole text
tokenize the text-data on one call
stemming & lemmatization powered by NLTK
The goal is to make basic cleaning of data hassle free. Most of the developers who are working with text data have faced this situation where data is not consumable and they end up wasting their time on these issues rather than fine tunning the model and get better accuracy. In that scenario this library can be useful and save you a tone of time.
Tech
textcleaner uses a number of open source projects to work properly:
And of course textcleaner itself is open source with a public repository on GitHub.
Installation
textcleaner requires Python 3.x to run.
Install the dependencies if you have not already installed it!
NLTK : steps to install [documentation]
REGEX :
pip install regextextcleaner :
pip install textcleaneror
pip install textcleaner==0.4.26Usage
import textcleaner as tc
tc.main_cleaner('<FILE_NAME>')
#or
tc.document('<FILE_NAME>')Above command will convert the text file into list of words with cleaning. Default response of the function is list of list use op argument and set it to ‘words’ and you will get a flat list of words.
Todos
more advanced features
ability to read more formats rather than only .txt
License
MIT
Free Software, Hell Yeah!
1 maintainer
Metadata
1 maintainer
Release files for textcleaner 0.4.26
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| textcleaner-0.4.26.tar.gz | 4.9 kB | Details |
Release files / textcleaner-0.4.26.tar.gz
| Download URL | textcleaner-0.4.26.tar.gz |
|---|---|
| Size | 4.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f4016192ce70b024c509fb48a4dc152330be40024cd4b81f23ffeaadef15257a
|
|
BLAKE2b-256 checksum How to use checksums |
13ea1f90a073066e56e13022460832ae3d3484de5bfad41ca5f94f5e0ed3db2b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.12.1 pkginfo/1.4.2 requests/2.18.4 setuptools/38.5.1 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.3
|
