# Saniti
Sanitise lists of text documents quickly, easily and whilst maintaining your sanity
The aim was to streamline processing lists of documents into the same outputs into simply specifying the list of texts and defining the sanitization pipeline.
### Usage:
As a function-ish
>original_text = [“I like to moves it, move its”, “I likeing to move it!”, “the of”]
>text = saniti(original_text, [“token”, “destop”, “depunct”, “unempty”, “stem”, “out_corp_dict”]) #sanitise the text while initalising the class
>print(text.text)
As a class
>sani1 = saniti() # initialise the santising class
>text = sani1.process(original_text, [“token”, “destop”, “depunct”, “unempty”, “lemma”, “out_tag_doc”]) # sanitise the text
>print(text)
## Pipeline Components
“token” - tokenise texts
“depunct” - remove punctuation
“unempty” - remove empty words within documents
“lemma” - lemmatize text
“destop” - remove stopwords
“stem” - stem texts
“out_tag_doc” - turns the texts into gensim tagged documents for Doc2Vec
“out_corp_dict” - turns the texts into gensim corpus and dictionary
1 maintainer
Metadata
1 maintainer
Release files for saniti 0.1.21
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| saniti-0.1.21.tar.gz | 2.7 kB | Details |
Release files / saniti-0.1.21.tar.gz
| Download URL | saniti-0.1.21.tar.gz |
|---|---|
| Size | 2.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d20090af50cd0586a9259967cfc7a762c6c811efc7da36a9aa51f58f17294d58
|
|
BLAKE2b-256 checksum How to use checksums |
f5cd3bc3dd56d0c7df695d421545ee14dc6a889c291f1d8a0f32c3934b0e356e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
