doc2term
A fast NLP tokenizer that detects sentences, words, numbers, urls, hostnames, emails, filenames, and phone numbers. Tokenize integrates and standardize the documents, remove the punctuations and duplications.
Installation
pip install doc2term
Compilation
The installation requires to compile the original C code using gcc.
Usage
Example notebook: doc2term
Example
>>> import doc2term
>>> doc2term.doc2term_str("Actions speak louder than words. ... ")
"Actions speak louder than words ."
>>> doc2term.doc2term_str("You can't judge a book by its cover. ... from thoughtcatalog.com")
"You can't judge a book by its cover . from thoughtcatalog.com"
1 maintainer
Metadata
1 maintainer
Release files for doc2term 0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| doc2term-0.1.tar.gz | 8.3 kB | Details |
Release files / doc2term-0.1.tar.gz
| Download URL | doc2term-0.1.tar.gz |
|---|---|
| Size | 8.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
1b684765faecccd53c1c1509e09e476b4c5bbd2698520dbad2cac25bc540e654
|
|
BLAKE2b-256 checksum How to use checksums |
ab40088c628f964db9636e947ab8fe5686f0b47865ff44edfd5af81f9638e68d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.57.0 CPython/3.9.2
|
