A simple tokenizer function for NLP
Project description
This tokenizer function turns text into lowercase word tokens, removes English stopwords, lemmatize the tokens and replaces URLs with a placeholder.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Close
Hashes for jf_tokenize_package-1.0.1.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | ab4ebeaed6f01c6c53b08254ede9d83dd17b2a7d71965f676dac1b6a60726bba |
|
| MD5 | 5d31d747c5efe432a7516c7238b55abb |
|
| BLAKE2b-256 | 4159efbcabf7726bcb38de47272ee0638efb3d7e367f821af7468250203d4e21 |
