Download and pre-processing data for nlp tasks
Project description
Feature
- handle over 100 dataset
- generate statistic report about processed dataset
- support many pre-processing ways
- Provide a panel for entering your parameters at runtime
- easy to adapt your own dataset and pre-processing utility
Documentation
Learn more from the docs.
Quick Start
Installing via pip
pip install nlprep
get one of the dataset
nlprep --dataset clas_udicstm --outdir sentiment
You can also try nlprep in Google Colab:
Overview
$ nlprep
arguments:
--dataset which dataset to use
--outdir processed result output directory
optional arguments:
-h, --help show this help message and exit
--util data preprocessing utility, multiple utility are supported
--cachedir dir for caching raw dataset
--infile local dataset path
--report generate a html statistics report
Contributing
Thanks for your interest.There are many ways to contribute to this project. Get started here.
License ![PyPI - License](https://pypi-camo.freetls.fastly.net/c47e783a04aa22961a54559219a85ad487ab6ed1/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f6c6963656e73652f766f696466756c2f6e6c70726570)
Icons reference
Icons modify from Darius Dan from www.flaticon.com
Icons modify from Freepik from www.flaticon.com
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
nlprep-0.1.21.tar.gz
(19.0 kB
view hashes)
Built Distributions
nlprep-0.1.21-py3.7.egg
(65.5 kB
view hashes)
nlprep-0.1.21-py3-none-any.whl
(32.4 kB
view hashes)