A Framework for Cross-Dialectal NLP
Project description
Multi-VALUE: The VernAcular Language Understanding Evaluation benchmark
Setup
Prerequisites:
- Create a virtual environment
conda create --name value python=3.7.13
conda activate value
- Install requirements:
pip install -r requirements.txt
- Install spaCy English pipeline and nltk wordnet
bash downloads.sh
- Confirm that your setup is correct by running the unittest
python -m unittest tests.py
Build Multi-VALUE CoQA (optional)
- Pull data
bash pull_coqa.sh
- Run for each dialect
python -m src.build_coqa_value --dialect aave &
python -m src.build_coqa_value --dialect appalachian &
python -m src.build_coqa_value --dialect chicano &
python -m src.build_coqa_value --dialect indian &
python -m src.build_coqa_value --dialect multi &
python -m src.build_coqa_value --dialect singapore &
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
value_nlp-0.1.3.tar.gz
(4.0 MB
view hashes)
Built Distribution
Close
Hashes for value_nlp-0.1.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4a71ce8b5c67e44408d2cb9fc9d2523be213bcffa3e236bdebcdef7acec4b8bc |
|
MD5 | 848ad454f9634d4f4cdc6f32cd1fe363 |
|
BLAKE2b-256 | 0f1027011a183ef1195efce597e7f3c71cbe6b2ee08c0c8b946f3fbed459c1ae |