Disagreement based metrics for the processing and evaluation of crowdsourced annotations
Project description
CrowdTruth is an approach to machine-human computing for collecting annotation data on text, images and videos. The approach is focussed specifically on collecting gold standard data for training and evaluation of cognitive computing systems. The original framework was inspired by the IBM Watson project for providing improved (multi-perspective) gold standard (medical) text annotation data for the training and evaluation of various IBM Watson components, such as Medical Relation Extraction, Medical Factor Extraction and Question-Answer passage alignment.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for crowdtruth-2.0a1-py2-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | f7c60162e15cef3fe36d73c0683482fe96ed8d0db6ee5783ad0b3e06c5e542c7 |
|
| MD5 | 620a207f9d62b861007644868b66da3e |
|
| BLAKE2b-256 | 307c8518216992bbb2be959f68fe979ef304c788cfe38b6c3101dfa11c80ea5c |
