Annotation Analysis
Project description
BRR
Annotation Analysis
Package to analyse inter-annotator agreement.
Simple package to compute Interannotation Agreement with a simple interface.
Getting Started
Install package:
pip3 install annotation_analysis
Documentation
function krippendorf_alpha(annotations:List[List],labels:Optional[List], ignore=Optional[List])
"""
Arguments:
- annotations:
- The list of annotations, assumed to be one "row" per annotator (i.e. annotations[0] is annotator #1).
- Number of Columns represents the number of datapoints.
- Important: at this stage only hashable values are allowed.
- E.g. with labels ["A","B"], and three datapoints and 2 annotators the following would be the valid structure:
- [["A","B","A"],["A","B","B"]],
- Annotator #1 (index=0) -> ["A","B","A"]
- Annotator #2 (index=1) -> ["A","B","B"]
- labels:
- Represents the optional list of valid labels.
- Important: at this stage only hashable values are allowed.
- ignore:
- Represents a list of optional labels that should be ignored
- I.e. If this is non-empty then for any datapoint, if any of the annotators has the ignored label, the data point with all annotators is ignored
Return:
- Krippendorff Alpha score for all annotators.
"""
function fleiss_kappa(annotations:List[List],labels:Optional[List], ignore=Optional[List])
"""
Arguments:
- annotations:
- The list of annotations, assumed to be one "row" per annotator (i.e. annotations[0] is annotator #1).
- Number of Columns represents the number of datapoints.
- Important: at this stage only hashable values are allowed.
- E.g. with labels ["A","B"], and three datapoints and 2 annotators the following would be the valid structure:
- [["A","B","A"],["A","B","B"]],
- Annotator #1 (index=0) -> ["A","B","A"]
- Annotator #2 (index=1) -> ["A","B","B"]
- labels:
- Represents the optional list of valid labels.
- Important: at this stage only hashable values are allowed.
- ignore:
- Represents a list of optional labels that should be ignored
- I.e. If this is non-empty then for any datapoint, if any of the annotators has the ignored label, the data point with all annotators is ignored
Return:
- Fleiss Kappa score for all annotators.
"""
(C) - Nikolai Rozanov 2022
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for annotation_analysis-0.0.0.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | c1ffe8892759790595e0a15ba12290dd60c15e21ccee0d96e2404c280b101cd6 |
|
MD5 | 5dd16d47e87d8d51b68b99544b8db158 |
|
BLAKE2b-256 | a64cf0a7d441a9d998fce5fc636caa8efecc7daa88b127d3e529a4f95f0883e6 |
Close
Hashes for annotation_analysis-0.0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 98b7f50d7c1d83a0f40078af13a189f5b94c74b68f3d75813cd55666c0f19082 |
|
MD5 | d7dd1fbd8c036a6d06ecc2c8b2e7f5c1 |
|
BLAKE2b-256 | fade511c17a64c2f06b120dbf3649593bef02ee3888744e681849f241963eaef |