Calculate Krippendorff's Alpha on any DataFrame
Project description
SimpleDorff - Calculate Krippendorff's Alpha on a DataFrame
Krippendorff's Alpha is a commonly used inter-annotator reliability metric, but it's hard to calculate on a Dataframe. This package makes it easy.
Made with ❤️ by LightTag - The Text Annotation Tool For Teams. We use this in production to give our customers a single number to understand the quality of their labeled data. Read the blog post here
Problem It Solves
Calculating Krippendorff's Alpha assumes data is formatted in a way that just doesn't appear in the wild. We wanted a package that could read a Dataframe in the formats we see in real life and give us the Alpha in one line.
Installing
pip install simpledorff
Usage
import simpledorff
import pandas as pd
Data = pd.read_csv('./examples/from_paper.csv') #Load Your Dataframe
Data.head()
Unnamed: 0 | document_id | annotator_id | annotation | |
---|---|---|---|---|
0 | 0 | 1 | A | 1.0 |
1 | 1 | 1 | B | 1.0 |
2 | 2 | 1 | D | 1.0 |
3 | 3 | 1 | C | NaN |
4 | 4 | 2 | A | 2.0 |
simpledorff.calculate_krippendorffs_alpha_for_df(Data,experiment_col='document_id',
annotator_col='annotator_id',
class_col='annotation')
0.743421052631579
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
simpledorff-0.0.1.tar.gz
(2.0 kB
view hashes)
Built Distribution
Close
Hashes for simpledorff-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a558de30116d5e5d2650965a23e2faf1677cc781e3104e5cfc5041d3b9f6872e |
|
MD5 | 44f87957ccc74e9ace0f5bbc38e45c06 |
|
BLAKE2b-256 | 3088bc535aa3e52c97b08bad6d95d4e54e92fef2befb44ce4624909a5958d6e5 |