Quick Inter Coder Agreement in Python
Project description
Quick Inter Coder Agreement in Python
Quica (Quick Inter Coder Agreement in Python) is a tool to run inter coder agreement pipelines in an easy and effective ways. Multiple measures are run and results are collected in a single table than can be easily exported in Latex. quica supports binary or multiple coders.
Quick Inter Coder Agreement in Python
Free software: MIT license
Documentation: https://quica.readthedocs.io.
Installation
pip install -U quica
Get Quick Agreement
If you already have a python dataframe you can run Quica with few liens of code!
coder_1 = [0, 1, 0, 1, 0, 1]
coder_3 = [0, 1, 0, 1, 0, 0]
dataframe = pd.DataFrame({"coder1" : coder_1,
"coder3" : coder_3})
quica = Quica(dataframe=dataframe)
print(quica.get_results())
from quica.measures.irr import *
from quica.dataset.dataset import IRRDataset
from quica.quica import Quica
coder_1 = [0, 1, 0, 1, 0, 1]
coder_3 = [0, 1, 0, 1, 0, 0]
disagreeing_coders = [coder_1, coder_3]
disagreeing_dataset = IRRDataset(disagreeing_coders)
quica = Quica(disagreeing_dataset)
print(quica.get_results())
print(quica.get_latex())
you should get these in output:
Out[1]:
score
names
krippendorff 0.685714
fleiss 0.666667
scotts 0.657143
cohen 0.666667
Out[2]:
\begin{tabular}{lr}
\toprule
{} & score \\
names & \\
\midrule
krippendorf & 0.685714 \\
fleiss & 0.666667 \\
scotts & 0.657143 \\
cohen & 0.666667 \\
\bottomrule
\end{tabular}
Features
from quica.measures.irr import *
from quica.dataset.dataset import IRRDataset
from quica.quica import Quica
coder_1 = [0, 1, 0, 1, 0, 1]
coder_2 = [0, 1, 0, 1, 0, 1]
coder_3 = [0, 1, 0, 1, 0, 0]
agreeing_coders = [coder_1, coder_2]
agreeing_dataset = IRRDataset(agreeing_coders)
disagreeing_coders = [coder_1, coder_3]
disagreeing_dataset = IRRDataset(disagreeing_coders)
kri = Krippendorff()
cohen = CohensK()
assert kri.compute_irr(agreeing_dataset) == 1
assert cohen.compute_irr(agreeing_dataset) == 1
assert cohen.compute_irr(disagreeing_dataset) < 1
assert cohen.compute_irr(disagreeing_dataset) < 1
Credits
This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.
History
0.1.0 (2020-11-08)
New API to get the output
Fixed test cases
Extended documentation on the README file
0.1.0 (2020-11-05)
First release on PyPI.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for quica-0.1.1-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f79eb9a153b8b4a6687ed6e167645c33c8b61cb1c0cd6f5e26c7d2ff09a79b5c |
|
MD5 | 765e83f47d7f4dc86b3db25210b69e6a |
|
BLAKE2b-256 | d81da14f9d780f4bd850093b96fd626cd86de197b0c753c36a113bdf4314ad28 |