quica 0.1.1

Quick Inter Coder Agreement in Python

Quick Inter Coder Agreement in Python

Quica (Quick Inter Coder Agreement in Python) is a tool to run inter coder agreement pipelines in an easy and effective ways. Multiple measures are run and results are collected in a single table than can be easily exported in Latex. quica supports binary or multiple coders.

Quick Inter Coder Agreement in Python

• Free software: MIT license

• Documentation: https://quica.readthedocs.io.

Installation

pip install -U quica

Get Quick Agreement

If you already have a python dataframe you can run Quica with few liens of code!

coder_1 = [0, 1, 0, 1, 0, 1]
coder_3 = [0, 1, 0, 1, 0, 0]

dataframe = pd.DataFrame({"coder1" : coder_1,
              "coder3" : coder_3})

quica = Quica(dataframe=dataframe)
print(quica.get_results())

from quica.measures.irr import *
from quica.dataset.dataset import IRRDataset
from quica.quica import Quica

coder_1 = [0, 1, 0, 1, 0, 1]
coder_3 = [0, 1, 0, 1, 0, 0]
disagreeing_coders = [coder_1, coder_3]
disagreeing_dataset = IRRDataset(disagreeing_coders)

quica = Quica(disagreeing_dataset)
print(quica.get_results())
print(quica.get_latex())

you should get these in output:

Out[1]:
                score
names
krippendorff  0.685714
fleiss        0.666667
scotts        0.657143
cohen         0.666667

Out[2]:

\begin{tabular}{lr}
\toprule
{} &     score \\
names       &           \\
\midrule
krippendorf &  0.685714 \\
fleiss      &  0.666667 \\
scotts      &  0.657143 \\
cohen       &  0.666667 \\
\bottomrule
\end{tabular}

Features

from quica.measures.irr import *
from quica.dataset.dataset import IRRDataset
from quica.quica import Quica

coder_1 = [0, 1, 0, 1, 0, 1]
coder_2 = [0, 1, 0, 1, 0, 1]
coder_3 = [0, 1, 0, 1, 0, 0]

agreeing_coders = [coder_1, coder_2]
agreeing_dataset = IRRDataset(agreeing_coders)

disagreeing_coders = [coder_1, coder_3]
disagreeing_dataset = IRRDataset(disagreeing_coders)

kri = Krippendorff()
cohen = CohensK()

assert kri.compute_irr(agreeing_dataset) == 1
assert cohen.compute_irr(agreeing_dataset) == 1
assert cohen.compute_irr(disagreeing_dataset) < 1
assert cohen.compute_irr(disagreeing_dataset) < 1

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

History

0.1.0 (2020-11-08)

• New API to get the output

• Fixed test cases

• Extended documentation on the README file

0.1.0 (2020-11-05)

• First release on PyPI.