Skip to main content


Project description


Implement 4 papers:

  • Water from Two Rocks: Maximizing the Mutual Information (MCG)
  • Dominantly Truthful Multi-task Peer Prediction with a Constant Number of Tasks (DMI)
  • A Bayesian truth serum for subjective data (BTS)
  • Informed Truthfulness in Multi-Task Peer Prediction (CA)


To begin

You can install the package femtools using pip

pip install femtools

To begin, import femtools

import numpy as np
import femtools as fem


For Bayesian Truth Serum, we implemented the version with finite players. Call the function BTS with answers x and predicted frequencies y, score for every respondent is returned. x and y can be given in the numpy.array form or list form. If there are n respondents and m possible answers, x should be an n-dimensional vector and each answer in x should be an integer in [0, m). Similarly, y is a n*m matrix denoting the predicted frequencies. BTS score is composed of information-score and prediction score, thus optional parameter alpha controlling the weight given to the prediction score could be assigned between (0,1]. By default, alpha is 1.

Here are examples

>>> fem.BTS([3, 2, 1, 1, 0],
... [[0.1, 0.1, 0.3, 0.5],
... [0.1, 0.2, 0.5, 0.2],
... [0.3, 0.4, 0.2, 0.1],
... [0.3, 0.4, 0.1, 0.2],
... [0.1, 0.3, 0.2, 0.4]])
array([-3.28030172, -2.40787449, -0.29706308, -0.29706308, -1.074341  ])

>>> fem.BTS([0, 0, 0],
... [[0.5, 0.5],
... [0.5, 0.5],
... [0.5, 0.5]], alpha = 0.5)
array([0.51873113, 0.51873113, 0.51873113])


For Correlated Agreement Mechanism, we implemented the detail-free version. CA Detail-Free is designed for multi-task problem with n agents and m tasks. Call the function CA with a n*m report matrix reports, score for every agent is returned. reports can be given in the numpy.array form or list form. For convenience, matrix reports can be given transposed with optional parameter agent_first = False. By default, agent_first is set to True. In addition, function CA does not expect that elements are integers.

Here is the example

>>> fem.CA([['subway', 'subway', 'subway', 'burgerK', 'burgerK', 'burgerK'],
... ['burgerK', 'McDonald', 'subway', 'McDonald', 'burgerK', 'burgerK'],
... ['burgerK', 'McDonald', 'subway', 'McDonald', 'burgerK', 'burgerK'],
... ['KFC', 'KFC', 'KFC', 'PizzaHot', 'McDonald', 'McDonald'],
... ['PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'McDonald'],
... ['PizzaHot', 'PizzaHot', 'PizzaHot', 'KFC', 'PizzaHot', 'subway'],
... ['McDonald', 'McDonald', 'McDonald', 'McDonald', 'McDonald', 'McDonald'],
... ['burgerK', 'burgerK', 'McDonald', 'burgerK', 'burgerK', 'burgerK'],
... ['burgerK', 'subway', 'subway', 'PizzaHot', 'subway', 'subway'],
... ['burgerK', 'burgerK', 'McDonald', 'burgerK', 'burgerK', 'burgerK'],
... ['PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'McDonald'],
... ], agent_first = False)
array([23, 20, 12, 23, 25, 25])


Call the function DMI with answers x and the number of choices C. x should be given in the numpy.array form or list form. If there are n agents and m tasks, x is a n*m matrix. Please make sure m >= 2c and each answer in x is an integer in [0, c), otherwise the function will raise a ValueError. DMI scores will return in numpy.array form.

Here is an example

>>> fem.DMI([[1, 1, 0, 1, 1, 0, 1, 1, 1], [1, 1, 0, 0, 1, 0, 1, 0, 1]], 2)
array([1.5, 1.5])


We implemented the multi-task common ground mechanism MCG(f) for Bernoulli distribution case. Call the function MCG with answers, function f and prior. The answers should be a 2*n matrix in numpy.array form or list form for 2 agents' prediction and all the number in answers should in [0, 1]. The prior is a number in [0, 1], too. f should be in ["TVD", "KLD"] for Total Variation Distance and KL divergence, respectively. By default, f = "TVD" . More functions will be supported in the future. The payments will return in numpy.array form.

Here is an example

>>> fem.DMI([[0.2, 0.3, 0.2], [0.3, 0.5, 0.3]], 'TVD', 0.3)
array([0.3333333333333333, 0.3333333333333333])

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

femtools-0.0.5-py3-none-any.whl (7.1 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page