Skip to main content

Forecast-elicitation-Mechanism

Project description

Forecast-elicitation-Mechanism

Implement 4 papers:

  • Water from Two Rocks: Maximizing the Mutual Information (wftr)
  • Dominantly Truthful Multi-task Peer Prediction with a Constant Number of Tasks (dmi)
  • A Bayesian truth serum for subjective data (BTS)
  • Informed Truthfulness in Multi-Task Peer Prediction (CA)

Usage

To begin

To begin, import femtools::

import numpy as np
import femtools as fem

BTS

For Bayesian Truth Serum, we implemented the version with finite players. Call the function BTS with answers x and predicted frequencies y, score for every respondent is returned. x and y can be given in the numpy.array form or list form. If there are n respondents and m possible answers, x should be an n-dimensional vector and each answer in x should be an integer in [0, m). Similarly, y is a n*m matrix denoting the predicted frequencies. BTS score is composed of information-score and prediction score, thus optional parameter alpha controlling the weight given to the prediction score could be assigned between (0,1]. By default, alpha is 1.

Here are examples::

>>> BTS([3, 2, 1, 1, 0],
[[0.1, 0.1, 0.3, 0.5],
[0.1, 0.2, 0.5, 0.2],
[0.3, 0.4, 0.2, 0.1],
[0.3, 0.4, 0.1, 0.2],
[0.1, 0.3, 0.2, 0.4]])
array([-3.28030172, -2.40787449, -0.29706308, -0.29706308, -1.074341  ])

>>> BTS([0, 0, 0],
[[0.5, 0.5],
[0.5, 0.5],
[0.5, 0.5]], alpha = 0.5)
array([0.51873113, 0.51873113, 0.51873113])

CA

For Correlated Agreement Mechanism, we implemented the detail-free version. CA Detail-Free is designed for multi-task problem with n agents and m tasks. Call the function CA with a n*m report matrix reports, score for every agent is returned. reports can be given in the numpy.array form or list form. For convenience, matrix reports can be given transposed with optional parameter agent_first = False. By default, agent_first is set to True. In addition, function CA does not expect that elements are integers.

Here is the example::

>>> CA([['subway', 'subway', 'subway', 'burgerK', 'burgerK', 'burgerK'],
	['burgerK', 'McDonald', 'subway', 'McDonald', 'burgerK', 'burgerK'],
	['burgerK', 'McDonald', 'subway', 'McDonald', 'burgerK', 'burgerK'],
	['KFC', 'KFC', 'KFC', 'PizzaHot', 'McDonald', 'McDonald'],
	['PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'McDonald'],
	['PizzaHot', 'PizzaHot', 'PizzaHot', 'KFC', 'PizzaHot', 'subway'],
	['McDonald', 'McDonald', 'McDonald', 'McDonald', 'McDonald', 'McDonald'],
	['burgerK', 'burgerK', 'McDonald', 'burgerK', 'burgerK', 'burgerK'],
	['burgerK', 'subway', 'subway', 'PizzaHot', 'subway', 'subway'],
	['burgerK', 'burgerK', 'McDonald', 'burgerK', 'burgerK', 'burgerK'],
	['PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'PizzaHot', 'McDonald'],
	], agent_first = False)
array([23, 20, 12, 23, 25, 25])

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Built Distribution

femtools-0.0.4-py3-none-any.whl (6.7 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page