Skip to main content

A fast implementation of ranking metrics for information retrieval and recommendation.

Project description

Latest PyPI version Latest GitHub actions build status

A fast numpy/numba-based implementation of ranking metrics for information retrieval and recommendation. Coded with efficiency in mind and support for edge cases.

Find the full documentation here.

Features

  • Wide array of evaluation metrics for information retrieval and top-N recommender systems:

    • Binary labels: Recall, Precision, MAP, HitRate, MRR, MeanRanks, F1

    • Numeric and binary labels: DCG, nDCG

  • Minimal dependencies: Numpy and Numba (required), SciPy (optional)

  • Flexible input formats: Supports arrays, lists and sparse matrices

  • Built-in support for confidence intervals via bootstrapping

Usage

from rankereval import BinaryLabels, Rankings, Recall

y_true = BinaryLabels.from_positive_indices([[0,2], [0,1,2]])
y_pred = Rankings.from_ranked_indices([[2,1], [1]])

recall_at_3 = Recall(3).mean(y_true, y_pred)
print(recall_at_3["score"])

To get confidence intervals (95% by default), specify conf_interval=True:

recall_at_3 = Recall(3).mean(y_true, y_pred, conf_interval=True)
print(recall_at_3["conf_interval"])

Input formats

RankerEval allows for a variety of input formats, e.g.,

# specify all labels as lists
y_true = BinaryLabels.from_dense([[1,0,1], [1,1,1]])

# specify labels as numpy array
y_true = BinaryLabels.from_dense(np.asarray([[1,0,1], [1,1,1]]))

# or use a sparse matrix
import scipy.sparse as sp
y_true = BinaryLabels.from_sparse(sp.coo_matrix([[1,0,1], [1,1,1]]))

Installation

To install (requires Numpy 1.18 or newer):

pip install rankereval

Licence

This project is licensed under MIT.

Authors

RankerEval was written by Tobias Schnabel.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com <mailto:opencode@microsoft.com> with any additional questions or comments.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

rankerEval-0.2.0.tar.gz (13.0 kB view details)

Uploaded Source

Built Distribution

rankerEval-0.2.0-py3-none-any.whl (14.3 kB view details)

Uploaded Python 3

File details

Details for the file rankerEval-0.2.0.tar.gz.

File metadata

  • Download URL: rankerEval-0.2.0.tar.gz
  • Upload date:
  • Size: 13.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.7.1 importlib_metadata/4.8.1 pkginfo/1.8.2 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.7.10

File hashes

Hashes for rankerEval-0.2.0.tar.gz
Algorithm Hash digest
SHA256 b61d9bc3f1caf501249b5be8d2c9ec44cb8c9f032705eb6b1308ba794fe07bb1
MD5 063fc1c56eb83798516b0ab68bda47b5
BLAKE2b-256 1182cbf208f76073849b96d906452d1ee54364765431f0e7127c6576cf7274d6

See more details on using hashes here.

File details

Details for the file rankerEval-0.2.0-py3-none-any.whl.

File metadata

  • Download URL: rankerEval-0.2.0-py3-none-any.whl
  • Upload date:
  • Size: 14.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.7.1 importlib_metadata/4.8.1 pkginfo/1.8.2 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.7.10

File hashes

Hashes for rankerEval-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 307568fbad3b1daff81131e2d799b5b40059766125e1c5d0309def2a706eced4
MD5 9ce93ed00c97b894015566a9c0236982
BLAKE2b-256 ed0ff5e57af3617bac0a3b259c5792ce399f07c8395b8221637721880192615f

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page