Implicit feedback-based recommender systems, packed for practitioners.
Project description
irspack - Implicit recommender systems for practitioners
irspack is a Python package for recommender systems based on implicit feedback, designed to be used by practitioners.
Some of its features include:
- Efficient parameter tuning enabled by C++/Eigen implementations of core recommender algorithms and optuna.
- In particular, if an early stopping scheme is available, optuna can prune out unpromising trial based on the intermediate validation scores.
- Various utility functions, including
- ID/index mapping utilities
- Fast, multithreaded argsort for batch recommendation retrieval
- Efficient and configurable evaluation of recommender system performance
Installation & Optional Dependencies
In most cases, you can install the pre-build binaries via
pip install irspack
The binaries have been compiled to use AVX instruction. If you want to use AVX2/AVX512 or your environment does not support AVX (like Rosetta 2 on Apple M1), install it from source like
CFLAGS="-march=native" pip install git+https://github.com/tohtsky/irspack.git
In that case, you must have a decent version of C++ compiler (with C++11 support). If it doesn't work, feel free to make an issue!
Optional Dependencies
I have also prepared a wrapper class (BPRFMRecommender
) to train/optimize BPR/warp loss Matrix factorization implemented in lightfm. To use it you have to install lightfm
separately, e.g. by
pip install lightfm
If you want to use Mult-VAE, you'll need the following additional (pip-installable) packages:
- jax
- jaxlib
- If you want to use GPU, follow the installation guide https://github.com/google/jax#installation
- dm-haiku
- optax
Basic Usage
Step 1. Train a recommender
To begin with, we represent the user/item interaction as a scipy.sparse matrix. Then we can feed it into recommender classes:
import numpy as np
import scipy.sparse as sps
from irspack import IALSRecommender, df_to_sparse
from irspack.dataset import MovieLens100KDataManager
df = MovieLens100KDataManager().read_interaction()
# Convert pandas.Dataframe into scipy's sparse matrix.
# The i'th row of `X_interaction` corresponds to `unique_user_id[i]`
# and j'th column of `X_interaction` corresponds to `unique_movie_id[j]`.
X_interaction, unique_user_id, unique_movie_id = df_to_sparse(
df, 'userId', 'movieId'
)
recommender = IALSRecommender(X_interaction)
recommender.learn()
# for user 0 (whose userId is unique_user_id[0]),
# compute the masked score (i.e., already seen items have the score of negative infinity)
# of items.
recommender.get_score_remove_seen([0])
Step 2. Evaluation on a validation set
To evaluate the performance of a recommenderm we have to split the dataset to train and validation sets:
from irspack.split import rowwise_train_test_split
from irspack.evaluation import Evaluator
# Random split
X_train, X_val = rowwise_train_test_split(
X_interaction, test_ratio=0.2, random_state=0
)
evaluator = Evaluator(ground_truth=X_val)
recommender = IALSRecommender(X_train)
recommender.learn()
evaluator.get_score(recommender)
This will print something like
{
'appeared_item': 435.0,
'entropy': 5.160409123151053,
'gini_index': 0.9198367595008214,
'hit': 0.40084835630965004,
'map': 0.013890322881619916,
'n_items': 1682.0,
'ndcg': 0.07867240014767263,
'precision': 0.06797454931071051,
'recall': 0.03327028758587522,
'total_user': 943.0,
'valid_user': 943.0
}
Step 3. Hyperparameter optimization
Now that we can evaluate the recommenders' performance against the validation set, we can use optuna-backed hyperparameter optimization.
best_params, trial_dfs = IALSRecommender.tune(X_train, evaluator, n_trials=20)
# maximal ndcg around 0.43 ~ 0.45
trial_dfs["ndcg@10"].max()
Of course, we have to hold-out another interaction set for test, and measure the performance of tuned recommender against the test set.
See examples/
for more complete examples.
TODOs
- more benchmark dataset
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for irspack-0.3.1-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 03af236ded28d0d4f2477d8d089a36c5ae93f2b6bfc5a1e8563761789f211bab |
|
MD5 | 681e45287aeba462c8d60e592b452515 |
|
BLAKE2b-256 | b128fec8e2286225045ad9676fa7637bdbaecf9e6ad424d5671edec6de729c63 |
Hashes for irspack-0.3.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b111f6c471c8dcb7512fd8136d2cf9c78d3d0ba3f853fe7c1cc4790e5069a793 |
|
MD5 | 85b36988a4576b1a7405f14f608a62aa |
|
BLAKE2b-256 | dcd6db13f868f99f09639b2fbea0c45fd0ed1ce5c1a3d0e08d2aa1e8d6f714ae |
Hashes for irspack-0.3.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e8c8fa2df7fe48f25fa4050dc4202b76d99a449ed6c9ff5c65eecb722a051734 |
|
MD5 | cc09a6cad9e37fe05ee0156c4fbecbaf |
|
BLAKE2b-256 | b7698b249a6652d95edcb55608d4c9d88a4c97f22fbdad599da311bb24dd65c6 |
Hashes for irspack-0.3.1-cp311-cp311-macosx_10_14_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e396858205ba7e039d0f4d17fa461bf615cfd8bb10f6d5ec3500e6a145fddeb8 |
|
MD5 | 4f06414f906320f3f562888a5a163258 |
|
BLAKE2b-256 | 520ec3f3ede912dac7af9540b856be849485f463fda88fabd69affff3ce59820 |
Hashes for irspack-0.3.1-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 40fa33e1623fa5dda6b9a5248016844b581af1bf49b128515d62f5e23d77a0b6 |
|
MD5 | f5ccbdc9ee184d2fcaa59eb3ce9c292f |
|
BLAKE2b-256 | 875aa88337564886d13c318c7366ee1d276d428d55214d98417c00fb2d79f3dc |
Hashes for irspack-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8fc452e5de8140f328156c28ff1a66a7109555ea5a1588d190df27c6fc665aae |
|
MD5 | b4170ccbf66af7bb39541a0f5a8a107d |
|
BLAKE2b-256 | ad5325dd1403c8c3158f1cafbb0b463cc70c7c13a86ca9966f07c1d1a6bdc762 |
Hashes for irspack-0.3.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9c2bb1b5c890a070c89b0f78eab97f8a9cf9e0b60eea5516b8ff4e22e5bf2a5a |
|
MD5 | 4b0e864cb2b06be8c1b85161fdcc605d |
|
BLAKE2b-256 | 251c966986ce0f77b42aa590e0ff3543571520e06905ad4b4c3ddde03e575a48 |
Hashes for irspack-0.3.1-cp310-cp310-macosx_10_14_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 950a038be08e120de9705a5cf13c0ede85ee9ace1150be41ed8f72dda711fc45 |
|
MD5 | c5a236d4524a13483aabaf9436baf18f |
|
BLAKE2b-256 | 71941e7c358827de8f957ed419aaf01a7fe946084949c4d3afe818b5ee9910c1 |
Hashes for irspack-0.3.1-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c4dbac12fd183cd7f90f870a5731c5475775803848ce9e2a0c3f061d1b1eddee |
|
MD5 | 8346a804d6510190cfbe867f0033a7bf |
|
BLAKE2b-256 | 6040bf845c388a167843b8fd1da0eda4e764f0844c14f65639b1c2a9b0605ceb |
Hashes for irspack-0.3.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 396084974b6a072c41749ff8e913de771e97d906e282572cfd4dcddb16e805da |
|
MD5 | b8599d25bf5c47c8f4090911cb05f3f4 |
|
BLAKE2b-256 | bbbca4066fe64de05edb3e3688eea1f255ef9e727e8cf99ea4e291dc5edbde1b |
Hashes for irspack-0.3.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f0361d65a908c4925b8f8e4d4507208846deb7c259afd19bb19337cd56c7abb4 |
|
MD5 | 04666433cffdfa1ee8a6eba00d340331 |
|
BLAKE2b-256 | 22a0db6df769888edc090c55b4a98835e2c17a13a8deca9c280633f81df2b263 |
Hashes for irspack-0.3.1-cp39-cp39-macosx_10_14_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 704ba610d28f0d39a3a03c6e15f1d52e0933d07c640f9981a9ff9b01769f0a78 |
|
MD5 | 650e3b44bdc11d3e2d8c8882af323d2b |
|
BLAKE2b-256 | 8929eca1325539265f6b58697d3757f3da9afe153318d3814d8bff0a04090727 |
Hashes for irspack-0.3.1-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 881e3477613f32d146adf3753de54027695a212e6313235bc8222576c9d59547 |
|
MD5 | b9cecc4068a24bce51858e5e568dfabd |
|
BLAKE2b-256 | 8ba3377f4a2ad7f773aa3a6ececcbb5e9dd4246b5f3d1eb4982e8f5fcaf01fe0 |
Hashes for irspack-0.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 00a62ef3054b03b881698a2f5058a933c9d5269269858601f00150c138a8db1f |
|
MD5 | 6ed06ef6f27be4e9a5109179d10d3318 |
|
BLAKE2b-256 | 0e9c9ab37507dbf84a04998d3d3c71dfa9d68e9fb193b3a2ea0bc602ab586ccb |
Hashes for irspack-0.3.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b66c368bda6f267c41f1a8380eca9d2b56264726450fa6bdf6b98990bbda1225 |
|
MD5 | 4ba752d20b057386dd27ccb26f2d5552 |
|
BLAKE2b-256 | 2f2d947a863850eff497d68a6064dd274b6635d1afdb6addd82b207c86a406c2 |
Hashes for irspack-0.3.1-cp38-cp38-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f8f020b9a9001b96f1cadd78dfa151d7c7961ea4b6711a0dc20963a9dbe9edbf |
|
MD5 | 660630dfcbd599716669a6d9def68997 |
|
BLAKE2b-256 | 5e180750c83c531d620f3e4c0281640c44e9dd5e77c0ebe0038f559215499b97 |
Hashes for irspack-0.3.1-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b6dadbd5ee1a6803215217fde5b14a32166a9720026306e45087717c8c7f4e61 |
|
MD5 | d7ce224bf5f8b15a01ef2fffef50ed5e |
|
BLAKE2b-256 | 06d626c096a21d439b2dad6085e4e92ee0f85f753868a846d10d00a92bd4c3dd |
Hashes for irspack-0.3.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8bd47de8c6a0b0611b35aac87f8fcea1919d61a1c27f5f064e1914ef028b1711 |
|
MD5 | fca108fed8ae77b0d85d19ff28f9ec5e |
|
BLAKE2b-256 | 2f595397cdecd7aa39ba3e34f5c1cb53a79ed619eea31ab66500fb1104b8aa55 |
Hashes for irspack-0.3.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 64bd81c33aec4386f72e81d717a87a85272ad698c483d9acdc82c77d04e83e25 |
|
MD5 | e177d1863c9b52ad6e6c1b3dca31e556 |
|
BLAKE2b-256 | fbc38350cd7134fc8e8d74a5e161c4f723e1895ca0ef34aac2cb7cf0871f088b |
Hashes for irspack-0.3.1-cp37-cp37m-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b95a79aee01b775bf5f7b454fdc7945822dd904b3503f4db07365c6c714894c6 |
|
MD5 | 6bfb8a2c920611423581322882278710 |
|
BLAKE2b-256 | c66694af777d3bbee26930fff5b0502a01899fbb1aabcc690ae17d37a174b900 |