Implicit feedback-based recommender system pack
Project description
irspack
irspack is a Python package to train, evaluate, and optimize recommender systems based on implicit feedback.
There are already great packages for this purpose like
- implicit
- daisyRec
- RecSys2019_DeepLearning_Evaluation (which has influenced this project the most)
However, I decided to implement my own one to
- Use optuna for more efficient parameter search. In particular, if an early stopping scheme is available, optuna can prune unpromising trial based on the intermediate validation score, which drastically reduces overall running time for tuning.
- Use multi-threaded implementations wherever possible. Currently, several important algorithms (KNN, iALS, SLIM) and performance evaluators are parallelized using C++ thread.
Installation & Optional Dependencies
There are binaries for Linux, MacOS, and Windows with python>=3.6 with x86 architectures. You can install them via
pip install irspack
The binaries have been compiled to use AVX instruction. If you want to use AVX2/AVX512 or your environment does not support AVX (like Rosetta 2 on Apple M1), install it from source like
CFLAGS="-march=native" pip install git+https://github.com/tohtsky/irspack.git
In that case, you must have a decent version of C++ compiler (with C++11 support). If it doesn't work, feel free to make an issue!
Optional Dependencies
I have also prepared a wrapper class (BPRFMRecommender
) to train/optimize BPR/warp loss Matrix factorization implemented in lightfm. To use it you have to install lightfm
separately, e.g. by
pip install lightfm
If you want to use Mult-VAE and CB2CF features in cold-start scenarios, you'll need the following additional (pip-installable) packages:
- scikit-learn
- jax
- jaxlib
- If you want to use GPU, follow the installation guide https://github.com/google/jax#installation
- dm-haiku
- optax
Basic Usage
Step 1. Train a recommender
We first represent the user/item interaction as a scipy.sparse matrix. Then we can feed it into our Recommender
classes:
import numpy as np
import scipy.sparse as sps
from irspack.recommenders import P3alphaRecommender
from irspack.dataset.movielens import MovieLens100KDataManager
df = MovieLens100KDataManager().read_interaction()
unique_user_id, user_index = np.unique(df.userId, return_inverse=True)
unique_movie_id, movie_index = np.unique(df.movieId, return_inverse=True)
X_interaction = sps.csr_matrix(
(np.ones(df.shape[0]), (user_index, movie_index))
)
recommender = P3alphaRecommender(X_interaction)
recommender.learn()
# for user 0 (whose userId is unique_user_id[0]),
# compute the masked score (i.e., already seen items have the score of negative infinity)
# of items.
recommender.get_score_remove_seen([0])
Step 2. Evaluate on a validation set
We have to split the dataset to train and validation sets
from irspack.split import rowwise_train_test_split
from irspack.evaluator import Evaluator
# Random split
X_train, X_val = rowwise_train_test_split(
X_interaction, test_ratio=0.2, random_state=0
)
evaluator = Evaluator(ground_truth=X_val)
recommender = P3alphaRecommender(X_train)
recommender.learn()
evaluator.get_score(recommender)
This will print something like
{
'appeared_item': 106.0,
'entropy': 3.840445116672292,
'gini_index': 0.9794929280523742,
'hit': 0.8854718981972428,
'map': 0.11283343078231302,
'n_items': 1682.0,
'ndcg': 0.3401244303579389,
'precision': 0.27560975609756017,
'recall': 0.19399215770339678,
'total_user': 943.0,
'valid_user': 943.0
}
Step 3. Optimize the Hyperparameter
Now that we can evaluate the recommenders' performance against the validation set, we can use optuna-backed hyperparameter optimizer.
from irspack.optimizers import P3alphaOptimizer
optimizer = P3alphaOptimizer(X_train, evaluator)
best_params, trial_dfs = optimizer.optimize(n_trials=20)
# maximal ndcg around 0.38 ~ 0.39
trial_dfs["ndcg@10"].max()
Of course, we have to hold-out another interaction set for test,
and measure the performance of tuned recommender against the test set.
See examples/
for more complete examples.
TODOs
- more benchmark dataset
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
File details
Details for the file irspack-0.2.0.tar.gz
.
File metadata
- Download URL: irspack-0.2.0.tar.gz
- Upload date:
- Size: 139.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1c0bca3b2052a1962765a3e2eec3d43e2c9f31dc57db588afe99b82efc02e22b |
|
MD5 | 81df3442b4a1b594716a3209809eb39e |
|
BLAKE2b-256 | 5ce6ec9f7bb0b3b76f655ca6159177c6c8494799e259384c4315ee169f3222ee |
File details
Details for the file irspack-0.2.0-cp310-cp310-win_amd64.whl
.
File metadata
- Download URL: irspack-0.2.0-cp310-cp310-win_amd64.whl
- Upload date:
- Size: 563.3 kB
- Tags: CPython 3.10, Windows x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | d05f1cefa4302fc723315d060a5cc8a5f4ece1ac7f4ba5f619ace627cce87227 |
|
MD5 | 32256aaa34f78382acf8c95b36088547 |
|
BLAKE2b-256 | 7e47d5cd5f6a4db008e052cab270bb1c393c82e1d768f219b2c915c61484ec32 |
File details
Details for the file irspack-0.2.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
.
File metadata
- Download URL: irspack-0.2.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- Upload date:
- Size: 10.8 MB
- Tags: CPython 3.10, manylinux: glibc 2.17+ ARM64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 12a0052829e4c247aa3597f4162593d76c9dccad39d9caea0cee36458e421060 |
|
MD5 | aa175eb11506f68ded036b8001da3827 |
|
BLAKE2b-256 | 16d4b597b3a3155362ed01051aa8502f89f5724e8c05f5c93b524180a06de18a |
File details
Details for the file irspack-0.2.0-cp310-cp310-macosx_10_9_universal2.whl
.
File metadata
- Download URL: irspack-0.2.0-cp310-cp310-macosx_10_9_universal2.whl
- Upload date:
- Size: 1.3 MB
- Tags: CPython 3.10, macOS 10.9+ universal2 (ARM64, x86-64)
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a2ee65c33bff420997612f4bce1a0de165465d5cb81219679e2ebffce945b0c7 |
|
MD5 | b31e01a48a8776a9c4804d6cd895d490 |
|
BLAKE2b-256 | b4726ec46b5397f7214768c9f3fcfeb658a6d3533e98efe5e8e5a5fd0b2526a3 |
File details
Details for the file irspack-0.2.0-cp39-cp39-win_amd64.whl
.
File metadata
- Download URL: irspack-0.2.0-cp39-cp39-win_amd64.whl
- Upload date:
- Size: 557.7 kB
- Tags: CPython 3.9, Windows x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 07150c5b6dffcaf96aad27093ecfd78c3cde319129be649e648588a2e211be08 |
|
MD5 | 90a17f44cd585013833a97ab92ddb46c |
|
BLAKE2b-256 | b9fcac0b95ceb0d3ce8dc64ab659f3ff7928d93abc81842d64d648df9f290b14 |
File details
Details for the file irspack-0.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
.
File metadata
- Download URL: irspack-0.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Upload date:
- Size: 10.8 MB
- Tags: CPython 3.9, manylinux: glibc 2.17+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 084a75025b92c8f5e060a7e183aa5f11c1f686201ec601c80a24d270d0dce375 |
|
MD5 | 10d9ea8d4af317e95d66dd36bc45d9f0 |
|
BLAKE2b-256 | 55ab7ad67c5f32dde8c2c1284b813fb48879b87ea337298025cfaef758ae7b71 |
File details
Details for the file irspack-0.2.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
.
File metadata
- Download URL: irspack-0.2.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- Upload date:
- Size: 10.8 MB
- Tags: CPython 3.9, manylinux: glibc 2.17+ ARM64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 93a8827ada1cd3fcc8eaee646854ca4892c620d869af583efcf958ee4096cc9f |
|
MD5 | 457c6b99be428111ee4240b655027a3c |
|
BLAKE2b-256 | e883e7c9a2ebc6bfc3b92f0c6dd8696f8fe2d8ea3340eae794afa059d27ba8e0 |
File details
Details for the file irspack-0.2.0-cp39-cp39-macosx_10_9_universal2.whl
.
File metadata
- Download URL: irspack-0.2.0-cp39-cp39-macosx_10_9_universal2.whl
- Upload date:
- Size: 1.3 MB
- Tags: CPython 3.9, macOS 10.9+ universal2 (ARM64, x86-64)
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a3aac7e516171aa61e8a3c21f088cdeeabe1d8c929c46a3e83c9fd106b4bb588 |
|
MD5 | 13c1312a315fd11a1474cab515af2150 |
|
BLAKE2b-256 | 31ceef9cefebc1fec427fd7accbc598f7daed952f65adcdecb8a29555cec274b |
File details
Details for the file irspack-0.2.0-cp38-cp38-win_amd64.whl
.
File metadata
- Download URL: irspack-0.2.0-cp38-cp38-win_amd64.whl
- Upload date:
- Size: 563.0 kB
- Tags: CPython 3.8, Windows x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | f8aa016c640da8f275919821b5adedea6ac28f36a2ef3d8c1e81d4fda1b208c6 |
|
MD5 | ef849fe161248447b7d94b9c0c917970 |
|
BLAKE2b-256 | 80d44f67f702750c76afad7b7edb6c12b0797d960299833135615fddcd012809 |
File details
Details for the file irspack-0.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
.
File metadata
- Download URL: irspack-0.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Upload date:
- Size: 10.8 MB
- Tags: CPython 3.8, manylinux: glibc 2.17+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5ab45753e473f8331841ba24a837a81bbe861e56d09055cfbb3bd4b099a7bca5 |
|
MD5 | 765b21fd1196bf54cb86f7c6198d9022 |
|
BLAKE2b-256 | eaa303c256fd5682841d68d9c7408906a0ee2dde51e16032cd96291980c0b06c |
File details
Details for the file irspack-0.2.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
.
File metadata
- Download URL: irspack-0.2.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- Upload date:
- Size: 10.8 MB
- Tags: CPython 3.8, manylinux: glibc 2.17+ ARM64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 58e71402f272d1577e7ece4c2ed2178d0ba227ac25ebf71fd49eab5d5072e38e |
|
MD5 | 7910a49f23f4b4e8569cda2725084e3f |
|
BLAKE2b-256 | c9e83b235c03513987af1d8dfad152372715551b2429b455184629bb415ae6df |
File details
Details for the file irspack-0.2.0-cp38-cp38-macosx_10_9_x86_64.whl
.
File metadata
- Download URL: irspack-0.2.0-cp38-cp38-macosx_10_9_x86_64.whl
- Upload date:
- Size: 709.4 kB
- Tags: CPython 3.8, macOS 10.9+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | cc9cf6ee54e7f76482d9c63210610cc680ef830d61a01de4bffd23f301dce6d9 |
|
MD5 | 00a10c6eed5809924bbe6d37dacd2b9e |
|
BLAKE2b-256 | 19e4e4115f5ffb854ffd392428d3cddc1d13eea7c03fb5c2ae177a73bd90048d |
File details
Details for the file irspack-0.2.0-cp37-cp37m-win_amd64.whl
.
File metadata
- Download URL: irspack-0.2.0-cp37-cp37m-win_amd64.whl
- Upload date:
- Size: 563.4 kB
- Tags: CPython 3.7m, Windows x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 82a26b5adde4da52565ad83f0bf2b546ab24666dee76c814b1949cc293d6dcd0 |
|
MD5 | 76616a388280fdfe99e9e674e8e37cac |
|
BLAKE2b-256 | d52e3aeaad5f15ec8fd5c337ac1723e06219d2afbee47168f30df3744f3dc941 |
File details
Details for the file irspack-0.2.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
.
File metadata
- Download URL: irspack-0.2.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- Upload date:
- Size: 11.0 MB
- Tags: CPython 3.7m, manylinux: glibc 2.17+ ARM64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9e015f01fdcc139e63af078d94ebc947436f9ec232f6b69a5d9f742778343ce3 |
|
MD5 | f3199ebb64b36d3b1d380623bdfdf210 |
|
BLAKE2b-256 | 16f3d719bbb0d34580baa6162dbd6b52e39eb33864d6ce0b47a53cbf75a86051 |
File details
Details for the file irspack-0.2.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
.
File metadata
- Download URL: irspack-0.2.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
- Upload date:
- Size: 10.7 MB
- Tags: CPython 3.7m, manylinux: glibc 2.12+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5200b34d2ac53072582ffd03733037a94c6ac03e47102f8d21d9b352506c1677 |
|
MD5 | d8c2969191013350bd6b90ba1d5a8ed2 |
|
BLAKE2b-256 | 454ea93c94f61d6adca2ec9f79c5287d89e766cd2157168d3f835f55fec3de41 |
File details
Details for the file irspack-0.2.0-cp37-cp37m-macosx_10_9_x86_64.whl
.
File metadata
- Download URL: irspack-0.2.0-cp37-cp37m-macosx_10_9_x86_64.whl
- Upload date:
- Size: 702.8 kB
- Tags: CPython 3.7m, macOS 10.9+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | b2cffc463a9ac295bf52dddb6bea4b7a995afd4d54f7dfb717f81a6d596d01f5 |
|
MD5 | 7782dbf3e3473538741ad9b0a90b43bd |
|
BLAKE2b-256 | 6197325f5f19c148821007a4caa8312c79e92d2ee4c6ac5e06e7b02c48a3b0a8 |
File details
Details for the file irspack-0.2.0-cp36-cp36m-win_amd64.whl
.
File metadata
- Download URL: irspack-0.2.0-cp36-cp36m-win_amd64.whl
- Upload date:
- Size: 563.4 kB
- Tags: CPython 3.6m, Windows x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 63f7fe40802e0a14cb7fcbbd238a0b73e9ccf5aa767b61d4ff5c79c68866394d |
|
MD5 | 67367a11a0426f22ff6eacd51c0101a8 |
|
BLAKE2b-256 | a331bf62e2ab7426a840114253856a2e31976c66b0a8e1c17dc6d8eaa47c5e8f |