Skip to main content

Implicit feedback-based recommender system pack

Project description

irspack

Python pypi GitHub license Build Read the Docs codecov

Docs

irspack is a Python package to train, evaluate, and optimize recommender systems based on implicit feedback.

There are already great packages for this purpose like

However, I decided to implement my own one to

  • Use optuna for more efficient parameter search. In particular, if an early stopping scheme is available, optuna can prune unpromising trial based on the intermediate validation score, which drastically reduces overall running time for tuning.
  • Use multi-threaded implementations wherever possible. Currently, several important algorithms (KNN, iALS, SLIM) and performance evaluators are parallelized using C++ thread.

Installation & Optional Dependencies

There are binaries for Linux, MacOS, and Windows with python>=3.6 with x86 architectures. You can install them via

pip install irspack

The binaries have been compiled to use AVX instruction. If you want to use AVX2/AVX512 or your environment does not support AVX (like Rosetta 2 on Apple M1), install it from source like

CFLAGS="-march=native" pip install git+https://github.com/tohtsky/irspack.git

In that case, you must have a decent version of C++ compiler (with C++11 support). If it doesn't work, feel free to make an issue!

Optional Dependencies

I have also prepared a wrapper class (BPRFMRecommender) to train/optimize BPR/warp loss Matrix factorization implemented in lightfm. To use it you have to install lightfm separately, e.g. by

pip install lightfm

If you want to use Mult-VAE and CB2CF features in cold-start scenarios, you'll need the following additional (pip-installable) packages:

Basic Usage

Step 1. Train a recommender

We first represent the user/item interaction as a scipy.sparse matrix. Then we can feed it into our Recommender classes:

import numpy as np
import scipy.sparse as sps
from irspack.recommenders import P3alphaRecommender
from irspack.dataset.movielens import MovieLens100KDataManager

df = MovieLens100KDataManager().read_interaction()
unique_user_id, user_index = np.unique(df.userId, return_inverse=True)
unique_movie_id, movie_index = np.unique(df.movieId, return_inverse=True)
X_interaction = sps.csr_matrix(
  (np.ones(df.shape[0]), (user_index, movie_index))
)

recommender = P3alphaRecommender(X_interaction)
recommender.learn()

# for user 0 (whose userId is unique_user_id[0]),
# compute the masked score (i.e., already seen items have the score of negative infinity)
# of items.
recommender.get_score_remove_seen([0])

Step 2. Evaluate on a validation set

We have to split the dataset to train and validation sets

from irspack.split import rowwise_train_test_split
from irspack.evaluator import Evaluator

# Random split
X_train, X_val = rowwise_train_test_split(
    X_interaction, test_ratio=0.2, random_state=0
)

evaluator = Evaluator(ground_truth=X_val)

recommender = P3alphaRecommender(X_train)
recommender.learn()
evaluator.get_score(recommender)

This will print something like

{
  'appeared_item': 106.0,
  'entropy': 3.840445116672292,
  'gini_index': 0.9794929280523742,
  'hit': 0.8854718981972428,
  'map': 0.11283343078231302,
  'n_items': 1682.0,
  'ndcg': 0.3401244303579389,
  'precision': 0.27560975609756017,
  'recall': 0.19399215770339678,
  'total_user': 943.0,
  'valid_user': 943.0
}

Step 3. Optimize the Hyperparameter

Now that we can evaluate the recommenders' performance against the validation set, we can use optuna-backed hyperparameter optimizer.

from irspack.optimizers import P3alphaOptimizer

optimizer = P3alphaOptimizer(X_train, evaluator)
best_params, trial_dfs  = optimizer.optimize(n_trials=20)

# maximal ndcg around 0.38 ~ 0.39
trial_dfs["ndcg@10"].max()

Of course, we have to hold-out another interaction set for test, and measure the performance of tuned recommender against the test set. See examples/ for more complete examples.

TODOs

  • more benchmark dataset

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

irspack-0.2.0.tar.gz (139.0 kB view details)

Uploaded Source

Built Distributions

irspack-0.2.0-cp310-cp310-win_amd64.whl (563.3 kB view details)

Uploaded CPython 3.10 Windows x86-64

irspack-0.2.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.8 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARM64

irspack-0.2.0-cp310-cp310-macosx_10_9_universal2.whl (1.3 MB view details)

Uploaded CPython 3.10 macOS 10.9+ universal2 (ARM64, x86-64)

irspack-0.2.0-cp39-cp39-win_amd64.whl (557.7 kB view details)

Uploaded CPython 3.9 Windows x86-64

irspack-0.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (10.8 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

irspack-0.2.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.8 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARM64

irspack-0.2.0-cp39-cp39-macosx_10_9_universal2.whl (1.3 MB view details)

Uploaded CPython 3.9 macOS 10.9+ universal2 (ARM64, x86-64)

irspack-0.2.0-cp38-cp38-win_amd64.whl (563.0 kB view details)

Uploaded CPython 3.8 Windows x86-64

irspack-0.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (10.8 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

irspack-0.2.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.8 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARM64

irspack-0.2.0-cp38-cp38-macosx_10_9_x86_64.whl (709.4 kB view details)

Uploaded CPython 3.8 macOS 10.9+ x86-64

irspack-0.2.0-cp37-cp37m-win_amd64.whl (563.4 kB view details)

Uploaded CPython 3.7m Windows x86-64

irspack-0.2.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (11.0 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARM64

irspack-0.2.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (10.7 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ x86-64

irspack-0.2.0-cp37-cp37m-macosx_10_9_x86_64.whl (702.8 kB view details)

Uploaded CPython 3.7m macOS 10.9+ x86-64

irspack-0.2.0-cp36-cp36m-win_amd64.whl (563.4 kB view details)

Uploaded CPython 3.6m Windows x86-64

File details

Details for the file irspack-0.2.0.tar.gz.

File metadata

  • Download URL: irspack-0.2.0.tar.gz
  • Upload date:
  • Size: 139.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for irspack-0.2.0.tar.gz
Algorithm Hash digest
SHA256 1c0bca3b2052a1962765a3e2eec3d43e2c9f31dc57db588afe99b82efc02e22b
MD5 81df3442b4a1b594716a3209809eb39e
BLAKE2b-256 5ce6ec9f7bb0b3b76f655ca6159177c6c8494799e259384c4315ee169f3222ee

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: irspack-0.2.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 563.3 kB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for irspack-0.2.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 d05f1cefa4302fc723315d060a5cc8a5f4ece1ac7f4ba5f619ace627cce87227
MD5 32256aaa34f78382acf8c95b36088547
BLAKE2b-256 7e47d5cd5f6a4db008e052cab270bb1c393c82e1d768f219b2c915c61484ec32

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for irspack-0.2.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 12a0052829e4c247aa3597f4162593d76c9dccad39d9caea0cee36458e421060
MD5 aa175eb11506f68ded036b8001da3827
BLAKE2b-256 16d4b597b3a3155362ed01051aa8502f89f5724e8c05f5c93b524180a06de18a

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp310-cp310-macosx_10_9_universal2.whl.

File metadata

  • Download URL: irspack-0.2.0-cp310-cp310-macosx_10_9_universal2.whl
  • Upload date:
  • Size: 1.3 MB
  • Tags: CPython 3.10, macOS 10.9+ universal2 (ARM64, x86-64)
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for irspack-0.2.0-cp310-cp310-macosx_10_9_universal2.whl
Algorithm Hash digest
SHA256 a2ee65c33bff420997612f4bce1a0de165465d5cb81219679e2ebffce945b0c7
MD5 b31e01a48a8776a9c4804d6cd895d490
BLAKE2b-256 b4726ec46b5397f7214768c9f3fcfeb658a6d3533e98efe5e8e5a5fd0b2526a3

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: irspack-0.2.0-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 557.7 kB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for irspack-0.2.0-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 07150c5b6dffcaf96aad27093ecfd78c3cde319129be649e648588a2e211be08
MD5 90a17f44cd585013833a97ab92ddb46c
BLAKE2b-256 b9fcac0b95ceb0d3ce8dc64ab659f3ff7928d93abc81842d64d648df9f290b14

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for irspack-0.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 084a75025b92c8f5e060a7e183aa5f11c1f686201ec601c80a24d270d0dce375
MD5 10d9ea8d4af317e95d66dd36bc45d9f0
BLAKE2b-256 55ab7ad67c5f32dde8c2c1284b813fb48879b87ea337298025cfaef758ae7b71

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for irspack-0.2.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 93a8827ada1cd3fcc8eaee646854ca4892c620d869af583efcf958ee4096cc9f
MD5 457c6b99be428111ee4240b655027a3c
BLAKE2b-256 e883e7c9a2ebc6bfc3b92f0c6dd8696f8fe2d8ea3340eae794afa059d27ba8e0

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp39-cp39-macosx_10_9_universal2.whl.

File metadata

  • Download URL: irspack-0.2.0-cp39-cp39-macosx_10_9_universal2.whl
  • Upload date:
  • Size: 1.3 MB
  • Tags: CPython 3.9, macOS 10.9+ universal2 (ARM64, x86-64)
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for irspack-0.2.0-cp39-cp39-macosx_10_9_universal2.whl
Algorithm Hash digest
SHA256 a3aac7e516171aa61e8a3c21f088cdeeabe1d8c929c46a3e83c9fd106b4bb588
MD5 13c1312a315fd11a1474cab515af2150
BLAKE2b-256 31ceef9cefebc1fec427fd7accbc598f7daed952f65adcdecb8a29555cec274b

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp38-cp38-win_amd64.whl.

File metadata

  • Download URL: irspack-0.2.0-cp38-cp38-win_amd64.whl
  • Upload date:
  • Size: 563.0 kB
  • Tags: CPython 3.8, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for irspack-0.2.0-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 f8aa016c640da8f275919821b5adedea6ac28f36a2ef3d8c1e81d4fda1b208c6
MD5 ef849fe161248447b7d94b9c0c917970
BLAKE2b-256 80d44f67f702750c76afad7b7edb6c12b0797d960299833135615fddcd012809

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for irspack-0.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 5ab45753e473f8331841ba24a837a81bbe861e56d09055cfbb3bd4b099a7bca5
MD5 765b21fd1196bf54cb86f7c6198d9022
BLAKE2b-256 eaa303c256fd5682841d68d9c7408906a0ee2dde51e16032cd96291980c0b06c

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for irspack-0.2.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 58e71402f272d1577e7ece4c2ed2178d0ba227ac25ebf71fd49eab5d5072e38e
MD5 7910a49f23f4b4e8569cda2725084e3f
BLAKE2b-256 c9e83b235c03513987af1d8dfad152372715551b2429b455184629bb415ae6df

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: irspack-0.2.0-cp38-cp38-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 709.4 kB
  • Tags: CPython 3.8, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for irspack-0.2.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 cc9cf6ee54e7f76482d9c63210610cc680ef830d61a01de4bffd23f301dce6d9
MD5 00a10c6eed5809924bbe6d37dacd2b9e
BLAKE2b-256 19e4e4115f5ffb854ffd392428d3cddc1d13eea7c03fb5c2ae177a73bd90048d

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp37-cp37m-win_amd64.whl.

File metadata

  • Download URL: irspack-0.2.0-cp37-cp37m-win_amd64.whl
  • Upload date:
  • Size: 563.4 kB
  • Tags: CPython 3.7m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for irspack-0.2.0-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 82a26b5adde4da52565ad83f0bf2b546ab24666dee76c814b1949cc293d6dcd0
MD5 76616a388280fdfe99e9e674e8e37cac
BLAKE2b-256 d52e3aeaad5f15ec8fd5c337ac1723e06219d2afbee47168f30df3744f3dc941

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for irspack-0.2.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 9e015f01fdcc139e63af078d94ebc947436f9ec232f6b69a5d9f742778343ce3
MD5 f3199ebb64b36d3b1d380623bdfdf210
BLAKE2b-256 16f3d719bbb0d34580baa6162dbd6b52e39eb33864d6ce0b47a53cbf75a86051

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for irspack-0.2.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 5200b34d2ac53072582ffd03733037a94c6ac03e47102f8d21d9b352506c1677
MD5 d8c2969191013350bd6b90ba1d5a8ed2
BLAKE2b-256 454ea93c94f61d6adca2ec9f79c5287d89e766cd2157168d3f835f55fec3de41

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp37-cp37m-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: irspack-0.2.0-cp37-cp37m-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 702.8 kB
  • Tags: CPython 3.7m, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for irspack-0.2.0-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 b2cffc463a9ac295bf52dddb6bea4b7a995afd4d54f7dfb717f81a6d596d01f5
MD5 7782dbf3e3473538741ad9b0a90b43bd
BLAKE2b-256 6197325f5f19c148821007a4caa8312c79e92d2ee4c6ac5e06e7b02c48a3b0a8

See more details on using hashes here.

File details

Details for the file irspack-0.2.0-cp36-cp36m-win_amd64.whl.

File metadata

  • Download URL: irspack-0.2.0-cp36-cp36m-win_amd64.whl
  • Upload date:
  • Size: 563.4 kB
  • Tags: CPython 3.6m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for irspack-0.2.0-cp36-cp36m-win_amd64.whl
Algorithm Hash digest
SHA256 63f7fe40802e0a14cb7fcbbd238a0b73e9ccf5aa767b61d4ff5c79c68866394d
MD5 67367a11a0426f22ff6eacd51c0101a8
BLAKE2b-256 a331bf62e2ab7426a840114253856a2e31976c66b0a8e1c17dc6d8eaa47c5e8f

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page