reppi 0.1.2

A Python toolkit for representation-based learning/classification algorithms

reppi

A Python library for representation learning — sparse coding and dictionary learning algorithms implemented close to their original formulations.

Installation

pip install reppi

Algorithms

Algorithm	Class	Reference
Orthogonal Matching Pursuit	OMP	Elad et al., 2008
K-SVD	KSVD	Aharon et al., 2006
Label Consistent K-SVD	LCKSVD	Jiang et al., 2011

Convention

reppi follows the column-major convention common in the sparse representation literature: signals are columns, so data matrices are shaped (n_features, n_samples). This matches the MATLAB toolboxes the implementations are based on. If your data is in sklearn's (n_samples, n_features) layout, transpose before passing it in.

Quick Start

Installaion

pip install reppi

Sparse Coding with OMP

from reppi import OMP
import numpy as np

# D: (n_features, n_atoms), unit-norm columns
# X: (n_features, n_samples)
omp = OMP(n_nonzero_coefs=10)
Gamma = omp.encode(X, D)  # (n_atoms, n_samples)

Dictionary Learning with K-SVD

from reppi import KSVD

ksvd = KSVD(
    n_components=128,     # number of atoms
    n_nonzero_coefs=10,   # sparsity level T
    n_iter=20,
    verbose=True,
)
ksvd.fit(X_train)         # X_train: (n_features, n_samples)

D = ksvd.D_               # learned dictionary (n_features, n_components)
Gamma = ksvd.transform(X) # sparse codes (n_components, n_samples)

Discriminative Dictionary Learning with LC-KSVD

LC-KSVD jointly learns a dictionary and a linear classifier from labelled data. Labels are passed as a one-hot matrix H of shape (n_classes, n_samples).

LC-KSVD1 — reconstruction + label-consistency:

from reppi import LCKSVD

model = LCKSVD(
    n_components=570,
    n_nonzero_coefs=30,
    alpha=4.0,            # weight for label-consistency term
    variant="lcksvd1",
    n_iter=50,
    n_iter_init=20,       # K-SVD iterations for initialisation
    verbose=True,
)
model.fit(X_train, H_train)
Gamma = model.transform(X_test)  # (n_components, n_samples)

LC-KSVD2 — reconstruction + label-consistency + classification error:

model = LCKSVD(
    n_components=570,
    n_nonzero_coefs=30,
    alpha=4.0,
    beta=2.0,             # weight for classifier term
    variant="lcksvd2",
    n_iter=50,
    n_iter_init=20,
    verbose=True,
)
model.fit(X_train, H_train)

predictions = model.predict(X_test)        # integer class indices
accuracy    = model.score(X_test, H_test)  # float in [0, 1]

Supplying Your Own Initialisation

If you already have a pre-trained dictionary or want to control initialisation:

from reppi.dictionary.lc_ksvd import initialization4lcksvd

D_init, A_init, W_init, Q = initialization4lcksvd(
    X_train, H_train,
    n_components=570,
    n_iter_init=20,
    n_nonzero_coefs=30,
)

model.fit(X_train, H_train, D_init=D_init, A_init=A_init, W_init=W_init, Q=Q)

API Reference

OMP

OMP(n_nonzero_coefs, mode='batch', check_dict=True)

Parameter	Description
n_nonzero_coefs	Maximum non-zeros per signal (sparsity T)
mode	'batch' (Batch-OMP, fast) or 'cholesky' (single-signal, low memory)
check_dict	Verify unit-norm columns before encoding

Methods: encode(X, D, G=None) → Gamma

KSVD

KSVD(n_components, n_nonzero_coefs, n_iter=10, exact_svd=False,
     mu_thresh=0.99, mem_usage='normal', random_state=None, verbose=False)

Parameter	Description
n_components	Number of dictionary atoms
n_nonzero_coefs	Sparsity level T
n_iter	Training iterations
exact_svd	Use full SVD in atom update (slower, slightly better)
mu_thresh	Mutual-incoherence threshold; atoms with correlation above this are replaced (default 0.99)
mem_usage	'high' / 'normal' / 'low' — controls whether G=D'D and DtX are precomputed

Methods: fit(X, D_init=None), transform(X), fit_transform(X)
Attributes: D_, errors_

LCKSVD

LCKSVD(n_components, n_nonzero_coefs, alpha=4.0, beta=2.0,
       variant='lcksvd2', n_iter=50, n_iter_init=20, exact_svd=False,
       mu_thresh=0.99, random_state=None, verbose=False)

Parameter	Description
alpha	Weight for label-consistency term (√α in the paper)
beta	Weight for classifier term (√β); LC-KSVD2 only
variant	'lcksvd1' or 'lcksvd2'
n_iter_init	K-SVD iterations for the warm-start initialisation phase

Methods: fit(X, H, ...), transform(X), predict(X), score(X, H)
Attributes: D_, W_, A_, errors_

References

• M. Aharon, M. Elad, A. Bruckstein. "The K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation". IEEE Trans. Signal Processing, 54(11), 2006.
• M. Elad, R. Rubinstein, M. Zibulevsky. "Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit". Technion Technical Report, 2008.
• Z. Jiang, Z. Lin, L. Davis. "Learning A Discriminative Dictionary for Sparse Coding via Label Consistent K-SVD". CVPR, 2011.