hornets 0.1.2

HorNets: Learning from Discrete and Continuous Signals with Routing Neural Networks

HorNets: Learning from Discrete and Continuous Signals with Routing Neural Networks

HorNets is a Python package implementing the HorNets architecture of the HorNets: Learning from Discrete and Continuous Signals with Routing Neural Networks paper.

Installation

Follow the instructions below to install HorNets using your preferred method.

Using PyPI stable version

You can install HorNets directly from PyPI using pip. This is the simplest method if you just want to use the package.

pip install hornets

Using PyPI latest version

pip install git+https://github.com/bkolosk1/hornets.git

Local Development Installation

To install locally with uv follow the following steps:

1. Install uv: pip install uv
2. Clone the repo: git clone [e-mail]:bkolosk1/hornets.git && cd hornets
3. Run: uv sync
4. Test the installation with: uv run python examples/examples.py

Usage

You can run examples/examples.py

Or you can run the minimal code snippet:

from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from hornets import HorNetClassifier, generate_synthetic_data

# Generate synthetic data
X, y = generate_synthetic_data(
    num_features=64,
    num_instances=128,
    operation="xor"
)

# Split into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# Initialize and train HorNetClassifier
classifier = HorNetClassifier(
    num_rules=256,
    exp_param=4,
    activation="polyclip",
    order=5,
    learning_rate=0.1,
    batch_size=10,
    stopping_crit=100,
    num_epochs=500,
    verbose=True
)
classifier.fit(X_train, y_train)

# Predict on test data
y_pred = classifier.predict(X_test)

# Evaluate
print("Classification Report:")
print(classification_report(y_test, y_pred))

Extracting Embeddings

After training, you can extract learned representations using transform():

# Extract embeddings (pre-classification hidden representations)
embeddings = classifier.transform(X_test)
print(embeddings.shape)  # (n_samples, num_rules) for binary input

For binary inputs, embeddings are the attention-weighted combination scores over feature rules. For continuous inputs, they are the attention-reweighted features. See examples/tox171_evaluation.py for a full example with t-SNE, UMAP, and PCA visualization on a real dataset.

Hyperparameters

Parameter	Type	Default	Description	Suggested HPO Range
batch_size	int	4	Number of samples per training batch.	4, 8, 16, 32
num_epochs	int	1000	Maximum number of epochs (complete passes through the training data). Large values allow more training time but risk overfitting.	100, 300, 500, 1000
learning_rate	float	1e-4	Learning rate for the optimizer. Higher values speed up training but risk overshooting minima; lower values may stabilize training but at the cost of slower convergence.	1e-5 to 1e-2
stopping_crit	int	10	Number of consecutive epochs without improvement on validation metric to wait before stopping training early.	5, 10, 15
feature_names	Optional[List[str]]	None	Names of input features.	Not applicable
num_rules	int	256	Number of rules (feature combinations) in the HorNet model. Increasing this can capture more complex patterns but may lead to higher risk of overfitting or longer training times.	64, 128, 256, 512
activation	str	"polyclip"	Activation function used in the model. Options include "polyclip" or "relu" functions.	"polyclip", "relu", "sigmoid", etc.
comb_samples_fp	int	48	Number of combination samples for feature processing. Larger values may capture more interaction patterns but at higher computational cost.	16, 32, 48, 64
exp_param	int	1	Expansion parameter for polynomial clipping within the activation. Primarily relevant for "polyclip" activations, controlling how inputs are processed.	1, 2, 3
order	int	5	Order of feature combinations. Higher orders enable capturing more complex relationships but can drastically increase computational demands.	2, 3, 5
device	str	"cpu"	Device for model training and inference. Using "cuda" (if available) can speed up training significantly, but "cpu" is a safer default.	"cpu", "cuda"
random_state	Optional[int]	None	Seed for random number generation, ensuring reproducible results. Not usually tuned as a hyperparameter, but important for experiments where repeatability is required.	Not applicable
verbose	bool	False	If True, prints progress and debug messages. Can be helpful during development or troubleshooting but is not generally part of hyperparameter optimization.	True or False

Citation

Cite this work as:

@Article{Koloski2025,
author={Koloski, Boshko
and Lavra{\v{c}}, Nada
and {\v{S}}krlj, Bla{\v{z}}},
title={HorNets: learning from discrete and continuous signals with routing neural networks},
journal={Machine Learning},
year={2025},
month={Feb},
day={21},
volume={114},
number={4},
pages={101},
issn={1573-0565},
doi={10.1007/s10994-024-06673-1},
url={https://doi.org/10.1007/s10994-024-06673-1}
}