swimnetworks 0.0.2

SWIM training of neural networks.

swimnetworks implements the algorithm SWIM for sampling weights of neural networks. The algorithm provides a way to quickly train neural networks on a CPU. For more details on the theoretical background of the method, refer to our paper [1].

The package documentation can be found at https://fd-research.gitlab.io/swimnetworks/.

Installation

To install the main package with the requirements, one needs to clone the repository and execute the following command from the root folder:

pip install .

Example

Here is a small example of defining a sampled network:

from sklearn.pipeline import Pipeline
from swimnetworks import Dense, Linear

steps = [
    ("dense", Dense(layer_width=512, activation="tanh",
                     parameter_sampler="tanh",
                     random_seed=42)),
    ("linear", Linear(regularization_scale=1e-10))
]
model = Pipeline(steps)

Then, one can use model.fit(X_train, y_train) and model.transform(X_test) to train and evaluate the model. The numerical experiments from [1] can be found in a separate repository.

Running Tests

Run all the tests using:

python3 -m unittest tests/*.py

You can test coverage after installing coverage (e.g., using pip):

pip install coverage

Then run:

coverage run -m unittest discover
coverage report

Citation

If you use the SWIM package in your research, please cite the following paper:

[1] (1,2)

E. Bolager, I. Burak, C. Datar, Q. Sun, F. Dietrich. Sampling weights of deep neural networks. arXiv:2306.16830, 2023.