PyCasCor

Description

A python implementation of a Cascade Correlation Network Simulator

Installation

Easiest Option - Install from PyPi

pip install pycascor

Using the source repository

1. Clone the repository locally
   1. git clone --recurse-submodules https://gitlab.com/cpsimpson/cascade-correlation-network-simulator.git
      1. If you clone without the --recurse-submodules the proben1 repository will not be cloned. You can pull this file after cloning with the following commands git submodule init and git submodule update.
2. Create and activate virtual environment
   1. python -m venv venv
   2. . venv/bin/activate
3. Install dependencies
   1. pip install .

Usage

Example of Training on XOR

Cascade Correlation Network

from pycascor.network import CCNetwork
from pycascor.activation_functions import Sigmoid
from pycascor.trainer import Trainer

xor_inputs = [[0, 0], [0, 1], [1, 0], [1, 1]]
xor_targets = [[-0.5], [0.5], [0.5], [-0.5]]
      
trainer = Trainer([(8, Sigmoid())])
net = CCNetwork(2, 1, Sigmoid())

net = trainer.train_network(xor_inputs, xor_targets, net)

for x in xor_inputs:
    net.full_forward_pass(x)

output_values = [node.value for node in net.output_nodes]

Small Magnitude Weight Pruning

from pycascor.network import PrunableCCNetwork
from pycascor.activation_functions import Sigmoid
from pycascor.trainer import PruneTrainer

xor_inputs = [[0, 0], [0, 1], [1, 0], [1, 1]]
xor_targets = [[-0.5], [0.5], [0.5], [-0.5]]
      
trainer = PruneTrainer([(8, Sigmoid())])
net = PrunableCCNetwork(2, 1, Sigmoid())

net = trainer.train_network(xor_inputs, xor_targets, net)

for x in xor_inputs:
    net.full_forward_pass(x)

output_values = [node.value for node in net.output_nodes]

Optimal Brain Damage Pruning

from pycascor.network import PrunableCCNetwork
from pycascor.activation_functions import Sigmoid
from pycascor.trainer import OptimalBrainDamageTrainer

xor_inputs = [[0, 0], [0, 1], [1, 0], [1, 1]]
xor_targets = [[-0.5], [0.5], [0.5], [-0.5]]
      
trainer = OptimalBrainDamageTrainer([(8, Sigmoid())])
net = PrunableCCNetwork(2, 1, Sigmoid())

net = trainer.train_network(xor_inputs, xor_targets, net)

for x in xor_inputs:
    net.full_forward_pass(x)

output_values = [node.value for node in net.output_nodes]

Simulator Examples

In addition to the library code pycascor, you will find an examples directory this repository. This contains a simulator script simulation.py which is able to run multiple trials of various datasets across the 3 versions of the network training algorithms. This includes some data which comes from the Proben1 benchmarking dataset project originally compiled by Lutz Prechelt. A copy of the data is linked as a submodule.

. venv/bin/activate
pip install .
python examples/simulation.py <example> <number-of-trials>

Example options: spirals, cancer, glass, xor

Various output files will be written to a result folder for your analysis.

spirals trains the networks on a subset of the two-spirals classification problem. The resulting networks are then validated and tested on the remaining points

cancer trains the networks on the "Wisconsin breast cancer database" from the UCI machine learning dataset included in the Proben1 repository.

glass trains the networks on the "glass" from the UCI machine learning dataset included in the Proben1 repository.

xor trains and tests the networks on the XOR dataset.

Support

Feel free to create an Issue or MR if you find a bug or want to add a contribution.

Contributing

Running Tests

To run the tests you can run pytest from the project root directory.

Authors and acknowledgment

This implementation is based on the code of Scott Fahlman and Thomas Shultz. Thanks to Dr. Britt Anderson, Hanbin Go, James Houle, Julia Schirmeister, Vivian DiBerardino, and Ciaran Neely for contributing to my understanding of the algorithm and inspiring this project.

License

This code and related materials is made available under the MIT License.

Project status

This is a project that I created for a directed studies course at while studying at University of Waterloo.