growingnn - Dynamic Neural Network Architecture Learning
The growingnn project introduces an innovative algorithm for data-driven neural network model construction. This algorithm comprises two fundamental components: the first component focuses on weight adjustment, while the second component acts as an orchestrator, launching a guided procedure to dynamically change the network architecture. This architectural modification occurs at regular intervals, specifically every $K$ epochs, and is driven by the outcome of a Monte Carlo tree search. The algorithm's core, outlined in the accompanying research paper, leverages the principles of Stochastic Gradient Descent (SGD) without relying on advanced tools commonly used in neural network training.
This repository is the basic version of growingnn, implemented with NumPy only. A newer PyTorch-based version is available at growingNN_pytorch.
Repozytorium GitHub
Click the link above to navigate to the GitHub repository, where you can find the source code, issues, and other project details.
Dokumentacja
This link will take you to the project documentation. Here you'll find instructions, usage information, and other resources helpful for using the project.
PyTorch version
Newer implementation of growingnn built with PyTorch.
Algorithm Overview
Weight Adjustment Component
The first component of the algorithm is dedicated to weight adjustment. It operates within the framework of Stochastic Gradient Descent (SGD), a foundational optimization algorithm for training neural networks. The simplicity of this approach makes it suitable for educational settings, emphasizing fundamental machine learning principles.
Orchestrator and Network Architecture Modification
The second component, the orchestrator, plays a crucial role in initiating a procedure to dynamically change the network architecture. This change occurs systematically at predefined intervals, specifically every $K$ epochs. The decision-making process for architectural changes is facilitated by a guided Monte Carlo tree search. This sophisticated mechanism ensures that architectural modifications are well-informed and contribute to the overall improvement of the neural network model.
Implementation Details
Model Structure
The model is the main structure that stores layers as nodes in a directed graph. It operates based on layer identifiers, treating each layer as an independent structure that contains information about incoming and outgoing connections. The default starting structure is a simple graph with an input and output layer connected by a single connection. In each generation, the algorithm has the flexibility to add new layers or remove existing ones. As the structure grows, each layer gains more incoming and outgoing connections.
Propagation Phase
During the propagation phase, each layer waits until it receives signals from all input layers. Once these signals are received, they are averaged, processed, and propagated through all outgoing connections. This iterative process allows the neural network to dynamically adapt its architecture based on the evolving data and training requirements.
Results and Testing
The proposed algorithm has undergone rigorous testing, particularly in visual pattern classification problems. The results have consistently demonstrated high levels of satisfaction, showcasing the efficacy of the dynamic architecture learning approach in enhancing model performance.
x_train, x_test, y_train, y_test, labels = data_reader.read_mnist_data(mnist_path, 0.9)
gnn.trainer.train(
x_train = x_train,
y_train = y_train,
x_test = x_test,
y_test = y_test,
labels = labels,
input_paths = 1,
path = "./result",
model_name = "GNN_model",
epochs = 10,
generations = 10,
input_size = 28 * 28,
hidden_size = 28 * 28,
output_size = 10,
input_shape = (28, 28, 1),
kernel_size = 3,
depth = 2
)
This code trains a simple network on the MNIST dataset
Credits
Szymon Swiderski Agnieszka Jastrzebska
Acknowledgments
The contribution was supported by the National Science Center, grant No. 2024/53/B/ST6/00021.
Disclosure
This is the first beta version of the growingnn package. We are not liable for the accuracy of the program’s output nor actions performed based upon it.
For more in-depth information on the algorithm, its implementation, and testing results, refer to the accompanying research paper. The provided Python source code is a valuable resource for understanding and implementing the presented method. Feel free to explore, contribute, and adapt it for your specific needs.
Metadata
Release files for growingnn 0.4.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| growingnn-0.4.1.tar.gz | 87.1 kB | Without provenance | Details for growingnn-0.4.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| growingnn-0.4.1-py3-none-any.whl | Python 3 | none | any | Details for growingnn-0.4.1-py3-none-any.whl |
Total release size: 135.9 kB
Release files / growingnn-0.4.1.tar.gz
| Download URL | growingnn-0.4.1.tar.gz |
|---|---|
| Size | 87.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
7fef4e8528d877ed8935ab7e7d726f2b55a2b97eb69d18e019f37248c9e37633
|
|
BLAKE2b-256 checksum How to use checksums |
4e4ad8f17eb429b8f516f03db449987e2a18e7c669bdf5e8e50e81ab8b5901e8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.13.14
|
Release files / growingnn-0.4.1-py3-none-any.whl
| Download URL | growingnn-0.4.1-py3-none-any.whl |
|---|---|
| Size | 48.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ab1e6a49bf1baea5ce8f4f49b1c2afd9b954ed6128d8e383cf47fefed725ebe3
|
|
BLAKE2b-256 checksum How to use checksums |
9eee2ef5cca1802e8ff72c4903323defccd8d666c8b7d9ca34af7469ea446412
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.13.14
|
Security for growingnn 0.4.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.