Skip to main content

growingnn - Dynamic Neural Network Architecture Learning

The growingnn project introduces an innovative algorithm for data-driven neural network model construction. This algorithm comprises two fundamental components: the first component focuses on weight adjustment, while the second component acts as an orchestrator, launching a guided procedure to dynamically change the network architecture. This architectural modification occurs at regular intervals, specifically every $K$ epochs, and is driven by the outcome of a Monte Carlo tree search. The algorithm's core, outlined in the accompanying research paper, leverages the principles of Stochastic Gradient Descent (SGD) without relying on advanced tools commonly used in neural network training.

This repository is the basic version of growingnn, implemented with NumPy only. A newer PyTorch-based version is available at growingNN_pytorch.

Repozytorium GitHub

Click the link above to navigate to the GitHub repository, where you can find the source code, issues, and other project details.

Dokumentacja

This link will take you to the project documentation. Here you'll find instructions, usage information, and other resources helpful for using the project.

PyTorch version

Newer implementation of growingnn built with PyTorch.

Algorithm Overview

Weight Adjustment Component

The first component of the algorithm is dedicated to weight adjustment. It operates within the framework of Stochastic Gradient Descent (SGD), a foundational optimization algorithm for training neural networks. The simplicity of this approach makes it suitable for educational settings, emphasizing fundamental machine learning principles.

Orchestrator and Network Architecture Modification

The second component, the orchestrator, plays a crucial role in initiating a procedure to dynamically change the network architecture. This change occurs systematically at predefined intervals, specifically every $K$ epochs. The decision-making process for architectural changes is facilitated by a guided Monte Carlo tree search. This sophisticated mechanism ensures that architectural modifications are well-informed and contribute to the overall improvement of the neural network model.

Implementation Details

Model Structure

The model is the main structure that stores layers as nodes in a directed graph. It operates based on layer identifiers, treating each layer as an independent structure that contains information about incoming and outgoing connections. The default starting structure is a simple graph with an input and output layer connected by a single connection. In each generation, the algorithm has the flexibility to add new layers or remove existing ones. As the structure grows, each layer gains more incoming and outgoing connections.

Propagation Phase

During the propagation phase, each layer waits until it receives signals from all input layers. Once these signals are received, they are averaged, processed, and propagated through all outgoing connections. This iterative process allows the neural network to dynamically adapt its architecture based on the evolving data and training requirements.

Results and Testing

The proposed algorithm has undergone rigorous testing, particularly in visual pattern classification problems. The results have consistently demonstrated high levels of satisfaction, showcasing the efficacy of the dynamic architecture learning approach in enhancing model performance.

x_train, x_test, y_train, y_test, labels = data_reader.read_mnist_data(mnist_path, 0.9)
gnn.trainer.train(
    x_train = x_train, 
    y_train = y_train, 
    x_test = x_test,
    y_test = y_test,
    labels = labels,
    input_paths = 1,
    path = "./result", 
    model_name = "GNN_model",
    epochs = 10, 
    generations = 10,
    input_size = 28 * 28, 
    hidden_size = 28 * 28, 
    output_size = 10, 
    input_shape = (28, 28, 1), 
    kernel_size = 3, 
    depth = 2
)

This code trains a simple network on the MNIST dataset

Credits

Szymon Swiderski Agnieszka Jastrzebska

Acknowledgments

The contribution was supported by the National Science Center, grant No. 2024/53/B/ST6/00021.

Disclosure

This is the first beta version of the growingnn package. We are not liable for the accuracy of the program’s output nor actions performed based upon it.

For more in-depth information on the algorithm, its implementation, and testing results, refer to the accompanying research paper. The provided Python source code is a valuable resource for understanding and implementing the presented method. Feel free to explore, contribute, and adapt it for your specific needs.

Metadata

Release files for growingnn 0.4.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for growingnn 0.4.1
File Size Uploaded Provenance Actions
growingnn-0.4.1.tar.gz 87.1 kB Without provenance Details for growingnn-0.4.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for growingnn 0.4.1
File Interpreter ABI Platform Actions
growingnn-0.4.1-py3-none-any.whl Python 3 none any Details for growingnn-0.4.1-py3-none-any.whl

Total release size: 135.9 kB

Release files / growingnn-0.4.1.tar.gz

Download URL growingnn-0.4.1.tar.gz
Size 87.1 kB
Tags Source
SHA-256 checksum
How to use checksums
7fef4e8528d877ed8935ab7e7d726f2b55a2b97eb69d18e019f37248c9e37633
BLAKE2b-256 checksum
How to use checksums
4e4ad8f17eb429b8f516f03db449987e2a18e7c669bdf5e8e50e81ab8b5901e8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.13.14

Release files / growingnn-0.4.1-py3-none-any.whl

Download URL growingnn-0.4.1-py3-none-any.whl
Size 48.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
ab1e6a49bf1baea5ce8f4f49b1c2afd9b954ed6128d8e383cf47fefed725ebe3
BLAKE2b-256 checksum
How to use checksums
9eee2ef5cca1802e8ff72c4903323defccd8d666c8b7d9ca34af7469ea446412
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.13.14

Security for growingnn 0.4.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page