Neural Network package
Project description
Neural Network Framework
A project done at the university JMU Würzburg by fachter together with marja-w and mamadfrhi
The package is being used by the remaining implementation of that project in another repository called gesture-detection-neural-network and further usage of this package can be seen in the implementation
Project by Felix Achter, Marja Wahl, Mohammad Farrahi
This repository is designated to creating, training, and using a neural network. There are five main tasks you can tackle:
- Data Preprocessing: get the data ready for being trained on
- Creating the Network: create your network by using information of your training data
- Training the Network: train the resulting network by optionally choosing number of iterations, learning rate, and optionally optimizer
- Evaluation: evaluate the trained network with metrics like accuracy and F1 score
- Predicting: use the final network for predicting on unseen data
Data Preprocessing
Scaling
You don't have to scale your data in advance.
Creating a new NeuralNetwork instance, the Standard Scaler is set as the default scaler.
You can read more about creating a NeuralNetwork instance in section "Creating the Network".
The feature_scaling.py file provides two classes for scaling your data: the StandardScaler and the NormalScaler class. You can scale your data, before instantiating a neural network, by creating an instance of either class, fitting the scaler to your data, and then transform your data. The data can be transformed back by using the inverse_transform() function.
from netneural.network.feature_scaling import StandardScaler
standard_scaler = StandardScaler()
standard_scaler.fit(X) # X being the training data
X_scaled = standard_scaler.transform(X)
X = standard_scaler.inverse_transform(X_scaled) # returns original X
Don't forget to set the right scaler, if you did not use the default StandardScaler, when creating a NeuralNetwork instance:
from netneural.network.feature_scaling import NormalScaler
normal_scaler = NormalScaler()
normal_scaler.fit(X) # X being the training data
X_scaled = normal_scaler.transform(X)
nn = NeuralNetwork(shape, scaler='normal')
Encoding
You don't have to encode your target values in advance.
If no encoder is handed over when instantiating a NeuralNetwork, the data is encoded, if necessary.
You can read more about creating a NeuralNetwork instance in section "Creating the Network".
The one_hot_encoder.py can be used for creating your own OneHotEncoder instance and encoding multiclass labels into one hot encoded label lists.
from netneural.network.one_hot_encoder import OneHotEncoder
encoder = OneHotEncoder()
y_one_hot = encoder.encode(y) # y being the target values with multiple labels
unique_labels = encoder.unique_labels # get unique labels found by encoder
Don't forget to pass the created encoder to a NeuralNetwork instance, if you encode the labels in advance:
nn = NeuralNetwork(shape, encoder=encoder)
Training Test Split
In order to properly analyse your network's training, you will need to split the data into a training and a test set.
You can do that by the provided function train_test_split()
in data_loader.py:
# take only the first four outputs, because no validation set is produced
X_train, y_train, X_test, y_test = train_test_split(X, y, train_per=0.8, test_per=0.2, randomized=True)[:4]
You can also divide your data into training, test, and validation set:
X_train, y_train, X_test, y_test, X_val, y_val = train_test_split(X, y, train_per=0.8, test_per=0.1, valid_per=0.1)
Principal Component Analysis (PCA)
Additionally, you can perform PCA using methods provided in pca.py.
If you do not know anything about PCA you can simply reduce your feature count by specifying how much variance of data you want to cover with the remaining features:
from netneural.pca.pca import PCA
pca = PCA()
X_pca = pca.pca(X, var_per=0.99)
If you want to directly specify the number of desired features, you can also do so:
X_pca_10 = pca.pca(X, number_pcs=10)
You can also specify both, where number_pcs
would act like a maximum for the returned constructed features:
X_pca = pca.pca(X, var_per=0.99, number_pcs=10) # returns 10 principal components, if more are needed for covering 99% of variance
You can also directly work with eigenvectors and eigenvalues:
eigenvectors, eigenvalues = pca.get_eigenvectors_and_values(X)
# gets matrix with 'rank' of every attribute in every principal component
ordered_attributes_matrix = pca.analyze_eigenvectors(eigenvectors, attributes) # attributes is list of original attributes
# gets a list of the original attributes, ordered by its position in each eigenvector
ordered_attributes = pca.compute_order_attributes(ordered_attributes_matrix)
# returns the number of principal components needed for covering the input percentage
n_pc = pca.explained_variance(eigenvalues, 0.99, plots=True) # if plots is True, plots variance covered per n principal components
X_pca = pca.get_n_dimensions(X, n_pc, eigenvectors)
Creating the Network
For creating the network, you need to create a new instance of the NeuralNetwork class, provided in nn.py. You have to set several parameters when instantiating.
- Activation Function: you can choose between sigmoid, tanh, and relu activation
- Architecture: you will need to provide the number of neurons per layer
- Weigths: you don't have to specifiy weights, but you can do so by passing a list of lists, each containing weights for each layer
from netneural.network.nn import NeuralNetwork
shape = (input_features, 2, 2, output_classes) # variables store the number of input features/output classes
nn = NeuralNetwork(shape, activation_function='sigmoid') # uses randomly assigned weights
If you want to train a neural network for regression, you will have to set the regression
parameter to true:
nn = NeuralNetwork(shape, regression=True)
Training the Network
Training the network requires several hyperparameters you have to set. The train()
function of the NeuralNetwork class
takes severable variables in order to define them:
- Iterations: choose how many epochs the neural network will train for
- Learning Rate: decide on the learning rate for training
- Test Data: you have the choice of handing over a test data set in order to receive feedback about the evaluation of the network on this data set
- Optimizer: you can choose between the default gradient descent or the Adam optimizer
- Cost function: the network decides on the cost function on its own, whether your last layer has one or multiple nodes, it uses the Mean Squared Error function or Cross Entropy (with Softmax)
- Batch Size: if a batch size is set, the model trains batches of the training data of this size every iteration, it updates its weights after training each batch, and it sees each batch during one iteration. Each iteration, the training examples are randomly assigned to a batch. If the batch size is not given, the model will normally train on all the training data per one iteration
- Plotting: if the
plots
variable is set to True, the function will plot the error, accuracy, and F1 score on training and eventually test data set after training - Encode: if you did not encode your data yet (multiclass classification) and did not hand over an encoder during the creation of the neural network, you need to set this parameter to true, if your multiclass labels are not encoded yet
The function returns the training history on either the training data set or the test data set, which can later be used for analyzing and visualizing.
f1_history, acc_history, err_history = nn.train(X, y, 100, 0.1) # trains with default settings (optimizer=gradient descent, no test data, no plots)
f1_history, acc_history, err_history = nn.train(X, y, 100, 0.1, X_test, y_test, optimizer='adam', batch_size=500, plots=True) # trains while evaluating on test set, uses Adam optimizer, uses batches, plots results
For the format of data sets applies:
- the data matrix (X) should be formatted data points x features
- the target values (y) should be formatted class x data points (one row for binary classification)
When training a regression neural network, the only metric output afterwards is the history of the Mean Squared Error (MSE):
error_history = nn.train(X, y, 100, 0.1)
Training Sessions
The functions provided in the session_util.py can be used for saving and loading a trained network.
Along with the network the function save_session()
also takes other parameters, like the unique labels, PCA object,
training history, training data, iterations, and learning rate, in order to easily reenact the documented session.
from netneural.session.nn_session_util import save_session
learning_rate = 0.1
iterations = 100
f1_history, acc_history, err_history = nn.train(X, y, iterations, learning_rate) # we train a neural network
# save_session stores a .pkl file with a name consisting of current time and the final f1 score
save_session(nn, nn.encoder.unique_labels, pca, f1_history[-1])
After the file is stored, it can be used to again load the neural network:
from netneural.session.nn_session_util import load_from_config
config_file = "config_90_f1_2022-03-25 14-56-20.json" # path to the config file
nn, pca, f1_history, learning_rate, X_train, y_train, X_test, y_test = load_session_from_config(config_file)
You can also use these functions for a regression neural network and if you did not use PCA:
save_session(nn, learning_rate, X, y, X_test, y_test, iterations, err_history) # saves file in 'config_file'
# returns None for values that weren't set, like pca and f1_history
nn, pca = load_from_config(config_file)
Evaluation
Metrics
For evaluating a trained model intrinsic metrics can be calculated using the methods provided in metrics.py.
These include accuracy and the F1 score, which essentially compare the predictions of the model h
to the ground truth y
.
The output differs, depending if binary or multiclass classification is demanded.
from netneural.network.metrics import get_accuracy, get_f1_score
accuracy_per_class, accuracy_mc = get_accuracy(h_multiclass, y_multiclass) # additionally returns accuracy per class
accuracy = get_accuracy(h, y)
f1_score_per_class, f1_score_mc = get_f1_score(h_multiclass, y_multiclass)
f1_score = get_f1_score(h, y)
For the format of the input values it is required that the rows stand for the number of classes.
The error can be calculated using the get_error()
function of the NeuralNetwork class:
errors = nn.get_error(h,y) # returns error for every data point in a list
error = errors.mean() # final error
When using a regression neural network, you don't have to take the mean over the error. We use the MSE, which will already output the mean over the errors:
error = nn.get_error(h,y) # for nn with regression=True
Plotting
For making plotting a little bit easier, two functions are introduced in the plot_lib.py script. One that plots a line and one function that creates a scatter plot.
from netneural.network.plot_lib import plot_line, plot_scatter
plot_line(list_to_plot, x_label, y_label, title) # plots a line
plot_scatter(x_data, y_data, colors, x_label, y_label, title): # creates scatter plot
Predicting
Finally, we can use the NeuralNetwork instance for predicting values. This can be done using the predict()
provided in
the NeuralNetwork class:
predictions = nn.predict(input_values)
The input values of this function need to be a numpy array
, also if it is only one value.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for netneural-0.0.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 157d75aa3dce25da2f4800f0001e327004086553ad382203551b0e612547bdfa |
|
MD5 | 5f5fd3f2fcdab589bd5686f83ff2572b |
|
BLAKE2b-256 | 01782668cddd23aef4655f9822964f146626ae2f651b3f58b5cd7f2327906d00 |