MiniTensorflow 0.0.9

Ease working with neural networks

This project aims to ease the workflow of working with neural networks, I will be updating the code as I learn. I am currently pursuing bachelor's in data science and I am interested in Machine Learning and Statistics
Github Profile
Medium Profile
Website
Github Repository of the source code
PyPI link of the project

This code contains two classes, one for the Layer and one for the Network

Layer Class

This class creates a Layer of the neural network which can be used for further calculations

def __init__(self,inputs:np.array,n,activation = 'sigmoid',weights=None,bias=None,random_state=123,name=None) -> None:

The constructor of the Layer class takes the following arguments:

1. name - The name of the layer, defaults to None
2. inputs - The inputs for the layer, shape = (n_x,m)
3. n - The number of neurons you would like to have in the layer
4. weights - The weights for the layer, initialized to random values if not given, shape = (n[l], n[l-1])
5. bias - The bias for the layer, initialized to random values if not given, shape = (n[l],1)
6. activation- The activation function you would like to use, defaults to sigmoid
   Can chose from ['sigmoid','tanh','relu']
   Raises ValueError if the activaion function is not among the specified functions
   Equations of activation functions for reference
   
7. random_state - The numpy seed you would like to use, defaults to 123 Returns: None
   Example
   

# goal - to create a layer of 5 neurons with relu activation function whose name is 'First hidden layer'and initializes random weights
x = np.random.randn(5,5)
layer1 = Layer(name='First hidden layer',inputs=x)

Fit function

def fit(self)->np.array:

Fits the layer according to the activation function given to that layer
For the process of fitting, it first calculates Z according to the equation
$Z^{[l]} = W^{[l]} \times X^{[l-1]} + b^{[l]}$
Then calculates the activation function by using the formula
$a^{[l]} = g^{[l]}(Z^{[l]})$ Returns: np.array - Numpy array of the outputs of the activation function applied to the Z function Example

# goal - you want to fit the layer to the activation function 
outputs = layer1.fit()

Derivative function

def derivative(self)->np.array:

Calculates the derivative of the acivation function accordingly
Returns: np.array - Numpy array containing the derivative of the activation function accordingly Example

# goal - You want to calculate the derivative of the activation function of the layer
derivatives = layer1.derivative()

Network Class

This class creates a neural network of the layers list passed to it

def __init__(self,layers:list) -> None:

The constructor of the Network class takes the following arguments:

1. layers - The list of layer objects Raises TypeError if any element in the layers list is not a Layer instance Example
   

# goal - To create a network with the following structure 
# Input layer - 2 neurons 
# First Hidden layer - 6 neurons with sigmoid activation function 
# Second Hidden Layer - 6 neurons with tanh activation function 
# Output Layer - 1 neuron with sigmoid activation function
X = np.random.randn(2,400)
layer1 = Layer('First hidden layer',n=6,inputs=X,activation='sigmoid')
layer2 = Layer('Second Hidden layer',n=6,activation='tanh',inputs=layer1.fit())
layer3 = Layer('Output layer',n=1,inputs=layer2.fit(),activation='sigmoid')
nn = Network([layer1,layer2,layer3])

Fit function

def fit(self)->np.array:

Propagates through the network and calcuates the output of the final layer i.e the output of the network
Returns: np.array - The numpy array containing the output of the network Example

# Goal- to propagate and find out the outputs of the network
outputs = nn.fit()

Summary function

def summary(self)->pd.DataFrame:

Returns the summary of the network which is a pandas dataframe containing the following columns:

1. Layer Name: The name of the layer
2. Weights: The shape of the weights
3. Bias: The shape of the bias
4. Total Parameters: Total number of parameters initialized in the layer Example
   

#Goal - To print the summary of the network
summary = nn.summary()
print(summary)

Output

Layer Name	Weights	Bias	Total parameters
First hidden layer	(6,2)	(6,1)	18
Second hidden layer	(6,6)	(6,1)	42
Output Layer	(1,6)	(1,1)	7

Attributes of a network

1. params - The list containing total number of parameters initialized at each layer of the network