MiniTensorflow 0.2.2

Ease working with neural networks

This project aims to ease the workflow of working with neural networks, I will be updating the code as I learn. I am currently pursuing bachelor's in data science and I am interested in Machine Learning and Statistics
Github Profile
Medium Profile
Website
Github Repository of the source code
PyPI link of the project

This code contains two classes, one for the Layer and one for the Network

Layer Class

This class creates a Layer of the neural network which can be used for further calculations

def __init__(self,inputs:np.array,n,activation = 'sigmoid',weights=None,bias=None,random_state=123,name=None) -> None:

Args:

1. name - The name of the layer, defaults to None
2. inputs - The inputs for the layer, shape = (n_x,m)
3. n - The number of neurons you would like to have in the layer
4. weights - The weights for the layer, initialized to random values if not given, shape = (n[l], n[l-1])
5. bias - The bias for the layer, initialized to random values if not given, shape = (n[l],1)
6. activation- The activation function you would like to use, defaults to sigmoid
   Can chose from ['sigmoid','tanh','relu']
   Raises ValueError if the activaion function is not among the specified functions
   Equations of activation functions for reference
   
7. random_state - The numpy seed you would like to use, defaults to 123 Returns: None
   Example
   

# goal - to create a layer of 5 neurons with relu activation function whose name is 'First hidden layer'and initializes random weights
x = np.random.randn(5,5)
layer1 = Layer(name='First hidden layer',inputs=x)

Fit function

def fit(self)->np.array:

Fits the layer according to the activation function given to that layer
For the process of fitting, it first calculates Z according to the equation

Then calculates the activation function by using the formula

Returns: np.array - Numpy array of the outputs of the activation function applied to the Z function Example

# goal - you want to fit the layer to the activation function 
outputs = layer1.fit()

Derivative function

def derivative(self)->np.array:

Calculates the derivative of the acivation function accordingly
Returns: np.array - Numpy array containing the derivative of the activation function accordingly Example

# goal - You want to calculate the derivative of the activation function of the layer
derivatives = layer1.derivative()

Network Class

This class creates a neural network of the layers list passed to it

def __init__(self,layers:list) -> None:

Args:

1. layers - The list of layer objects Raises TypeError if any element in the layers list is not a Layer instance Example
   

# goal - To create a network with the following structure 
# Input layer - 2 neurons 
# First Hidden layer - 6 neurons with sigmoid activation function 
# Second Hidden Layer - 6 neurons with tanh activation function 
# Output Layer - 1 neuron with sigmoid activation function
X = np.random.randn(2,400)
layer1 = Layer('First hidden layer',n=6,inputs=X,activation='sigmoid')
layer2 = Layer('Second Hidden layer',n=6,activation='tanh',inputs=layer1.fit())
layer3 = Layer('Output layer',n=1,inputs=layer2.fit(),activation='sigmoid')
nn = Network([layer1,layer2,layer3])

Fit function

def fit(self)->np.array:

Propagates through the network and calcuates the output of the final layer i.e the output of the network
Returns: np.array - The numpy array containing the output of the network Example

# Goal- to propagate and find out the outputs of the network
outputs = nn.fit()

Compute cost function

def compute_cost(self,y:np.array,natural_log=True)->float:

Calculates the cost of the network compared to the target

Args:

1. y (np.array): Target values for the network
2. natural_log (bool, optional): Whether you want to use log10 or natural log. Defaults to True.

Returns:
float: The cost of that network

Example:

# Goal - to compute the cost for the network 
cost = nn.compute_cost(y=y)

Train function
Trains the neural network for the specified iterations

def train(self, epochs:int,history=False)->dict:

Args:

1. epochs (int): The number of iterations for which you want to train the network
2. history (bool, optional): If you want the history of gradients at each iterations. Defaults to False.

Returns:
dict: The dictionary containing the gradients of parameters at each iteration

Example:

# Goal - to train the network for 500 epochs and get history at each epoch
nn.train(epochs=500, history=True)

Predict function
Performs predictions on the given values

def predict(self,values:np.array)->np.array:

Args:

1. values (np.array): The values on which you want to predict Returns:
   np.array: The array of predictions

Attributes of a network

1. params - The list containing total number of parameters initialized at each layer of the network
2. Summary
   Returns the summary of the network which is a pandas dataframe containing the following columns:
   

• Layer Name: The name of the layer
• Weights: The shape of the weights
• Bias: The shape of the bias
• Total Parameters: Total number of parameters initialized in the layer Output
  

Layer Name	Weights	Bias	Total parameters
First hidden layer	(6,2)	(6,1)	18
Second hidden layer	(6,6)	(6,1)	42
Output Layer	(1,6)	(1,1)	7