Self-Organizing Recurrent Neural Networks
Project description
Self-Organizing Recurrent Neural Networks
SORN is a class of neuro-inspired artificial network build based on plasticity mechanisms in biological brain and mimic neocortical circuits ability of learning and adaptation through neuroplasticity mechanisms.
The network is developed as part of my Master thesis at Universität Osnabrück, Germany. For the ease of maintainance, the notebooks and the use cases are moved to SORN-Notebook
SORN Reservoir and the evolution of synaptic efficacies
Contents
- Self-Organizing Recurrent Neural Networks
- Getting Started
- Simulate and Train
- Integrate with OpenAI gym
- Plotting functions
- Statistics and Analysis functions
- Citation
- Contributions
Installation
pip install sorn
The library is still in alpha stage, so you may also want to install the latest version from the development branch:
pip install git+https://github.com/Saran-nns/sorn
Dependencies
SORN supports Python 3.5+ ONLY. For older Python versions please use the official Python client. To install all optional dependencies,
pip install 'sorn[all]'
For detailed documentation about usage and development, please visit SORN-Documentation
Usage
Update Network configurations
There are two ways to update/configure the network parameters,
- Navigate to home/conda/envs/ENVNAME/Lib/site-packages/sorn
or
if you are unsure about the directory ofsorn
Run
import sorn
sorn.__file__
to find the location of the sorn package
Then, update/edit arguments in configuration.ini
- While instantiating the network using
Simulator
orTrainer
objects, override the defaults by assigning the values to thekwargs
shown below,
kwargs = {'_ne', '_nu', '_network_type_ee', '_network_type_ei', '_network_type_ie', '_lambda_ee','_lambda_ei', '_lambda_ie', '_eta_stdp','_eta_inhib', '_eta_ip', '_te_max', '_ti_max', '_ti_min', '_te_min', '_mu_ip','_sigma_ip'}
Plasticity Phase
The default _ne, _nu
values are overriden by passing them as kwargs inside simulate_sorn
method.
from sorn import Simulator
import numpy as np
# Sample input
num_features = 10
time_steps = 200
inputs = np.random.rand(num_features,time_steps)
# To simulate the network;
matrices_dict, Exc_activity, Inh_activity, Rec_activity, num_active_connections = Simulator.simulate_sorn(inputs = inputs, phase='plasticity', matrices=None, noise = True, time_steps=time_steps, _ne = 200, _nu=num_features)
# To resume the simulation, load the matrices_dict from previous simulation;
matrices_dict, Exc_activity, Inh_activity, Rec_activity, num_active_connections = Simulator.simulate_sorn(inputs = inputs, phase='plasticity', matrices=matrices_dict, noise= True, time_steps=time_steps,_ne = 200, _nu=num_features)
Training phase
from sorn import Trainer
inputs = np.random.rand(num_features,1)
# SORN network is frozen during training phase
matrices_dict, Exc_activity, Inh_activity, Rec_activity, num_active_connections = Trainer.train_sorn(inputs = inputs, phase='Training', matrices=matrices_dict,_nu=num_features, time_steps=1)
To turn off any plasticity mechanisms during simulation or training phase, you can use freeze
argument.
For example to stop intrinsic plasticity during training phase,
matrices_dict, Exc_activity, Inh_activity, Rec_activity, num_active_connections = Simulator.simulate_sorn(inputs = inputs, phase='plasticity', matrices=None, noise = True, time_steps=time_steps, _ne = 200, _nu=num_features, freeze=['ip'])
The other options are,
'stdp'
- Spike Timing Dependent Plasticity
'ss'
- Synaptic Scaling
'sp'
- Structural Plasticity
'istdp'
- Inhibitory Spike Timing Dependent Plasticity
Note: If you pass all above options to freeze
, then the network will behave as Liquid State Machine(LSM)
Network Output Descriptions
matrices_dict
- Dictionary of connection weights ('Wee','Wei','Wie') , Excitatory network activity ('X'), Inhibitory network activities('Y'), Threshold values ('Te','Ti')
Exc_activity
- Collection of Excitatory network activity of entire simulation period
Inh_activity
- Collection of Inhibitory network activity of entire simulation period
Rec_activity
- Collection of Recurrent network activity of entire simulation period
num_active_connections
- List of number of active connections in the Excitatory pool at each time step
Colaboratory Notebook
Sample simulation and training runs with few plotting functions are found at
Usage with OpenAI gym
Cartpole balance problem
With default network parameters.
from sorn import Simulator, Trainer
import gym
# Load the simulated network matrices
# Note that these matrices are obtained after the network achieved convergence under random inputs and noise
with open('simulation_matrices.pkl','rb') as f:
sim_matrices,excit_states,inhib_states,recur_states,num_reservoir_conn = pickle.load(f)
# Training parameters
NUM_EPISODES = 2e6
NUM_PLASTICITY_EPISODES = 20000
env = gym.make('CartPole-v0')
# Policy
def policy(state,w):
"Implementation of softmax policy"
z = state.dot(w)
exp = np.exp(z)
return exp/np.sum(exp)
for EPISODE in range(NUM_EPISODES):
# Environment observation; Input to sorn should be of shape (input_features,time_steps)
state = env.reset()[:, None] # (4,) --> (4,1)
state = np.array(state)
# Play the episode
while True:
state = np.array(state[:,None])
if EPISODE < NUM_PLASTICITY_EPISODE:
# Plasticity phase
sim_matrices, excit_states, inhib_states, recur_states, num_reservoir_conn = Simulator.simulate_sorn(inputs = state, phase ='plasticity', matrices = sim_matrices, time_steps = 1, noise=False)
else:
# Training phase with frozen reservoir connectivity
sim_matrices,excit_states,inhib_states,recur_states,num_reservoir_conn = Trainer.train_sorn(inputs = state, phase = 'training', matrices = sim_matrices, noise= False)
# Feed excit_states as input states to your RL algorithm, below goes for simple policy gradient algorithm
# Sample policy w.r.t excitatory states and take action in the environment
probs = policy(np.asarray(excit_states),output_layer_weights))
action = np.random.choice(action_space,probs)
state,reward,done,_ = env.step(action)
if done:
break
# YOUR CODE HERE
# COMPUTE GRADIENTS BASED ON YOUR OBJECTIVE FUNCTION
# OPTIMIZE `output_layer_weights` BASED ON YOUR OPTIMIZATION METHOD
There are several neural data analysis and visualization methods inbuilt with sorn
package. Sample call for few plotting and statistical methods are shown below;
Plotting functions
from sorn import Plotter
# Plot weight distribution in the network
Plotter.weight_distribution(weights= matrices_dict['Wee'], bin_size = 5, savefig = False)
# Plot Spike train of all neurons in the network
Plotter.scatter_plot(spike_train = np.asarray(Exc_activity), savefig=False)
Plotter.raster_plot(spike_train = np.asarray(Exc_activity), savefig=False)
Statistics and Analysis functions
from sorn import Statistics
#t-lagged auto correlation between neural activity
Statistics.autocorr(firing_rates = [1,1,5,6,3,7],t= 2)
# Fano factor: To verify poissonian process in spike generation of neuron 10
Statistics.fanofactor(spike_train= np.asarray(Exc_activity),neuron = 10,window_size = 10)
# Measure the uncertainty about the origin of spike from the network using entropy
Statistics.spike_source_entropy(spike_train= np.asarray(Exc_activity), num_neurons=200)
Citation
Package
@software{saranraj_nambusubramaniyan_2020_4184103,
author = {Saranraj Nambusubramaniyan},
title = {Saran-nns/sorn: Stable alpha release},
month = nov,
year = 2020,
publisher = {Zenodo},
version = {v0.3.1},
doi = {10.5281/zenodo.4184103},
url = {https://doi.org/10.5281/zenodo.4184103}
}
Thesis
Saranraj Nambusubramaniyan(2019): Prospects of Biologically Plausible Artificial Brain Circuits Solving General Intelligence Tasks at the Imminence of Chaos DOI: 10.13140/RG.2.2.25393.81762
Contributions
I am welcoming contributions. If you wish to contribute, please create a branch with a pull request and the changes can be discussed there. If you find a bug in the code or errors in the documentation, please open a new issue in the Github repository and report the bug or the error. Please provide sufficient information for the bug to be reproduced.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.