In recent years, considerable effort has been directed toward the identification of neural structures and mechanisms responsible for rewarding adaptive behaviors.
The system overview
The simulator -the conceptual framework of a Liquid State Machine (LSM) facilitates the analysis of the real-time computing capability of neural microcircuit Readout – the states are mapped by the memory-less readout functions to the outputs. Used when the goal is for the whole network to learn a mission.
In recent years, considerable effort has been directed toward the identification of neural structures and mechanisms responsible for rewarding adaptive behaviors. The behavioral concept of reward is believed to be correlated with neural processes that change the functionality of neurons, based on past performance of the system. These neurons are reported to be activated in response to surprising events such as new stimuli, unexpected primary rewards, and arbitrary stimuli that are associated with primary rewards, thus reporting an error in the prediction of the stimulus.
In this project we have researched a mechanism possibly responsible for rewarding adaptive behaviors – “”changing until convergence”” mechanism, characterized by random changes in neural connections followed by halting at “”successful”” states, therefore imitating the process of reward in biology. We will demonstrate this mechanism by applying 2 different models that affect the dispersion of neural correlations: STDP and Dopamine models.
Part 1- Learning by dispersion – dopamine induced network
The involvement of dopamine in learning in behavioral level process is believed to be large. Dopamine disperses correlations between individual neural activities while preserving global distribution of correlations at the network level. This is demonstrated in the following figure:
As the correlation (dependency) between the neurons increases, the dispersion of it, caused by dopamine is greater. It can be seen that it is much larger than the dispersion caused by a control (pink) solution. Dopamine has a wide range of random affects on synaptic plasticity and strength. It is believed to enhance changes in the network connectivity. Therefore, in the first part of our study we will examine the affects of dopamine on the network: We will examine the network’s response to external stimuli and learning ability with a readout present (see above) and without (when neurons are chosen and taught separately).
- To discover what are the optimal parameters of dopamine for the network’s
- How close can we get to desired results
- Explore the cause of the success
Part 2- Learning by dispersion – STDP synapses
Synchronization of neural activity is fundamental for many functions of the brain.
Spike-timing dependent plasticity (STDP) enhances synchronization in neural circuits. STDP based on Hebbian learning, has learning properties, and long-term bistable characteristics.
Therefore, in the second part of our study we will examine the affects of using these STDP synapses on the network
- Accomplishing a pre-defined mission on one or more chosen neurons
- Analyzing the received results
Synaptic distribution in STDP inducement Synaptic distribution at Dopamine synaptic model
before, during and after the mission performance before, during and after the mission performance
It appears that STDP dispersion resembles the dopamine dispersion, whereas synapses haven’t converged to the edges and dispersed throughout all