Biological Realistic Neural Network Simulation

Abstract
While creating large neural network models, finding network behaviors similar to those found in biological experiments, assuming reproducing biological brain computational power as an ultimate goal, could mark milestone on a way of building neural network simulations of a greater power.
Neuron selective adaptation, which we describe further, was chosen as main target phenomenon for this study. Though a neuron selective adaptation was studied biologically [1], generic artificial networks, which were not especially designed to exhibit this specific behavior, are known. Our assumption was that once artificial network that is biologically “realistic enough” is taken, it would exhibit such a behavior.
 
Selective Adaptation
General idea on selective adaptation can be found in the introduction chapter of [1]. In the experiment performed in [1]: two input sites stimulated with two different frequencies: rare (1/50 Hz) and frequent (1/5 Hz). If only frequent or rare input applied – output undergoes certain depression. If both applied simultaneously, while network response to frequent input undergoes depression, its response to rare input is amplified.
 
Network Response to Continuous Frequency
Another phenomenon described in [1] is change in network response to continuously applied input frequency. For higher frequency, network response drops with time, while for lower frequency it remains unchanged. This behavior can be observed in (copied from [1]). We’ll refer to this phenomenon as “frequency adaptation” (opposed to “selective frequency adaptation”).

Figure 1 – copied from [1]

Project overview
During our research we try different neuron and synapse models in search for “realistic enough” neural network model. “Straight forward” approach of building large LIF with CSIM simulation failed to reproduce selective adaptation behavior. More graduate approach was taken in order to explore related phenomena and finally build artificial network that would reproduce selective adaptation behavior.
These are the stages of our advancement:
1) Single synapse depression properties confirmation
2) Minimal 4 neuron network exhibiting selective adaptation behavior
3) Large scale 10,000 neural network exhibiting frequency adaptation

Single Synapse Depression
Networks with few neurons were used to assure single neuron biologically compatible behavior (we showed neuron’s synapse depression property, that is believed to be one of key mechanisms of selective adaptation). We checked post synapse neuron firing frequency patterns while changing synapse weight and depression parameters.

Minimal Network
Basing on a hypothesis presented in [1], claiming that inhibitory sub networks present a common network resource (independent on stimuli input sites) while excitatory neurons form independent pathways for each stimuli input site. Trying to support this hypothesis we create miniature 4 neurons network with predefined neuron connections. Connections between neurons are made in a special way creating such an “inhibitory common resource” and separate “excitatory pathways” (see Methods). Original idea for creation of minimal network was proposed by the author.

Large Scale Network Frequency Adaptation
Another step towards reproducing selective adaptation is showing frequency adaptation on a large neural network. Large networks, up to 10,000 neurons were used and effects of different parameters upon frequency adaptation behavior were explored.
Original source code, by Izhikevich [6] was extended to support working from MATLAB environment and supporting different kind of input channels and recorders. For more details see Appendix A,

Izhikevich Neuron Model
Different neuron and synapse computational models exist today [2]. In our study we mainly use Leak Integrate and Fire (LIF) and highly promising Izhikevich neuron model [3] [4]. We show differences in these models behaviors and while we succeed in reproducing some of behaviors with one model we fail to do it with another.
Izhikevich model exhibit several biologically realistic phenomena such as brain wave like activities and spontaneous pathways creation resembling pathways [4] mentioned by Danny Eytan, Naama Brenner and Shimon Marom in their research [1].

Tools
MATLAB
MATLAB was used as an environment for executing and analysing experiments data.
CSIM
CSIM framework (http://www.lsm.tugraz.at/) have been used for simulating small scale neural networks in “Single Synapse Depression” and “Minimal Network” experiments.
SPNet++
SPNet++ framework was developed during the project based on original code by E.M. Izhikevich  A special effort was made to implement convinient MATLAB integration to allow easy experiments managment and results analysis.

Results Highlights
Single Synapse Depression: phenomena was displayed using CSIM simulation.

REFERENCES

1. Danny Eytan, Naama Brenner and Shimon Marom: Selective Adaptation in Networks of Cortical Neurons J. of Neuroscience 23(28):9349-9356, 2003
2. Izhikevich E.M. (2004), Which Model to Use for Cortical Spiking Neurons? IEEE Transactions on Neural Networks, (special issue on temporal coding), in press.
3. Izhikevich E.M. (2003), Simple Model of Spiking Neurons. IEEE Transactions on Neural Networks (2003) 14:1569- 1572.
4. Izhikevich E.M. (2004), Simple Model of Spiking Network. IEEE Transactions on Neural Networks, (special issue on temporal coding), in press.
5. CSIM simulation framework, http://www.lsm.tugraz.at/.
6. Izhikevich E.M. (2004), spnet simulation, http://www.nsi.edu/users/izhikevich/publications/spnet.cpp
7. Tsodyks, M., Pawelzik, K., and Markram, H. (1998) Neural networks with dynamic synapses. Neural Computation 10, 821-835.
8. W. Maass, T. Natschläger, and H. Markram. Computational models for generic cortical microcircuits. In J. Feng, editor, Computational Neuroscience: A Comprehensive Approach. CRC-Press, 2002. to appear.