#] #] ********************* #] "$d_Refs"'Mathematics/9_Spiking Neural Network theory.txt' # www.BillHowell.ca 12Feb2024, converted from LibreWriter version, images as separate files # view in text editor, using constant-width font (eg courier), tabWidth = 3 #48************************************************48 #24************************24 # Table of Contents, generate with : # $ grep "^#]" "$d_Refs"'Mathematics/9_Spiking Neural Network theory.txt' | sed "s|^#\]| |" # ********************* "$d_web"'References/Mathematics/9_Spiking Neural Network theory.txt' Spiking Neural Network Theory +-----+ SNN models Liquid state machine Leaky integrate-and-fire model Eugene M. Izhikevich Nov2003 "Simple Model of Spiking Neurons" +-----+ WCCI2020 NN-20867 r Chevtchenko, Ludermir - Learning from Sparse and Delayed Rewards with a Multilayer Spiking Neural Network Lateral inhibition : k-WTA not WTA SNN: differentiation surrogates #24************************24 # Setup, ToDos, show all images : $ bash "$d_bin"'images/imageAll SNN theory.sh' # 24************************24 #] Spiking Neural Network Theory www.BillHowell.ca 11Feb2020 initial # 08********08 #] +-----+ #] SNN models # 08********08 #] Liquid state machine https://en.wikipedia.org/wiki/Liquid_state_machine A liquid state machine (LSM) is a particular kind of spiking neural network. An LSM consists of a large collection of units (called nodes, or neurons). Each node receives time varying input from external sources (the inputs) as well as from other nodes. Nodes are randomly connected to each other. The recurrent nature of the connections turns the time varying input into a spatio-temporal pattern of activations in the network nodes. The spatio-temporal patterns of activation are read out by linear discriminant units. # 08********08 #] Leaky integrate-and-fire model from : WCCI2020 NN-20867 r Chevtchenko, Ludermir - Learning from Sparse and Delayed Rewards with a Multilayer Spiking Neural Network [33] L. Lapicque, “Recherches quantitatives sur lexcitation electrique des nerfs traite comme une polarization,” J. Physiol. Pathol. Gen., vol. 9, pp. 620–635, 1907. >> predates Hebb by a few decades!! +-----+ Wulfram Gerstner, Werner Kistler 2002 "Spiking neuon models : Single neurons, populations, plasticity" Cambridge University Press, 4th printing 2008, 480pp ISBN 978-0-521-89079-3 paperback www.cambridge.org/9780521823555 Detailed conductance-based neuron models can reproduce electrophysiological measurements to a high degree of accuracy, but because of their instrinsic complexity these models are difficult to analyze. For this reason, simple phenomenological spiking neuron models are highly popular for studies of neural coding, memory, and network dynamics. In this chapter we discuss formal threshold models of neuronal firing. Spikes are generated whenever the membrane potential u crosses some threshold v from below. The moment of threshold crossing defines the firing time t(f) : (4.1) t(f) : u(t) = v and d[dt, t=t(f); u(t)] > 0 Since spikes are sterotped events they are fully characterized by their firing time. ...Derived from electrical RC circuit : (4.2) I(t) = u(t)/R + C*d[dt; u(t)] We multiply Eq. (4.2) by R and introduce the time constant τ_m = R*C of the "leaky integrator". This yields the standard form : (4.3) τ_m*d[dt; u(t)] = -u(t) + R*I(t) We refer to u as the membrane potential and to τ_m as the membrane time constant of euron. In integrate-and-fire models the form of an action potential is not described explicitly. Spikes are formal events characterized by a "firing time" t(f). The firing time t(f) is defiby a threshold criterion : (4.4) t(f) : u(t(f)) = ν Immediately after t(f) the potential is reset to a new value ur < ν : (4.5) lim[t -> t(f), t > t(f); u(t)] = uτ For t > t(f) the dynamics is again given by (4.) until the next threshold crossing occurs. The combination of leaky integration and reset defines the basic integrate-and-fire model (Stein 1967b). We note that, since the membrane potential is never above threshold, the threshold condition of Eq (4.1) reduces to the criterion in Eq. (4.4), i.e., the condition on the slope d[dt; u(t)] can be dropped. In its general version, the leaky integrate-and-fire nron may also incorporate an absolute refractory period, in which case we proceed as follows. If u reaches the threshold at time t=(f), we interrupt the dynamics (4.3) dring an absolute refractory period t(f) + Δabs with the new condition ur. +--+ 16Jun2020 NCAA-D-20-01176 Zhang, Li, Niu, Gao - Event-Driven Intrinsic Plasticity for Spiking Convolutional Neural Networks ∆Vmem(t(k+1)) = 1/τ_m *[Vreset - Vmem (t(k)) + Rm*I(t(k+1))*∆t Vmem(t(k+1)) ← V mem (t(k)) + ∆Vmem(t(k+1)) +--+ 27Feb2021 IJCNN20210506 Zhang, Li - Spiking Neural Networks with Laterally-Inhibited Self-Recurrent Units.pdf page 2 The neuronal membrane voltage u_p(t) of postsynaptic neuron p at time t is given by : (1) τ_m * d[dt: u_p(t)] = -u_p(t) + R*Ip(t) where R and τ_m are the effective leaky resistance and time constant of the membrane, I_p(t) the integrated input current. The neuron p is driven by the input current which is the weighted summation of postsynaptic current (PSC) from presynaptic neuron with the following general form: (2) I_p(t) = sum{over all q: w_pq * a_q(t) } where w_pq is the synaptic weight from presynaptic neuron q to postsynaptic neuron p, and a_q(t) the PSC induced by the spikes from neuron q. The postsynaptic current (PSC) a_q(t) is converted from the presynaptic spikes through a synaptic model. We adopt the first-order synaptic model [28] which is defined as (3) τ_s * d[dt : a_q(t)] = -a_q(t) + s_q(t) where τ_s is the synaptic time constant, s_q(t) the spiking events of presynaptic neuron q. s_q(t) can be expressed as (4) s_q(t) = sum{over all t_qf: δ(t - t_qf) } where δ is the Dirac delta function and t_qf denotes the firing time of presynaptic neuron q. During the simulation, we use the fixed-step first-order Euler method to discretize continuous membrane voltage updates into discrete time steps. Since the ratio of R and τ_m can be absorbed into the synaptic weights, (1) can be converted to (5) u_p[t] = θ_m * u_p[t - 1] * (1 - s_p[t - 1]) + sum{over all q: w_pq * a_q(t) } where θ_m = 1 - 1/ τ_m, and the 1 - s_p[t - 1] term reflects the effect of firing-and-resetting mechanism. The spiking neuron generates an output spike when u_p[t] reaches the predetermined threshold V_th and reset the u_p[t] to the rest potential which is 0 in this work. For the inhibitory neuron, by introducing the excitatory recurrent connection, the expression of the neuron model changes from (5) to: (6) u_i[t] = θ_m * u_i[t - 1] * (1 - s i [t - 1]) + I_i [t] + w_e*a_e[t - 1] where w e is the fixed weight of the excitatory connection and a_e[t - 1] the PSC of the corresponding excitatory neuron. Similarly, the membrane potential of the excitatory neuron can be concluded as (7) u_e[t] = θ_m * u_e[t - 1] * (1 - s_e[t - 1]) + sum{over q: w_eq * a_q[t]} + w_s * a_e[t - 1] + w_i * a_i[t - 1] where w_s is the weight of self-recurrent connection, w[i] the fixed weight of the inhibitory connection, and a_i[t - 1] the PSC of the corresponding inhibitory neuron. # 08********08 #] Eugene M. Izhikevich Nov2003 "Simple Model of Spiking Neurons" IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 14, NO. 6, NOVEMBER 2003, p1569-1572 https://www.izhikevich.org/publications/spikes.pdf Bifurcation methodologies [8] enable us to reduce many biophysically accurate Hodgkin–Huxley- type neuronal models to a two-dimensional (2-D) system of ordinary differential equations of the form (1) v' = 0.04*v^2 + 5*v + 140 -􀀀 u + I (2) u' = a*(b*v - u) with the auxiliary after-spike resetting (3) if v ≥ 30 mV then v <- c u <- u + d Here, v and u are dimensionless variables, and a, b, c, and d are dimensionless parameters, and ' = d/dt, where t is the time. The variable v represents the membrane potential of the neuron and u represents a membrane recovery variable, which accounts for the activation of K+ ionic currents and inactivation of Na+ ionic currents, and it provides negative feedback to v. After the spike reaches its apex (+30 mV), the membrane voltage and the recovery variable are reset according to the (3). Synaptic currents or injected dc-currents are delivered via the variable I. The part 0:04v2+5v+140was obtained by fitting the spike initiation dynamics of a cortical neuron (other choices also feasible) so that the membrane potential v has mV scale and the time t has ms scale. The resting potential in the model is between 70 and 60 mV depending on the value of b. As most real neurons, the model does not have a fixed threshold; Depending on the history of the membrane potential prior to the spike, the threshold potential can be as low as 55 mV or as high as 40 mV. • The parameter a describes the time scale of the recovery variable u. Smaller values result in slower recovery. A typical value is a = 0:02. • The parameter b describes the sensitivity of the recovery variable u to the subthreshold fluctuations of the membrane potential v. Greater values couple v and u more strongly resulting in possible subthreshold oscillations and low-threshold spiking dynamics. A typical value is b = 0:2. The case b < a(b > a) corresponds to saddle-node (Andronov–Hopf) bifurcation of the resting state [10]. • The parameter c describes the after-spike reset value of the membrane potential v caused by the fast high-threshold K+ conductances. A typical value is c = 􀀀65 mV. • The parameter d describes after-spike reset of the recovery variable u caused by slowhigh-threshold Na+ and K+ conductances. A typical value is d = 2. Various choices of the parameters result in various intrinsic firing patterns, including those exhibited by the known types of neocortical [1], [3], [4] and thalamic neurons as summarized in Fig. 2.Apossible extension of the model (1), (2) is to treat u, a and b as vectors, and use u instead of u in the voltage (1). This accounts for slow conductances with multiple time scales, but we find such an extension unnecessarily for cortical neurons. $ xviewer "$d_Refs"'Mathematics/images/Izhikevich Nov2003 Known types of neurons.png' #] +-----+ # 08********08 #] WCCI2020 NN-20867 r Chevtchenko, Ludermir - Learning from Sparse and Delayed Rewards with a Multilayer Spiking Neural Network 11Feb2020 paper submission [33] L. Lapicque, “Recherches quantitatives sur lexcitation electrique des nerfs traite comme une polarization,” J. Physiol. Pathol. Gen., vol. 9, pp. 620–635, 1907. A discrete-time update rule of neuron i is as follows: (1) vi(t) = vi(t -1)*(1 - 1/τ_m) + ξ(t) + I(t) (2) I(t) = Σ[j; wji*sj(t-1)] where vi(t) is the potential of neuron i, τ_m is the membrane discharge time constant and ξ(t) is the Gaussian noise from a distribution N(μ, σ^2). The injected current I(t) in Equation 2 is the sum of presynaptic spikes from neuron j, times the synaptic efficacy between neurons i and j. The following is a simplified version of the reward modulated STDP plasticity rule introduced by Florian [12]. When postsynaptic neuron i fires, the eligibility trace Zji is set to the value of the presynaptic trace P_􀀀, decaying exponentially with a time constant τe : (3) Zji(t) = Zji(t-1)*(1 - 1/ τe) The change in synaptic strength between two neurons depends on presynaptic and postsynaptic spike times, as well as on the reward signal: (4) wji(t) = wji(t-1) *(1 - 1/τs) + α*r(t)*Zji(t) where τs is the synaptic tag discharge time constant and α is the learning rate. The reward signal r(t) is broadcasted to all plastic synapses. The above rule is illustrated in Figure 3. Note that the time constants τe and τs are adjusted according to the duration of a trial for a specific task. the traditional Q-learning algorithm works well with a limited number of states and actions, but as demonstrated in Section IV-D, becomes inefficient for problems with multiple dimensions. One common solution is to replace a Q-table, representing all possible states and actions, with a parameterized function θ, such as a neural network: Q(s(t), a(t), θ(t)). Then, the update rule in the Equation 5 becomes: (7) θ(t+1) = θ(t) + α(Y(t)^Q) - Q(s(t), a(t), θ(t)))*r(θ(t))*Q(s(t), a(t), θ(t)) (8) Y(t)^Q = r + Qmax(s(t+1), θ(t)) where Y Q t is a target value for Q(st; at; #t), representing expected value from taking action at on state st. SpikeProp [12] SWAT [4] Self-Regulating Evolving Spiking Neural (SRESN) classifier [13] Two stage Margin Maximization SNN (TMM-SNN) [7] SEFRON [6] To train DC-SNN using biologically plausible mechanisms, we propose a Self-regulated Competitive STDP (SR-STDP) based learning algorithm. SR-STDP utilizes only the locally available information on a synapse to update its weight. Depending on the response of output neurons, SR-STDP self-regulates its learning strategy for each neuron in the network. # 08********08 #] Lateral inhibition : k-WTA not WTA "... We utilize a softer form of Lateral Inhibition like that of k-WTA, which is proven to be computationally less power intensive than a hard Lateral Inhibition [38] and leads to better shared feature selectivity in cortical pyramidal cells [39]. ..." from : Neural Nets WCCI2020 reviews - mine NN-21114 read & comment 21114 r George, Banerjee, Dey, Mukherjee - A Reservoir-based Convolutional Spiking Neural Network for Gesture Recognition from DVS Input.txt [38] W. Maass, “On the computational power of winner-take-all,” Neural Computation, vol. 12, no. 11, pp. 2519–2535, 2000. [39] Z. Jonke, R. Legenstein, S. Habenschuss, and W. Maass, “Feedback inhibition shapes emergent computational properties of cortical microcircuit motifs,” Journal of Neuroscience, vol. 37, no. 35, pp. 8511–8523, 2017. # 08********08 #] SNN: differentiation surrogates "... One of the major bottlenecks of direct SNN training is the non-differentiability of the spike generation. The infinite gradient of the spike function impedes the gradient propagation during the backward pass in training. Empowered by various • surrogate gradient (SG) functions [6]–[8], the inaccessible gradient of the spike function is approximated and propagated during learning. However, the inaccurate approximation and heuristic SG selection hurt the training stability with deep models (e.g., ResNet [1]), which further motivated the • temporal normalization method [9] and • output regularization techniques [6], [10] to smooth the loss. ..." "$d_PROJECTS"'2024 WCCI Yokohama, Japan/reviews- mine/93740 b Spiking Neural Network with Learnable Threshold for Event-Based Classification and Object Detection.txt' : [6] S. Deng, Y. Li, S. Zhang, and S. Gu, “Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting,” in International Conference on Learning Representations (ICLR), 2021. [7] J. H. Lee, T. Delbruck, and M. Pfeiffer, “Training Deep Spiking Neural Networks Using Backpropagation,” Frontiers in Neuroscience, vol. 10, p. 508, 2016. [8] Y. Wu, L. Deng, G. Li, J. Zhu, Y. Xie, and L. Shi, “Direct Training for Spiking Neural Networks: Faster, Larger, Better,” in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), vol. 33, no. 01, 2019, pp. 1311–1318. [9] H. Zheng, Y. Wu, L. Deng, Y. Hu, and G. Li, “Going Deeper with Directly-trained Larger Spiking Neural Networks,” in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), vol. 35, no. 12, 2021, pp. 11 062–11 070. [10] Y. Guo, X. Tong, Y. Chen, L. Zhang, X. Liu, Z. Ma, and X. Huang, “RecDis-SNN: Rectifying Membrane Potential Distribution for Directly Training Spiking Neural Networks,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 326–335. enddoc