http://www.BillHowell.ca/Neural nets/Paper reviews/5_NN math defs, symbols, nomenclature - time-scale systems.txt www.BillHowell.ca 29Jan2018 initial taken from : http://www.BillHowell.ca/Neural nets/Paper reviews/5_NN math defs, symbols, nomenclature - NN stability with delays, impulses.txt ***************************** >>>>> "Flat-line" UNICODE nomenclature : This style of writing expressions allows the use of any UNICODE-enabled text editor without need for subscripts or superscripts. It also facilitates faster copy-paste re-use of expression. Furthermore, although not used for this paper, the intent is to facilitae used of expressions in a symbolic processor (I've only used this in the past for very limited capablities). Admittedly, it is a nuisance to adapt and lacks the clarity of standard formatting. Various notations may be combined... A variable [scalar, vector, matrix] for use in this list of notations A(i,j) (i,j)th component of A A(i,j) (i,j)th component of A, notation just for simple indices A_T transpose of A |A| absolute value of matrix A (each element) ||A|| [Euclidean, spectral] norm of A ||A||2 p3h0.25 ||·|| denotes the Euclidean norm. ||A||_p = ( sum(i from 1 to n : |A(i)|^p )^(1/p) ) denotes the p-norm of vector z A_dot time derivative of A, same as d[dt: A] A_til A tilde over A_ovr A overlined A_und A underlined A_hat A with ^ above it A_sup(x) A superscript x - to avoid confuxion with A^x A_sub(x) A subscript x - as distinct from A(x) A^x A to the power of x λ max ( λ min ) refers to maximum (minimum) eigenvalue t_minus t approached from t < t_minus t_plus t approached from t_plus < t Indexing : yi y(i), simple concatenation (might be confused with y*i) y(i+1) array type notation Note the more clear & easy to read indexing used in this review for mixing indexes and t or other variables ... example ai(t) -> a(i,t) Inequalities (for faster typing, symbolic processing) ≤ <= ≥ >= ≠ != [super, sub]-script symbols "_cup" denotes variable values during ??? eg p4L23 Eq (9d) "_ovr" denotes variable values during times s ∈ [t_til(0),0]??? eg ξ_ovr(t) "_til" denotes variable values during ??? eg p4L38 Un-numbered [equations, inequalities] in the paper are denoted by their location, enclosed by parenthesis, eg : * y(t) ≤ ρ*sup[s∈[t−T,t]: y(s)] means the above equation on page 5, Line 42. Reviewer equations are denoted by some [equation, section, subsection, whatever] and index, enclosed by parenthesis, eg : (R.10a.5) z(t**) ≤ d/c*z(t*)*e^(-θ*ln(c)/T) , ∀t ≥ t0 Calculus : d[dt: x] total derivative of x with respect to t dp[dt: x] partial derivative of x with respect to t d^n[dt^n: x] nth total derivative of x with respect to t dp^n[dt^n: x] nth partial derivative of x with respect to t ∫[ds: f] indefinite integral of f with respect to s ∫[ds, a to b: f] definite integral of f with respect to s, from a to b Other : sum[i=1 to n: f(n,t)] sum of a series Π[tt_plus : (r(t,t) - r(s,t))/(t - s)) } D_minus D_minus(t) = lim{s->t_minus: (r(t,t) - r(s,t))/(t - s)) } ******* denotes start/end of topics & sub-topics +-----+ denotes sub-steps in a [proof, development].. denote [start,end] of checks on specific steps by the reviewer (me), on expression (1) in the example below : >>>>> check (?) : >> ???? check (?) unfinished <<<<< end check. >> start of short reviewer comments << end of short reviewer comments (Often this is not used as only one or two lines of comment are involved >>** comment for extraction to main part of review, Section "C6. MATH CHECKS - step-by-step" *************************** >>>>> Symbols +-----+ first characters of line - indicates definition used in a similar paper, which makes for interesting contrasts, plus confirmation of a usage of variable symbols that is similar to other papers in the area (*) taken from earlier papers, mostly for NN stability Theorems p2h0.8 (example) - taken from "180207 conference paper review - math only.txt" kwrite editor, regular expression -> Search : ^\(p[0-9]*L[0-9]*\), Replace : \(171127\) special contexts : - normally, I will use a Latin letter in place of Greek or some Unicode if that letter is not used elsewhere (for ease of typing) - Note that the authors use symbols in a manner consistent with other papers in the area, of course making this paper easier to follow, and to identify subtle differences in assumptions etc. +-----+ General list : normally in alphabetical order, except some "collections" of themes (start/end with +--+) +--+ Metric spaces, sets (hexa refers to U+hexa) LibreWrite (Ctrl+Shift)+U+hexa kwrite text editor enter into LibreWrite, copy to kwrite? * 𝕁 subset of ℝ^n * ℝ, real numbers * ℝ+, non-negative real numbers p3h0.25 ℝ denotes the real number set p3h0.25 ℝ+ denotes the nonnegative real number set * ℝ^n n-dimensional Euclidean spaces * ℝ^(n×n) n×n real matrix p2h0.75 𝓡 and 𝓡+ denote the set of regressive and positive regressive functions; * 𝕊 subset of ℝ p2h0.33 𝕋 represents a time scale which is an arbitrary nonempty closed subset of the real set ℝ. p3h0.3 min{𝕋} = 0 Without other statements, we always assume this p3h0.75 𝕋aux ∆= {t(k,2)}|k=1 to ∞ an auxiliary discrete time scale where p3h0.75 t(i,2) is the i-th right-scattered point of 𝕋. * ℤ positive integers p3h0.25 ℤ denotes the integer number set +--+ Latin symbols sorted in Latin alphabetical order parameters of inertial NN (1), real-valued continuous functions, i ∈ N * a(i,t) > 0 * b(i,t) > 0 * c(i,j,t) connection weights without delays * d(i,j,t) connection weights with delays * A(t) = diag[α(1,t),α(2,t),...,α(n,t)]_T * B(t) = diag[β(1,t),β(2,t),...,β(n,t)]_T * C(t) = ( c(i,j,t) )^(n*n) * D(t) = ( d(i,j,t) )^(n*n) p4h0.65 A = max[t≤T,t ∈ ℝ+(𝕋) : {G(t)} ] p4h0.65 B = max{M, A}, * C^+(S,J) = {z : S → J is continuous and its upper right Dini derivative D+z exists } ... for any interval S ⊆ ℝ, J ⊆ ℝ^n * D^+[dz: f(z)] ∆= lim[sup[h -> 0+: (f(z+h) - f(z))/h], Dini derivative of z ... https://en.wikipedia.org/wiki/Dini_derivative p3h0.0 c a regressive constant p2h0.75 Crd = Crd(𝕋) = Crd(𝕋,ℝ) denotes the set of rd-continuous functions p2h0.85 e(p,t,s) = exp( ∫[∆τ from s to t: ξ(µ(τ),p(τ)) ] ) for p ∈ 𝓡, s,t ∈ 𝕋^k ... exponential function definition p3h0.0(a) e(c,t,0) = exp(c*t) for any t ∈ ℝ, 𝕋 = ℝ timescale-type exponential function p3h0.0(b) e(c,t,0) = (1 + c*h)^(t/h) for any t ∈ h*ℤ, 𝕋 = h*ℤ timescale-type exponential function p2h0.85 e(p,t,s) = exp( ∫[∆τ from s to t: ξ(µ(τ),p(τ)) ] ) for p ∈ 𝓡, s,t ∈ 𝕋^k exponential function definition * f = (f1,···,fn) ∈ C(ℝn,ℝn) is the activation function * f(j,ξ(j,t)) represents the neuron activation function of the jth neuron at time t p1h0.75 f : 𝕋 × ℝ^n → ℝ^n is a rd-continuous function satisfying f(t,0) = 0 for all t ∈ 𝕋. p2h0.75 fσ(t) = f(σ(t)) p2h0.75 f∆(t) denotes the ∆-derivative of f at t ∈ 𝕋; * |f(j,u) - f(j,v)| ≤ l(j)*|u - v| ∀u,v ∈ R, j ∈ N maximum |"gradient"| sort-of p4h0.6 G(t) = ∫[∆τ, from 0 to t : g(τ) ] p3h0.5 g : ℝ+ → ℝ a regressive function * J(i,t) denotes the neuron of an external input on the ith neuron * J ⊆ ℝ^n interval * k ∈ N0 * K = {f ∈ C+(ℝ,ℝ+) | f(t) ≤ 1, if t ≤ 0; f(t) ≥ 1, if t > 0 } p1h0.95 k ∈ ℤ discrete time index * L = diag(l1,···,ln) * L = diag{l,l,...,l}, where l = max{1≤j≤n: l(j) } * l(j) are positive constants * l = max[1≤j≤n: l(j) ] * M ≥ 0 constant * M > 0 constant Eq (8) p4h0.65 M < 0 constant * N = {1,2···}, * N = {1,2,···,n} total number of neurons * N0 = N ∪ {0} * PC(J,_ovr,∆) = {φ : J_ovr → ∆ is continuous everywhere except at finite ... number of points t, at which φ(t+), φ(t−) exist and φ(t+) = φ(t)}. ... assume "PC" stands for "Pulse Condition" or something like that * PC(S,J) = {z : S → J is continuous everywhere except at finite number of points t ... at which z(t+), z(t−) exist and z(t−) = z(t) } p2h0.8 ⊖p = -p(t) /(1+µ(t)p(t)) p ∈ 𝓡+ p2h0.8 p⊖q = (p(t)−q(t))/(1+µ(t)q(t)) p, q ∈ 𝓡+ * R(l)  I, l ≥ 1, is impulsive matrix * R(t) = diag[γ(1,t),γ(2,t),...,γ(n,t)]_T but γ(1,t) = γ(1t)!?! p2h0.75 𝓡 and 𝓡+ denote the set of regressive and positive regressive functions p2h0.75 rd-continuous = right dense continuous (http://campus.mst.edu/adsa/contents/v6n2p1.pdf) * s ∈ [t_til(0),0], for Eq (3), (p3L0) S ⊆ ℝ interval * t ≥ t0 , denotes the reset rate of neurons * t_til(0) = inf{over t ∈ [0,∞): t − τ(t) } often use s ∈ [t_til(0),0] Eq (2) * t∗ = t∗ + l*T + θ, where θ ∈ [0, T ) * t* = ∈ [0,t1) * t∗∗ ∈ [tK,t(K+1)) * {tk}_sup(∞)_sub(k=1) is the impulsive time sequence * tk impulse time, satisfy 0 < t1 < t2 < ... < tk → ∞ as k → ∞ * T is the length of a time sub-interval within t* -> t** ... such that T(1) ≤ t(k+1) − t(k) ≤ T(2) with T(2) ≥ T(1) > 0 p3h0.65 UUB ultimate upper-bound p3h0.65 UUB = α^(−1)(V(0,x0)*e^λ ) by Theorem 1 * V(t) impulsive differential delay inequality function Eq (11a) ... while not introduced as a Lyapunov functional, as below with V_Lyap(t), that is what is used p3h0.5 V : ℝ+(𝕋) × ℝ^n → ℝ+ with V (t,0) = 0 for all t ∈ ℝ+(𝕋) ... an rd-continuous differentiable function p3h0.6(ii) V∆(t,x) ≤ g(t)*V(t,x) for t ∈ ℝ+(𝕋), with ∫[∆t, from 0 to ∞: g(t)*∆t] ≤ λ * V_Lyap(t) Lyapunov function Eq (29) * V_til(0) = sup[s ∈ [t_til(0),0]: V(s)], where V_ovr(t) ≠ 0, for Eq (14) * vk ≥ 1 constant between impulses for Eq (11) * W(t) == V(t)*ω(t) ≤ Π[k=1 to ι-1: vk*V_til(0) ] Eq (14) ... note "==" is used for U+2250, which doesn't display in this text editor * X(t) = (ξ(t)_T, η(t)_T)_T solution of (7) * Y(t) = (ξ_ovr(t)_T, η_ovr(t)_T)_T solution of (7) * x = (x1,···,xn) T ∈ ℝ^n is the state vector of neurons p1h0.75 x(t) ∈ ℝ^n is the state of the system, p1h0.6(1) x∆(t) = f (t,x(t)), t ∈ 𝕋, timescale-type time-varying system p1h0.8(3a) d[dt: x(t)] = f(t,x(t)), t ∈ ℝ, p1h0.95(4a) x(k+1) = x(k) + f(k,x(k)), k ∈ ℤ, * x(i,t) = ξ(i,t) - ξ_ovr(i,t) * y(i,t) = η(i,t) - η_ovr(i,t) * x(t) = (x(1,t),x(2,t),...,x(n,t))_T * y(t) = (y(1,t),y(2,t),...,y(n,t))_T * z(t) = (z(1,t),z(2,t),...,z(2*n,t))_T = (x(t)_T,y(t)_T] )_T p3h0.3 = inf{ x_til ∈ 𝕋|x_til > x} For any x ∈ ℝ Greek symbols sorted in greek alphabet order (variables for which I used similar Latin symbols instead of Greek are listed under Latin symbols) * α(i,t) = γ(i)*{a(i,t) - γ(i,t)} - b(i,t) Eq (7b) p3h0.5 α a K-class function * β(i,t) = a(i,t) - γ(i,t) Eq (7b) * γk positive constant * γ(i,t) -> seems to be an incorrect notation (1st used Eq (4)) * γ(i) - constant term (reviewer substitute for γ(i,t) : 1st used Eq (4)) p2h0.75 Δ derivative; f∆(t) denotes the ∆-derivative of f at t ∈ 𝕋; * δk constant * ζk ∈ [−2,0] represent the strength of impulses * η(t) = { η(1,t),η(2,t),...,η(i,n) }_T * η(i,t) expression related to d[dt: ξ(i,t)] and ξ(i,t) used to simplify expressions, defined by (4) * η(i,tk+) = lim[t→tk+0 : η(i,t)], i.e., the solution η(i,t) is right continuous at impulse point tk * θ ∈ [0,T) (p14L5), and as ε≥0 (p4L37), this implies that θ≥0? * θ(s) = { φ(s)_T, ψ(s)_T } * Θ(s) = { φ(1,s),φ(2,s),...,φ(2*n,s) }_T = (φ_cup(s)_T , ψ_cup(s)_T)_T for s ∈ [t_til(0),0] for Eq (10) * ι ∈ ℤ, eg t ∈ [0,tι) * λ > 0 constant Eq (8) p2h0.75 µ denotes the graininess function * ξ(t) = { ξ(1,t),ξ(2,t),...,ξ(i,n) }_T * ξ(i,t) denotes the state vectors of the ith neurons at time t, ... solution is right continuous at impulse point tk ... second-order derivative of ξ(i,t) is called an inertial term of system (1) * ξ(i,tk+) = lim[t->tk+0: ξ(i,t)] = ξ(i,tk), ... i.e., the solution ξ(i,t) is right continuous at impulse point tk p2h0.9 ξ(h,z) = { log(1+zh)/h for h ≠ 0; z for h = 0 } cylinder transformation definition * ∆ξ(i,tk) = ξ(i,tk+) − ξ(i,tk-) = δk*x(i,tk) Eq (7c) * ξ(i,s) = φ(i,s), s ∈ [t_til(0),0], i ∈ N * η(i,s) = ψ(i,s), s ∈ [t_til(0),0], i ∈ N * Π(1,t) = max{ max[1≤i≤n: sum[j=1 to n: |c(j,i,t)|*l] + |α(i,t)| - γ(i,t) ], max[1≤i≤n: 1 - β(i,t) ] } * Π(2,t) = { sum[i=1 to n: max[1≤j≤n: |d(j,i,t)| ] ] }*l * ρ(t) ≥ 0 continuous function for Eq (11) p2h0.75 ρ backward jump operator * ς(t) ≥ 0 continuous function for Eq (11) p2h0.75 σ denotes the forward jump operator * τ(t) is the time-varying delay satisfying τ(t) ≤ τ_ovr with τ_ovr > 0, * τ(t) ∈ PC(R+,R+) time delay satisfying t − τ(t) → ∞ as t → ∞ for Eq (1) * φ(s) ∈ PC([-τ_ovr,0],ℝ^n) is the initial condition * Φ(t) = { Φ(1,t),Φ(2,t),...,Φ(i,n) }_T +--+ initial conditions of inertial NN (1), real-valued continuous functions, s ∈ [t_til(0),0], i ∈ N * Φ(i,s) = ξ(i,s), s ∈ [t_til(0),0], i ∈ N p3L18 * φ(i,s) = d[ds : ξ(i,s)], s ∈ [t_til(0),0], i ∈ N p3L18 * Φ(s) = { φ(1,s),φ(2,s),...,φ(2*n,s) }_T = (φ_cup(s)_T , ψ_cup(s)_T)_T for s ∈ [t_til(0),0] for Eq (10) * φ_cup(s) = { φ_cup(1,s),φ_cup(2,s),...,φ_cup(n,s) }_T for Eq (10) +--+ * ψ(t) = { ψ(1,t), ψ(2,t),...,ψ(i,n) }_T for (7) * ψ(s) = { Φ_ovr(s)_T, ψ_ovr(s)_T }_T initial conditions for (8) * ψ_cup(s) = { ψ_cup(s),ψ_cup(s),...,ψ_cup(s) }_T initial conditions for (10) * ω(t) ∈ K satisfying ω(t) → ∞ as t → ∞, Eqs (11)(12) * ω_ovr(k) = { max |δk| + 1, |ζk| + 1 } Eq (28) ******************************************* >>>>> MATH DEFINITIONS +-----+ Definition 1. System (1) is said to be: 1. Bounded if for any initial value x0 , there exists a positive scalar δ = δ(x0 ) such that kx(t)k ≤ δ for all t ∈ T. 2. Lyapunov stable if for an arbitrary ε > 0, there exists δ(ε, t0 ) such that kx(0)k < δ ⇒ kx(t)k < ε, ∀t ∈ T. 3. Asymptotically stable if it is Lyapunov stable, and there is a constant c = c(x0 ) > 0 such that x(t) → 0 as t → ∞, for all kx0 k < c. inf (infinum) inf = inf{ x_tild ∈ T| x_tild > x} # enddoc