key: cord-0055910-8hsxj00p
authors: Mao, Xiaochen; Li, Xingyong; Ding, Weijie; Wang, Song; Zhou, Xiangyu; Qiao, Lei
title: Dynamics of a multiplex neural network with delayed couplings
date: 2021-02-02
journal: Appl Math Mech
DOI: 10.1007/s10483-021-2709-6
sha: e42c3b520e89a9b04c8a5a591066a6ea08827445
doc_id: 55910
cord_uid: 8hsxj00p

Multiplex networks have drawn much attention since they have been observed in many systems, e.g., brain, transport, and social relationships. In this paper, the nonlinear dynamics of a multiplex network with three neural groups and delayed interactions is studied. The stability and bifurcation of the network equilibrium are discussed, and interesting neural activities of the network are explored. Based on the neuron circuit, transfer function circuit, and time delay circuit, a circuit platform of the network is constructed. It is shown that delayed couplings play crucial roles in the network dynamics, e.g., the enhancement and suppression of the stability, the patterns of the synchronization between networks, and the generation of complicated attractors and multi-stability coexistence.

In the past few years, great efforts have been made on the investigations of multiplex networks due to their wide range of applicability in fields such as biology, physics, sociology, epidemiology, and engineering [1] [2] [3] . Multiplex networks are termed in disparate ways, e.g., interacting networks, networks of networks, multilevel networks, and hypergraphs [2] . In such networks, the nodes are distributed in individual networks, e.g., layers and loops, each of them accounts for a kind of interaction process existing among the nodes. For example, in the case of a C. elegans neuronal network, a proper way to describe it is a multiplex graph with two sub-networks, one for the chemical synaptic links and one for the gap junction interactions [2] . Moreover, due to the cortical connectivity of mammalian brains, the corticocortical network of the cat can be constructed by modeling each node (cortical area) with a sub-network of interacting neurons [4] [5] . Examples also include air transportation networks, power grids, coupled semiconductor lasers, communication networks, social networks, infectious disease, etc. [1] [2] [3] [4] [6] [7] [8] [9] .

The analysis of dynamical behaviors of multiplex networks is one of the focal issues in complex nonlinear systems, and has found important and extensive applications in science and engineering [2, [10] [11] [12] [13] [14] [15] [16] . For example, experimental evidence shows that chaotic behaviors exist in the brain networks and associate with the information processing, cognitive function, memory storage, and retrieval [10] [11] . In multiplex networks, the dynamic processes take place at the same time in interacting sub-networks with different structural and dynamical configurations. The interactions of the sub-networks open up a plethora of fascinating behaviors, e.g., complete/partial synchronization, intra/inter-layer synchronization, cluster synchronization, and explosive synchronization [7, [12] [13] [17] [18] . The phenomena of synchronization in networks are of great importance in nature, and have become focal subjects in many fields [7, [12] [13] [19] [20] [21] [22] [23] . For instance, the synchronization of neurons plays a crucial role in the context of cognition and learning and the pathogenesis of several neurological diseases, e.g., Parkinson's disease and essential tremor [7, 19, 21, 24] . It is therefore important and essential to investigate the dynamical behaviors arising from the interactions of separate networks.

Since signal propagation is always non-instantaneous, time delays commonly exist in realistic systems [15, [25] [26] [27] [28] [29] . In neural networks, for example, the switching speed of action potentials propagating across neuron axons is finite, and the time lapses occurring by both dendritic and synaptic processing are non-negligible [30] [31] . Neglecting the time delay in a dynamical system usually leads to false or even wrong results. In recent years, the investigations on the dynamics of multiplex neural networks with time delays have been witnessed [17] [18] [32] [33] [34] [35] [36] [37] . Previous studies were mostly devoted to the dynamical characterization of two-coupled networks since such models are fundamental and easily addressable, e.g., two-layer networks [7, [12] [13] 36, 38] . Little attention has been paid to neural networks made up of three or more interconnected networks. However, the multi-coupled structures are ubiquitous in neural systems [2, 9, 12] . In brain, many interconnected areas (sub-networks) can be seen, which usually consist of neural assemblies and their couplings represent the structural and functional interactions among them [9] . For instance, separate parallel neural loops in the cortex or thalamus operate through the basal ganglia and the interplay of them can lead to the generation of tremor oscillations in Parkinson's disease and epilepsy [32] . Thus, disregarding the multiplex structure of networks may result in misunderstanding of the properties of neural systems.

Motivated by the above discussion, the purpose of this paper is to study the dynamical behaviors of a multiplex network, as shown in Fig. 1 . The system consists of three networks, each of which has an arbitrary number of nodes and couplings between single neurons. Different time delays are introduced into the connections between networks. The remaining part of this paper is organized as follows. In Section 2, the local and global stability and bifurcation of the trivial equilibrium of the network are analyzed. Case studies of numerical simulations are shown in Section 3. A circuit platform is designed to validate the obtained results in Section 4. Finally, conclusions are made in Section 5.

As shown in Fig. 1 , the network can be described by

where i, j = 1, 2, 3, · · · , n, and x i , y i , and z i denote the states of the ith neuron in Networks 1, 2, and 3, respectively. a ij , b ij , and c ij are the connection weights within the networks. r 1 , r 2 , and r 3 are the coupling strengths between the first neuron of each sub-network. τ i represents the time delay in the couplings. Without loss of generality, the functions f within the individual sub-networks and the function g between sub-networks are absolutely smooth and satisfy f (0) = 0 and g(0) = 0. In this network, the local kinetics of each node is described by the Hopfield neuron [39] .

The linearization of Eq. (1) at the trivial equilibrium of the network can be written as follows:

where u(t) = (x 1 , x 2 , x 3 , · · · , x n , y 1 , y 2 , y 3 , · · · , y n , z 1 , z 2 , z 3 , · · · , z n ) T ,

In the above equations, O is an n × n zero matrix, i, j = 1, 2, 3, · · · , n, and l = 1, 2, 3. The characteristic equation of the network is

where I is an n × n identity matrix. Thus, the characteristic equation of the network reads

where I n−1 is an (n − 1) × (n − 1) identity matrix, and

In fact, A 1 , B 1 , and C 1 represent the connection matrices of the three sub-networks without the first neuron, respectively, i.e.,

The stability of the trivial equilibrium of the network can be determined by the root distribution of the characteristic equation (3). When the network has no coupling time delays, the Routh-Hurwitz criteria can be applied. As the time delays vary, let λ = ±iω (ω > 0) be the roots of the characteristic equation. By separating the real and imaginary parts of ∆(iω, τ ) = 0 and eliminating the harmonic terms, one arrives at

where P R (ω) = Re(P (iω)), P I (ω) = Im(P (iω)), Q R (ω) = Re(Q(iω)), and Q I (ω) = Im(Q(iω)). Suppose that D(ω) = 0 has some positive roots ω j . Then, a set of critical time delays can be obtained by τ j,l = (θ j + 2lπ)/ω j (l = 0, 1, 2, · · · ), where θ j ∈ [0, 2π) yields sets of triangle equations

To determine the variation direction of the real part of Eq. (4) with respect to the time delay, the differentiation of λ with respect to τ yields

Then, one arrives at

where

Thus, the system undergoes finite stability switches, and must become unstable with an increase in the time delay when D(ω) = 0 has positive roots. On the other hand, it is delay-independently stable or unstable for any given time delay when D(ω) = 0 has no positive root [25, 40] . In fact, the stability of the trivial equilibrium of the multiplex network is determined by the characteristic roots of the isolated sub-networks and the product of the coupling strength and the sum of the time delay.

Let n = 3,

Based on the above analysis, the region in the parameter planes indicating the stability of the trivial equilibrium of the triplex network is given in Fig. 2 . Along the black and red curves, the characteristic equation of the system has a pair of pure imaginary roots. Moreover, the characteristic equation adds a pair of conjugate roots with positive real parts for each crossing black curves while reduces a pair conjugate roots with positive real parts for each crossing red curves. Besides, the network has delay-independent and delay-dependant stability regions. Furthermore, the global asymptotic stability of the trivial equilibrium of the network is discussed as follows. In neural systems, due to the input-output relation of neurons, the activation functions f and g are often assumed to satisfy the normalization, monotonicity, boundedness, and concavity conditions, i.e., f ′ (0) = g ′ (0) = 1; f ′ (υ) > 0 and g ′ (υ) > 0 for all υ ∈ R; −∞ < lim υ→±∞ f (υ) < +∞ and −∞ < lim υ→±∞ g(υ) < +∞; υf ′′ (υ) < 0 and υg ′′ (υ) < 0 for all υ = 0. The sigmoid functions, e.g., hyperbolic tangent functions, are widely used in neural networks.

Theorem 1 If np + q < 1, the trivial equilibrium of the network is globally asymptotically stable for all time delays, where p = max{|a ij |, |b ij |, |c ij |}, and q = max{|r 1 |, |r 2 |, |r 3 |}.

Proof Define the following Lyapunov function:

Then, one arrives aṫ

where a = max{|a i j|}, b = max{|b ij |}, and c = max{|c ij |}. Since υf ′′ (υ) < 0 and υg ′′ (υ) < 0 for υ = 0, one has f ′ (υ) f ′ (0) = 1 and g ′ (υ) g ′ (0) = 1. In addition, one obtains

There exist ϕ * ∈ (0, 1] and φ * ∈ (0, 1] such that ϕ i (t) ϕ * 1 and φ i (t) φ * 1. Hence, one hasV

Then,V (t) < 0 holds true for np + q < 1 when υ = 0. This completes the proof.

In this section, the activation functions of neurons are chosen as f = g = tanh, which is a typical sigmoid function and has been widely used in neural networks.

Case 

The trivial equilibrium of the network free of coupling time delays is locally asymptotically stable according to the Routh-Hurwitz criteria. Solving the polynomial D(ω) = 0 gives two positive and simple roots ω 1 = 3.26 and ω 2 = 3.17. Then, two sets of critical time delays can be obtained as τ 1,l = 0.15, 2.08, 4.00, · · · , and τ 2,l = 1.26, 3.25, 5.23, · · · . It is easy to check that D ′ (ω 1 ) > 0 and D ′ (ω 2 ) < 0 hold. From sgn Re dλ(τ ) dτ λ=iω = sgn D ′ (ω), the characteristic equation of the network adds a new pair of conjugate roots with positive real parts for each crossing at τ 1,l , but reduces a pair of conjugate roots with positive real parts for each crossing at τ 2,l . Therefore, the trivial equilibrium of the network is locally asymptotically stable for τ ∈ [0, τ 1,0 ) ∪ · · · ∪ (τ 2,l , τ 1,l+1 ) ∪ · · · ∪ (τ 2,14 , τ 1,15 ), and becomes unstable for τ ∈ (τ 1,0 , τ 2,0 ) ∪ · · · ∪ (τ 1,l , τ 2,l ) ∪ · · · ∪ (τ 1,15 , +∞). Figure 3(a) shows that the trivial equilibrium of the network loses its stability and a branch of periodic oscillations arising from the Hopf bifurcation comes into being when τ = 0.3 ∈ (τ 1,0 , τ 2,0 ). Obviously, the corresponding neurons in sub-networks oscillate with the same waveform and in phase with each other, i.e., x i = y i = z i . Figure 3 (b) illustrates that the trivial equilibrium remains stable when τ = 1.8 ∈ (τ 2,0 , τ 1,1 ). Figure 3(c) gives that the trivial equilibrium becomes unstable again and periodic oscillations occur when τ = 2.4 ∈ (τ 1,1 , τ 2,1 ). As shown in Fig. 3(c) , the neurons in the sub-networks move with the same waveform and different phases. The red, green, and blue lines represent the responses of the first neurons in the three sub-networks. It is shown that the network undergoes the stable rest state, completely synchronous periodic oscillation, stable rest state, and periodic oscillations with different phases as the time delay increases. To show the effects of the property of the coupling on the network dynamics, let the parameters within the sub-networks be the same as those in the above case study and the time delay τ s = 0.1. Figure 4(a) gives the asynchronous periodic oscillations with x i = −y i = z i when k 1 = −k 2 = −k 3 . As shown in Fig. 4(a) , the neurons of Networks 1 and 3 oscillate with the same waveform and in phase with each other, but the neurons of Networks 1 and 2 move with the same waveform and half a period out of phase with each other. Figure 4(b) illustrates that the neurons of Networks 1 and 2 move synchronously, but the neurons of Networks 1 and 3 oscillate out-of-phase. Figure 4(c) shows the asynchronous periodic oscillations with x i = −y i = −z i when k 1 = k 2 = −k 3 . In this case, the neurons of Networks 2 and 3 move synchronously, but the neurons of Networks 1 and 2 oscillate out-of-phase. As shown in Fig. 4 , the red solid, green dashed, and blue dotted curves represent the responses of the first neurons Figure 5 shows the coexistence of two period-2 orbits and two period-4 oscillations under different initial conditions when τ s = 0.1. It is seen that the period-2 orbits on the x 1 -x 2 plane are colored in blue and red lines, while the period-4 trajectories are colored in purplish red and black lines. The blue, red, purplish red, and black lines correspond to the initial conditions, which are defined as (x 1 (0), x 2 (0), x 3 (0), y 1 (0), y 2 (0), y 3 (0), z 1 (0), z 2 (0), z 3 (0)), IC1 (0.5, 0.1, 0. Fig. 6 illustrates the coexistence of a new pair of period-2 oscillations and two separated chaotic motions. As shown in Fig. 7 , the period-2 responses disappear and a pair of period-4 solutions come into being when τ s = 0.8. Figure 8 gives the coexistence of two chaotic motions and two period-2 responses for τ s = 1.2. As shown in Figs. 5-8, different types of multiple coexisting attractors are observed when the coupling time delays vary. Figure 9 gives the bifurcation diagram as a function of the coupling time delay τ s . The Poincaré section is defined by = {(τ s , x 1 ) : (x 2 = 0,ẋ 2 > 0). When the coupling time delay varies, the multiplex network exhibits interesting and complicated behaviors, e.g., multi-periodic orbits and chaotic attractors, showing that time delay can be used to regulate the dynamic performances of networks, including the generation and transition of different complex oscillations. 

The circuit is constructed based on Hopfield neuron circuit, transfer function circuit, and time delay circuit. As shown in Fig. 10 , the Hopfield neuron circuit unit is achieved by inverting adder circuit, integral circuit, and inverting circuit. Blocks with tanh/− tanh represent the positive/negative hyperbolic tangent function circuit between neurons. The hyperbolic tangent function circuit unit is constructed by crucial bipolar transistors, operational amplifiers, potentiometers, and ±15 V direct current (DC) voltage sources [41] [42] . Blocks delay1, delay2, and delay3 are time delay circuits. The time delay circuit unit can be achieved by a resistor-capacitor Cx3 Fig. 10 A circuit for the multiplex Hopfield neural network (color online) (RC) low-pass filter consisting of operational amplifiers, capacitors, and resistors [34, 43] . The circuit state equations can be established as follows:

where X j , Y j , and Z j are the output voltages in the circuit, f is the hyperbolic tangent function, R xi , R yi , and R zi represent resistors, C xi , C yi , and C zi stand for capacitors, τ ′ 1 , τ ′ 2 , and τ ′ 3 denote time delays in the blocks deday1, delay2, and delay3, respectively, j = 1, 2, 3, i = 1, 2, 3, · · · , 19, and

It is easy to check that Eq. (1) is the dimensionless form of the circuit equation (6), where

Moreover, the connections of R x15 , R y15 , and R z15 should be removed when 

In biological and physiological systems, the interacting neural networks are crucial for the function of the brain and the efficient processing of information. The dynamical behaviors of multiplex neural systems are revealed through an example of three coupled networks, each of which has an arbitrary number of neurons. In the parameter plane of the product of coupling strength and the sum of time delays, the delay-independent and delay-dependent regions of the network equilibrium are shown. By regarding the sum of the coupling time delay as the pa-rameter, various dynamical phenomena are observed, e.g., multiple stability switches, different patterns of periodic oscillations, the coexistence of two period-2 and two period-4 responses, the coexistence of two period-2 orbits and two chaotic motions, and the coexisting period-4 and chaotic attractors. Moreover, it is found that the excitatory and inhibitory couplings can induce complete and partial synchronization between sub-networks for the same product of coupling strengths. An electronic circuit is designed, and the phenomena agree with the revealed results. The obtained results in this paper can lead to a broader understanding in the mechanisms of the rhythms and complex evolution patterns of neural systems.

Structural measures for multiplex networks

The structure and dynamics of multilayer networks

Diffusion dynamics on multiplex networks

Structure-function relationship in complex brain networks expressed by hierarchical synchronization

Analyzing inner and outer synchronization between two coupled discrete-time networks with time delays

The multilayer connectome of Caenorhabditis elegans

Complex partial synchronization patterns in networks of delay-coupled neurons

Modeling the multi-layer nature of the European Air Transport Network: resilience and passengers re-scheduling under random failures

Multilayer motif analysis of brain networks

Is there chaos in the brain? II. Eexperimental evidence and related models

Chaotic dynamics in a neural network under electromagnetic radiation

Cluster synchronization of two-layer nonlinearly coupled multiplex networks with multi-links and time-delays

Chimera states in a multilayer network of coupled and uncoupled neurons

Oscillation dynamics in an extended model of thalamic-basal ganglia

A physical view of computational neurodynamics

Global dynamics of delayed bidirectional associative memory (BAM) neural networks. Applied Mathematics and Mechanics

Delay regulated explosive synchronization in multiplex networks

Delay controls chimera relay synchronization in multiplex networks

Pathological synchronization in Parkinson's disease: networks, models and treatments

Stability, bifurcation, and synchronization of delay-coupled ring neural networks

Synchronization in complex networks and its application -a survey of recent advances and challenges

Global firing rate contrast enhancement in E/I neuronal networks by recurrent synchronized inhibition

Improving control effects of absence seizures using single-pulse alternately resetting stimulation (SARS) of corticothalamic circuit

Controlling a neuron by stimulating a coupled neuron

Dynamics of Controlled Mechanical Systems with Delayed Feedback

Design method of multiple time-delay controller for active structural vibration control

Delay effects in brain dynamics

Dynamics, control and information in delaycoupled systems

Delay-dependent stability of linear multistep methods for differential systems with distributed delays

How delays affect neural dynamics and learning

Dynamics of delay-coupled FitzHugh-Nagumo neural rings

Delayed coupling between two neural network loops

Periodic oscillations arising and death in delay-coupled neural loops

Dynamical analysis of coupled bidirectional FitzHugh-Nagumo neuronal networks with multiple delays

Synchronization in delayed multiplex networks

Induction of Hopf bifurcation and oscillation death by delays in coupled networks

Inphase and antiphase synchronization in a delay-coupled system with applications to a delay-coupled FitzHugh-Nagumo system

Inter-layer synchronization of periodic solutions in two coupled rings with time delay

Neurons with graded response have collective computational properties like those of two-state neurons

Theory and Application of Hopf Bifurcation

An electronic implementation for Liao's chaotic delayed neuron model with non-monotonous activation function

Coexisting behaviors of asymmetric attractors in hyperbolic-type memristor based Hopfield neural network

Novel chaotic delay systems and electronic circuit solutions

Acknowledgements The authors thank the anonymous reviewers for their helpful comments and suggestions that have helped to improve the presentation.