key: cord-0058863-uficv6ym
authors: Toso, Guilherme; Breve, Fabricio
title: Synchronization Analysis in Models of Coupled Oscillators
date: 2020-08-24
journal: Computational Science and Its Applications - ICCSA 2020
DOI: 10.1007/978-3-030-58799-4_64
sha: d24c91b3cf5e49cfbe4183f35e34ee325203d9bb
doc_id: 58863
cord_uid: uficv6ym

The present work deals with the analysis of the synchronization possibility in chaotic oscillators, either completely or per phase, using a coupling force among them, so they can be used in attention systems. The neural models used were Hodgkin-Huxley, Hindmarsh-Rose, Integrate-and-Fire, and Spike-Response-Model. Discrete models such as Aihara, Rulkov, Izhikevic, and Courbage-Nekorkin-Vdovin were also evaluated. The dynamical systems’ parameters were varied in the search for chaos, by analyzing trajectories and bifurcation diagrams. Then, a coupling term was added to the models to analyze synchronization in a couple, a vector, and a lattice of oscillators. Later, a lattice with variable parameters is used to simulate different biological neurons. Discrete models did not synchronize in vectors and lattices, but the continuous models were successful in all stages, including the Spike Response Model, which synchronized without the use of a coupling force, only by the synchronous time arrival of presynaptic stimuli. However, this model did not show chaotic characteristics. Finally, in the models in which the previous results were satisfactory, lattices were studied where the coupling force between neurons varied in a non-random way, forming clusters of oscillators with strong coupling to each other, and low coupling with others. The possibility of identifying the clusters was observed in the trajectories and phase differences among all neurons in the reticulum detecting where it occurred and where there was no synchronization. Also, the average execution time of the last stage showed that the fastest model is the Integrate-and-Fire.

Visual Attention systems are widely researched [6, 19, 27] and it is a technique used by biological neural network systems developed to reduce the large amount of visual information that it is received by natural sensors [8] . This mechanism selects a subset of the information coming from the sensors to recognize the environment. This is due to the limited hardware processing of the neural system of living beings.

The process happens due to factors that can be divided into two types: bottom-up and top-down. The factors of the first type arise from the combination of information from the retina and regions at the beginning of the visual cortex, that is, the attention occurs due to the scene information. On the other hand, the second type's factor is generated by the return signals from areas outside the visual cortex, so attention is also task-dependent [14, 15] .

In 1981, von der Malsburg [18] suggested that each object is represented by the temporal correlation of neural firing activities, which can be described by dynamic models and some can be found at the Cessac's work [5] . Hence, the correlation encodes different attributes of the object. A natural way of representing the coding of the temporal correlation is to use synchronization between oscillators, where each one encodes some attributes of an object [24] [25] [26] , so that the synchronized neurons are the ones that process the attributes of the same object and those that process different objects are out of sync.

The main objective of this research is the studying of synchronization in some biological oscillators' models which exhibit chaotic behaviors. This analysis is to evaluate the synchronization possibility, both complete and per phase, by using a coupling force between oscillators as in Breve et al. work [3] . The motivation of this work is to use this sync method for visual selection of objects that represents sync neurons' models, while the rest of the image is unsynced.

The oscillators used in this work are based upon the biological neural networks which are plausible systems from the biological point of view, as the models of Hodgkin-Huxley, Hindmarsh-Rose, Integrate-And-Fire, and Spike-Response-Model [9, 11, 12, 17] . Discrete models with computational advantages were also used, such as Aihara's, Rulkov's, Izhikevic's and the Courbage-Nekorkin-Vdovin model [1, 7, 16, 23] .

A good model for the visual attention task must allow that a group of oscillators synchronizes with each other if they are strongly coupled, while synchronization is lost compared to other oscillators in the lattice which have a small or even nonexistent coupling force. In this way, oscillators can be used to represent pixels or groups of pixels of an image, just as if they were neurons in the retina, so that neurons representing the same object, synchronize, at the same time different objects lose synchronization.

The present work is organized in the following: in Sect. 2 the theoretical elements will be presented such as the phase synchronization concept and the coupling term in dynamical systems. In Sect. 3 it will be shown the neural dynamical systems used in this work and their analysis, as the search for chaos and the addition of the coupling force in different structures as two oscillators, a vector, and a lattice. In Sect. 4 the results will be discussed, and finally, in Sect. 5 it will be presented the conclusion of which model or models satisfies the methodologies showed in Sect. 3 for a good model of visual attention.

The phase synchronization of two oscillators p and q happens when their phases difference |φ p − φ q | is kept below a certain phase threshold and the amplitudes not necessarily are synchronized (|X p − X q | = 0), this means that their rhythms are bonded. So as t −→ ∞, |φ p − φ q | < C. The phase φ i at time t i is calculated as following [22] 

where k is the number of neural activities prior to time t i , and t k and t k +1 are the last and the next times of neural activity, respectively. So that two oscillators can synchronize with each other, a coupling term is added to the dynamical system as the following:ẋ

whereẋ p j andẋ q j represents the time evolution of the x j state of the oscillators p and q respectively. At right, the F j (.) represents the behaviours' rate of the jth state which depends on the states' vector X = (x 1 , x 2 , ..., x j , ..., x J ) and the parameters' vector μ = (μ 1 , μ 2 , ..., μ l , ..., μ L ), and last the coupling term kΔ p,q , where k is the coupling force and Δ p,q is the difference between the states:

If k is strong enough, then the oscillators X p and X q synchronizes, and it's possible by analyzing their phases difference:

where C is a phase threshold.

The proposed models for the attention system are a two-dimensional network of neural models' dynamical systems with coupled terms. The neural models and their steps in the present work are the following.

This continuous model is ruled by a membrane potential V ij and the ionic channels (m ij , n ij , h ij ), where 1 < i < N and 1 < j < M are rows and columns positions in the lattice, remembering that the coordinate (i, j) is different from the sub-indexes i and j in Sect. 2, which represents the time and states coordinates respectively. So the states and constants will be replaced by a matrix representation as the generic one in 6:

So, the equations of the system are:

where • represents the Hadamard product. The V, M, N and H variables are the states matrices, and μ = (G Na ,

is the parameters matrices vector. Ξ M , Ξ N and Ξ H are white noises as done in [4] , 1l is a matrice of ones, α x , β x are matrix operations as showed in Table 1 : Table 1 . Rate probabilities of M, N and H

The electrical current is [10] :

Finally, K is a matrix of vectors, as much as ΔV, such that an element of k ij multiplying Δv i,j is the same as the expression 13:

which represents the coupling term between the oscillator at position (i, j) and its eight closest neighbors. So for a two-oscillator problem, i = 0 and j = 2, so there's only one neighbor, and for a vector case, i = 0 and 1 < j < M, there are two neighbors. These coupling terms were used in all neural models. The parameters used were the same as in [12] , and its variations study was the same as the range in Table 2 of [12] . For the bifurcation diagram only one oscillator was used with V 0 = 0 and the electrical current varying from zero to 500 µA, while there was no stochastic term in the ionic channels. After that, the stochastic terms were added into the model so that the trajectories are unpredictable with random initializations between 0 and 5 mV. Also, the electrical current variables were μ 0 = 4 µA, σ 0 = 0.001 µA, I syn = −2 µA, D = 10, N (2.0, 1.0) and the step time for the 4th-order Runge-Kutta numerical method were T ≈ 0.007, where T = f −1 , and f = 140 Hz as in [21] , and the ionic stochastic terms were N (0, 0.1). Next, the coupling force was varied from 0.001 to 3.0 for two oscillators, a vector (10 oscillators) and a lattice of 10 × 10 oscillators, the same structure configurations for the other models.

The model is described by the dynamical variables X, Y and Z which represents the membrane potential, the recovery variable and the adaptation current [11] .

where μ = (A, B, C, D, R, S, X r , I) is the vector of parameters and I is the stimuli current. The values used are the same as in [11] and the variations are shown in Table 2 : The bifurcation diagram used the electrical current variation from 0 to 20 µA. Then for the coupling force in a two-oscillator problem, k varied from 0 to 1.2, while for a vector and a lattice it was used k = 5 with a new range of parameters in a lattice showed at Table 3 . In all the 4th-order Runge-Kutta experiments, it was used a step time of 0.01.

Composed by the variable V and the constants μ = (τ, V rest , R, I):

the one linear differential equation dynamical system presents no chaos behaviour, so the current I was replaced by white noise N (2.5, 0.9). The other parameters were set as V rest = 1.0, while R = 1.0 and τ = 10. K varied from 0.001 to 1.1, for a vector-oscillator problem the current was N (3.0, 0.5) and for a lattice N (10.0, 2.0), while K was 0.5 for both structures. The 4th-order Runge-Kutta step was 1 ms.

The simplified model (SRM 0 ) with no external current, only with pre-synaptic stimuli, is described as a lattice of membrane potentials V non-connected with each other as in the other models, but with pre-synaptic neurons l:

the times t (f ) are the times of the last fire of every oscillator v ij , while Ω l is a weight connection between neurons lattice V and the pre-synaptic neurons l, and η and are kernels as:

where ϑ, M, N, D and τ are matrices of constants, while t (f ) l is a matrix of last spikes time of the pre-synaptic neuron l, and t ijl is the spike time of the neuron in position (i, j) of the pre-neuron l. Another constant was added and it was called limit, which means every limit times, a pre-synaptic spike is generated. For two and a vector of oscillators it was used two pre-synaptic neurons. As the model depends on the time and not from the previous v ij value, then it was made two tests, the variation of coupling force from 0.001 to 0.32 and the second test was the absence of the coupling term but the variation of the limit value for different neurons, so for one neuron the interval of time in which the pre-synaptic spikes arrives is 5, and the other is 6. For a vector, the same two tests were made and the coupling force varies from 0 to 0.01, while the other test set several oscillators with the same value of limit and the others were generated randomly.

This chaotic discrete neural model is defined by:

where Y(t) is a neuron internal state and X(t) = F{Y(t)} is a logistic function, while D, B, A, and 1l are constant matrices.

The values were varied as in Table 4 with initialization y(0) = 0.1: The bifurcation diagram was generated with the same parameter values as in [1] . For chaotic trajectories, it was tested several initializations (0.001, 0.01 and 0.1) with A set to 0.35. For the synchronization test, the force was varied from 0.001 to 0.3 for a two-oscillator problem, and for a vector, it was varied from 0 to 0.35, while Y(T 0 ) was generated randomly from 0 to 0.0001 due to its sensibility. Finally, for the lattice, the coupling force varied from 0 to 0.02.

For a spike and spiking-bursts dynamic, Rulkov developed the following system:

where X and Y are the fast and slow variables, while α, I, μ and σ are constants. The parameters were varied according to the Table 5 and X(t 0 ) = 0, Y(t 0 ) = −2.9. The current was set to 0 because σ already represents a stimulus. For chaotic trajectories, α > 4.0 while the other parameters were equal to the ones in [13] . The variation of the coupling force in a two-problem was from 0.0 to 0.5, and for a vector and a lattice was from 0.001 to 0.02. 

The model is described as:

where V and U are variables and A, B, C, D and I are parameters, both dimensionless. The parameters used were the same as in [20] which generates chaos, using initializations of −64 and −65 mV for a two-oscillator coupling while varying the force from 0 to 0.6. For a vector and lattice problem, the force was decreased and increased from 0.06, while the initializations varied from −65 to −65.0001 due to its high sensibility.

Similarly to the Rulkov's model, the CNV model is:

where,

where X and Y are the fast and slow variables, while , β, D (all greater than 0), J, A, M 0 and M 1 are parameters. As in [13] , it was used the burst-spike alternating behavior for the chaos analysis. For the coupling tests, the force varied from 0 to 0.1 in a two-oscillator system, for a vector it varied from 0.05 to 0.5, and finally, in a lattice, the force varied from 0 to 0.01.

At this stage of the present work, the coupling force was varied such that some oscillators were strongly coupled and others weakly, so that the first were synchronized and hence clusterized. This stage was present only in the first three models presented in Sect. 3, so for the Hodgkin-Huxley model, the synchronization force was set to 3 while the desync values were N (0, 0.1). For the Hindmarsh-Rose model, the parameters varied in the intervals shown in Table 6 . While the K for sync was set to 5, the desync was generated by N (0, 0.1). Finally, for the Integrate-and-Fire model, the threshold was set to 2 mV, I = 5 mV, τ = Ξ(10, 20) is a random sample. The step time of the Runge-Kutta was set to 0.5 instead of 1 as in the others simulations, and finally, K for sync was 0.5, and for desync was N (0, 0.01).

In this section the results obtained with the computer simulations regarding the steps and models described in Sect. 3 are presented. To make it possible to identify and to separate the clusters from the rest of the lattice, the neurons must be able to synchronize and desynchronize from each other. They also must be able to represent a great number of different trajectories, so the properties of chaos or even the stochastic ones are good approaches to achieve these objectives.

The models' parameters were varied so that it was possible to find chaotic properties. The models that presented these objectives were the discrete and the Hindmarsh-Rose. The properties of the latter are mentioned in [11] , where the variable Z generates unpredictable trajectories, thus randomly varying the variables, X, Y, and Z from values between −1 and 1, generated unpredictable trajectories. In other models, stochastic terms were added to generate different trajectories, except for the Spike Response Model, whose trajectories depend on the arrival time (parameter limit) of a peak of a presynaptic neuron. These behaviors can be seen in the Figs. 1, 2, 3 and 4 .

Unlike the other models that produced spikes, the Rulkov model generated spike explosions and, in the CNV model, it also produced these behaviors interspersed with spikes. Then, the models were tested if they could synchronize between two neurons of the same models, then in a vector and finally in a grid so that it represented an image. To measure synchronization, the same coupling force was defined between all oscillators (two, a vector and a grid) so that all could synchronize, and then a phase of a reference oscillator would be necessary to calculate the difference between all other oscillator phases. If all oscillators are synchronized, then all differences must be below a certain phase limit, which was determined to be 2π, where the phases of the differences do not increase with time [2] . Firstly, let's analyse the continuous models in Figs. 5, 6 and 7. The Figs. 5a, 6a and 7a shows the difference in the trajectories, however none shows a complete synchronization, otherwise the phase synchronization is clearly observed at the Figs. 5a, 6b and 7b in which the difference of phases evolves in time always below the threshold of 2π. Now, let's analyze the SRM 0 model, which is a particular model of the SRM that there is no external current, only pre-synaptic stimuli from pre-neurons. The model is also an integration of linear dynamical systems [9] , so do not present chaotic characteristics. However, as in the Fig. 8b shows, those trajectories (4, 5, 6, 7, 8) with different limit values of the initial one (0) shows a desynchronization of their phases, while the trajectories with same limit value as the initial, synchronizes with each other, showing that neurons that receive neurons' stimuli at the same time are synchronized in phase, but that leads to a problem, in a case with thousands of neurons that must represent an image, synchronizing a particular amount of them and desynchronizing the other ones, requires that all of them must have And for the discrete models, the experiments showed that for those ranges in the coupling force k presented in Sect. 3, the models did not sync completely or in phase in the reticle structure of neurons. For different values of coupling force above or below those ranges shows trajectories behaviors that are not typical of a relaxation oscillator. So, the desynchronization of the models are showed in Figs. 9, 10, 11 and 12. Finally, for models that successfully synchronized neurons in a network with variable parameters, it was tested whether some neurons could synchronize and others desynchronize by high values of coupling force and low values, respectively, so that the synchronized neurons can be grouped in a way that they can represent an image object in an attention system. The Figs. 13a, 14a and 15a show 9 trajectories, six of them synchronized and three unsynchronized, as in the Figs. 13b, 14b and 15b, so the synchronization represents the pixels of an object that receives attention and the desynchronized ones are pixels that do not receive any type of attention. 

This work proposes to analyze the occurrence of synchronization in oscillators that have chaotic and stochastic behaviors using a coupling force between oscillators. The studies were done to examine if there are possibilities of using of such models in visual attention systems. The models used were those based on biological neural networks such Hodgkin-Huxley, Hindmarsh-Rose, Integrateand-Fire, and Spike-Response-Model (SRM), which are biologically plausible, in addition to discrete-time models such as Aihara, Rulkov, Izhikevic, and

Courbage-Nekorkin-Vdovin (CNV), which have the advantage of reduced computational cost. The behaviors of the models' trajectories were verified by varying their parameters and analyzing which values lead to chaos. Stochastic terms were added so they could produce variability in the trajectories. As a result, a coupling term was applied and analyzed if complete and/or phase synchronization occurred between two identical oscillators (same parameter values). Then the same study was applied, first to a vector of equal oscillators, and later to a lattice, both with equal and variable parameters. Finally, for the models that satisfied the previous steps, tests were made in a lattice with variable parameters and different coupling forces to form a cluster of synchronized and desynchronized oscillators.

From this study it was found that the discrete-time models did not synchronize at all stages, failing in the vector's and lattice's stages. The continuous-time models were able to synchronize at all stages using certain values of coupling force, except for the SRM model which was able to synchronize without the need of a coupling force, only considering the arrival time of presynaptic stimuli. However the SRM model did not present chaos. The continuous models tested for the synchronization and desynchronization for a cluster formation depending on the coupling force showed a potential solution for a visual selection mechanism for an attention system. Finally, for the successful models, the Integrate-and-fire model showed a better execution time with a mean in seconds of 1.16 and a standard deviation of 0.05 s. As a future work, the synchronization of coupled oscillators will also be used for a semi-supervised classification method, in which each cluster of oscillators represents data with the same label.

Chaotic neural networks

Aprendizado de máquina utilizando dinâmica espaçotemporal em redes complexas

Chaotic phase synchronization for visual selection

Synchronization of two Hodgkin-Huxley neurons due to internal noise

A view of neural networks as dynamical systems

Multi attention module for visual tracking

Chaotic oscillations in a map-based model of neural activity

Neural mechanisms of selective visual attention

A framework for spiking neuron models: the spike response model

Spiking Neuron Models: Single Neurons, Populations, Plasticity

A model of neuronal bursting using three coupled first order differential equations

A quantitative description of membrane current and its application to conduction and excitation in nerve

Map-based models in neuronal dynamics

Computational modelling of visual attention

A model of saliency-based visual attention for rapid scene analysis

Simple model of spiking neurons

Recherches quantitatives sur l'excitation electrique des nerfs traitee comme une polarization

The correlation theory of brain function

Recursive visual attention in visual dialog

Analysis of chaotic resonance in Izhikevic neuron model

Noise suppression in a neuronal Hodgkin-Huxley model

Synchronization: A Universal Concept in Nonlinear Sciences

Modeling of spiking-bursting neural behavior using two-dimensional map

Global competition and local cooperation in a network of neural oscillators

A neural cocktail-party processor

The time dimension for scene analysis

Learning unsupervised video object segmentation through visual attention