Calibration of microsimulation traffic model using neural network approach Expert Systems with Applications 40 (2013) 5965–5974 Contents lists available at SciVerse ScienceDirect Expert Systems with Applications j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / e s w a Calibration of microsimulation traffic model using neural network approach 0957-4174/$ - see front matter � 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.eswa.2013.05.003 ⇑ Corresponding author. Tel.: +385 915220819. E-mail addresses: iirena@gfos.hr (I. Ištoka Otković), tomaz.tollazzi@um.si (T. Tollazzi), matjaz.Sraml@um.si (M. Šraml). Irena Ištoka Otković a,⇑, Tomaž Tollazzi b, Matjaž Šraml b a J.J. Strossmayer University of Osijek, Faculty of Civil Engineering, Drinska 16a, 31000 Osijek, Croatia b University of Maribor, Faculty of Civil Engineering, Smetanova 17, 2000 Maribor, Slovenia a r t i c l e i n f o Keywords: Calibration Microsimulation traffic model VISSIM Neural network Roundabout Validation a b s t r a c t This paper presents the results of research on the applicability of neural networks in the process of com- puter calibration of a microsimulation traffic model. VISSIM microsimulation model is used for calibra- tion done at the example of roundabouts in an urban area. The calibration method is based on the prediction of a neural network for one traffic indicator, i.e. for the traveling time between measuring points. Besides the traveling time, the calibration process further/also involves a comparison between the modeled and measured queue parameters at the entrance to the intersection. The process of valida- tion includes an analysis of traveling time and queue parameters on new sets of data gathered both at the modeled and at a new roundabout. A comparison of the traffic indicators measured in the field and those simulated with the calibrated and uncalibrated microsimulation traffic model provides an insight into the performance of the calibration procedure. � 2013 Elsevier Ltd. All rights reserved. 1. Introduction The functioning of a traffic system is under the influence of var- ious aspects of human behavior (Onieva, Milanés, Villagrá, Pérez, & Godoy, 2012). Studies show that the behavior of traffic participants is, among other things, territorially and culturally conditioned (Ol- stam & Tapani, 2004). Microsimulation models include variable behavior of drivers at a level of each particular entity, and the real- ity of modeling results depends on the initial choice of the model (Fang & Elefteriadou, 2005) and success of the calibration process (Transportation Research Bord, 2000). Calibration of a traffic model is actually an adjustment of a model to a specific local traffic network and its users. Calibration is defined as the process of comparing and minimizing the differ- ences between the modeling results and the real data obtained by counting and measuring in the local network (Transportation Research Bord, 2000). Validation of the model is an estimate of the success of the model calibration made through a comparison of indicators obtained from the calibrated model and the ones measured in traffic. Analysis of bigger spacial and time scopes of a traffic network and prediction of a future state both need a larger number of models, including traffic demand models (Ettema, Arentze, & Timmermans, 2011), dynamic traffic distribution (Flötteröd, Chen & Nagel, 2012), daily mobility (Flötteröd, Bierlaire, & Nagel, 2011), flow prediction under typical and atypical traffic conditions (Castro-Neto, Jeong, Jeong, & Han, 2009) etc. It is realistic to expect that modeling of the selection of a travel mode and users’ routes is connected with the modeling of traffic (Park & Kim, 2001; Rickert & Nagel, 2001), economic and environmental indicators, both for real time modeling and for the prediction of the future traffic demand and its consequences. The degree of reliability of results of model- ing a future state of the system is a subject to a debate (Dorothy, Ambadipudi, & Kill, 2006; Stevanović & Martin, 2008), but scien- tific efforts are directed towards an adoption of reliability criteria in selecting tools in planning and optimization processes. Prediction of selection of users’ routs on the microsimulation le- vel (Flötteröd et al., 2011) and calibrations of traffic distribution model are not included. In this paper, the selection of users’ routs, which is the traffic distribution in an examined segment of the traffic network, is a user defined input parameter of the microsim- ulation model. The aim of this paper is to obtain a calibrated micro- simulation model which is able to give good correlation with measured functional characteristics of a particular segment of the network in real traffic conditions. Genetic algorithm (GA) is the most commonly used calibration algorithm for input parameters of the simulation model. Positive utilization experiences of GA are reported in the following exam- ples: in the calibration of FRESIM model (Cheu, Jin, Ng, Ng, & Srinivasan, 1998), PARAMICS model, VISSIM and CORSIM model (Park, Won, & Yun, 2006) and in the calibration of a VISSIM model (Kim, 2006). Analysis of acceptable time gaps and determination of http://crossmark.dyndns.org/dialog/?doi=10.1016/j.eswa.2013.05.003&domain=pdf http://dx.doi.org/10.1016/j.eswa.2013.05.003 mailto:iirena@gfos.hr mailto:tomaz.tollazzi@um.si mailto:matjaz.Sraml@um.si http://dx.doi.org/10.1016/j.eswa.2013.05.003 http://www.sciencedirect.com/science/journal/09574174 http://www.elsevier.com/locate/eswa Fig. 1. The first (a) and the second (b) examined roundabout. 5966 I. Ištoka Otković et al. / Expert Systems with Applications 40 (2013) 5965–5974 a critical time interval by means of Greenshields’ model (Cicu, Illotta, Bared, & Isebrands, 2011) represent a description of the cal- ibration method of the VISSIM microsimulation model at round- abouts in New York. For the purpose of calibrating a microsimulation traffic model, a new calibration method is analyzed. This paper shows results ob- tained by applying neural networks in the microsimulation traffic model calibration process. The first traffic indicator, chosen in the calibration process, is the traveling time between measuring points. Measuring points were chosen in the real environment and connected to existing ob- jects – a light pole at the beginning of the section and an electric supply pole at the end, because it was easier to enter them into the model layout that way. The traveling time is easily measurable in the field, and it is under the influence of the variable driver’s reaction time. In this paper, the mean traveling time is used in or- der to test the response of neural networks for the given indicator. Furthermore, another easily measurable traffic indicator – queue parameters – is introduced in the calibration process. Validation is done for the traveling time and queue parameters for new sets of measured data. The findings of the comparison of modeled and measured data of the microsimulation traffic model, calibrated to one roundabout, are also shown, and the validation is done on another roundabout. Prediction of traveling time, obtained by application of neural network (Park et al., 1999; Dia, 2001) can be analyzed in a broader context of traffic modeling, but a special emphasis is, in this case, used in microsimulation modeling. The microsimulation model, selected to investigate the applica- bility of neural networks in the calibration process, is the VISSIM.1 Analyses are performed on the basis of results obtained by one-hour traffic simulations, corresponding to the duration of the data gather- ing in the field for every set of data measured in real traffic conditions. 2. Data gathering As the basis for testing the neural network applicability in the process of calibration, the one-lane urban roundabout Vink- ovačka–Drinska in Osijek (Croatia) – roundabout I (Fig. 1a) – was used. The calibration process was done in two steps. The first step is based on the traveling time prediction obtained from the neural network, and the other step introduces queue parameters into the analysis. Maximum queue length and the number of vehicle stop- ping at the entrance into the roundabout, both easily collected un- 1 VISSIM is a stochastic, discrete, micro-simulated (models each entity separately) model designed for traffic analyses. It started to develop in Germany at the University of Karlshruhe in the early 70-ies of the last century. der real traffic conditions, are selected. An estimate of the success of the calibration process is made iteratively also in two steps. The first validation is done at the same roundabout, but for new sets of data, and the second validation is done at a new one-lane round- about Kirova–Opatijska in Osijek – roundabout II (Fig. 1b). Data was gathered, during 2010, at both roundabouts by means of one-hour video camera recordings (Table 1). Both points for measuring the traveling time at both roundabouts are included in the video. Detailed data on the traffic load of the roundabouts for all sets of measured data are available in the dissertation (Ištoka Otković, 2011).2 The results of measured traffic indicators for all sets of mea- sured data are shown in the Table 1 with the calibration data set shaded. The other sets of measured data were used for validation of the calibrated model. The traffic load counted on March 3rd 2010, between 3 and 4 p.m., shows that nearly 1600 vehicles, overall, entered the roundabout. An hour later, one hundred vehicles less en- tered the roundabout. Measuring, done on July 14th 2010, shows that 250 less vehicles entered the roundabout, which reflected on the traffic indicators of the roundabout. Traffic loads of pedestrian flows are not large and this should be taken into account when the results of modeling are interpreted. 3. Calibration of microsimulation model using neural network approach Microsimulation models have a considerable number of model input parameters. Examination of all combinations of input param- eters of the model in their range by means of application of micro- simulation model, in order to find a combination of input parameters that approximates the real traffic situation the best, would be an extremely time-consuming task. A computer can examine a large number of combinations of val- ues of model input parameters in real time, in case that it can ob- tain software simulation output values (e.g. traveling time) of the examined microsimulation traffic model (in this case the VISSIM). The role of a neural network is to predict values of the VISSIM sim- ulations in the process of software examination of combinations of input parameters. In Fig. 2 a simplified scheme of program calibration is shown. The program calibration begins with the creation of a VISSIM simulation database for neural network training (Fig. 2). The task of the neural network is to predict the time of travel between mea- suring points for particular values of input parameters, which would be obtained by the microsimulation model. The program calibration (MATLAB) calls the prediction function, provided by 2 Dissertation is available on-line, in Digital library of the University of Maribor (DKUM). Table 1 Values of measured traffic indicators at examined roundabouts. Roundabout I Roundabout II March 3rd 3–4 p.m. March 3rd 4–5 p.m. July 14th 2–3 p.m. July 14th 8– 9 a.m. Mean value of traveling time (s) 21.8 19.9 18.1 13.3 Maximum queue (m) 26 21 15.5 23 Number of stopping at the entrance 89 61 54 56 Fig. 2. Scheme of computer program calibration. 3 The ratio between the number of input parameters and size of the network training set can be one of the causes of the overtraining. If the number of input parameters is large, and the number of examples in the learning set is small, the network will rather remember, than learn and generalize on the training data set. It should be noted that the ratio between the number of input parameters and the size of network learning set is not the only cause of bad network generalization effect. I. Ištoka Otković et al. / Expert Systems with Applications 40 (2013) 5965–5974 5967 the neural network within the calibration program (subroutine), for each combination of values of input parameters within the gi- ven ranges of values and by a chosen/defined step (Table 2). The preset criterion of comparison of modeled (by neural network pre- diction) and measured values of traveling time according to expression (1) generates a set of potentially optimal combinations of input parameters values into an output file. T MOD � T MEAS T MEAS � � � � � � � � 6 5% ð1Þ where: TMOD is the mean value of modeled traveling time between measurement points, and TMEAS is the mean value of measured trav- eling time between measurement points. The final selection of optimal input parameters from a set of potentially optimal solutions, generated by the calibration pro- gram, is performed based on the simulation results of the original microsimulation traffic model (VISSIM). Validation process of the calibration method is also done (Fig. 2) by application of the real microsimulation model (VISSIM). 3.1. Input parameters of the model The number and the extent of input parameters used in the analysis are directly correlated to the number of possible combina- tions of input parameters necessary to be analyzed in the calibra- tion process. In the process of optimizing the number of model input parameters and their ranges, various statistical methods, de- scribed in detail in literature (Kim, 2006; Park & Qi, 2005), are used. In this paper, neural networks were used for optimisation of input parameters and ranges. Backpropagation type of neural networks can provide a coefficient of influence of every input parameter on the output result, which gives a possibility to select only input parameters influential on the modeled traveling time. Neural net- works have the ability of extrapolation of results, which makes the question of selection of initial ranges of input parameters a less sensitive issue in the process of calibration. The selection of input parameters of the model and their ranges in two steps is described in detail in the dissertation (Ištoka Otković, 2011), and the final re- sult is shown in Table 2. 3.2. Application of neural networks to traveling time prediction The NeuroShell2 software package is used for neural network learning as well as prediction results analysis. The following basic neural networks were tested: Ward Nets, Standard Nets, Jump Connection Nets, Jordan–Elman Nets and General Regression Net (Fig. 3). In total, 70 neural networks, which differed in basic architec- ture, number of hidden layers, number of neurons in hidden layers and activation functions, are analyzed. Neural networks are compared by two basic criteria – training and the ability to generalize. Training is a success of prediction on a training set of data (the set network has learned on), and gen- eralization is the success of prediction on an unknown set of data. A sufficiently large training database partially prevents the net- work overtraining3 effect and creates a precondition for achieving better generalization (Lawrence, Giles, & Tsoi, 1997). For the specific problem, it is possible to create a database for neural network learn- ing of desired size and, therefore, a database of 1379 examples of variations of input parameters and output results of VISSIM simula- tions is generated. From the database, within the NeuroShell2 pro- gram, 20% of data (276 data) is separated in order to be used as a network validation test set. The program offers a possibility of mem- orizing the best result (the best prediction) on a test set of data, as well as memorizing the best result on set of data used for the net- work to learn (training data set). The following indicators were chosen to compare our particular trained networks: correlation coefficient between results of neural network prediction and results of VISSIM simulations, mean abso- lute error and maximum absolute error. The names of all researched networks were coded in order to make them easy-to-survey. The numbers indicate the number of neurons in hidden layers as well as the number of hidden layers (e.g. 13-24-13 is a neural network with three hidden layers having 13 neurons in the first and the third hidden layer and 24 neurons in the second hidden layer). The names of the neural network models included the activation functions of the hidden layers in situations where they differ from the (proposed) default activation functions. The best prediction on the validation (test) data set is the one that is the most often memorized, but there is also the possibility to memorize the best prediction on the training data set which is in this case marked with the ‘‘TR’’ mark. For the General Regression Network, adaptive (AD) and iterative (IT) types of network were analyzed. The comparison of those neural networks that achieved the best prediction results, according to the three chosen criteria, is shown in Table 3. According to the obtained neural network learning results, the best correlation was obtained for the iterative type of regression neural network, as evident from Table 3. Considering the fact that traffic flow modeling is a stochastic problem and that a basic VIS- SIM model uses a random number generator, the achieved correla- tion of 88.3% is satisfactory. In graphs, one set of input data on the abscissa represents one combination of input parameter values (Table 2). One set of values of model input parameters gives one modeled mean traveling time Table 2 Selection of input parameters of the VISSIM calibration model. P Input parameters Value range Step Default value P1 Simulation resolution 1–10 1 5 P2 Number of observed proceeding vehicles 1–4 1 2 P3 Max look ahead distance (m) 100–300 1 250 P4 Min look ahead distance (m) 0–20 1 0 P5 Average standstill distance (m) 1–3 0,1 2 P6 Additive part of desired safety distance (m) 1–5 0,1 2 P7 Multiplicative part of desired safety distance (m) 1–6 0,1 3 P8 Desired speed (km/h) 25–50 10 40* * Expected (desired) travel speed at access lanes of the observed urban intersection. The selected range is designed based on measured velocity on access roads of the observed intersection. ADAPTIVE TYPE ITERATIVE TYPE WARD NETS STANDARD NETS JUMP CONNECTION NETS JORDAN-ELMAN NETS GENERAL REGRESSION NET BEST PREDICTION RESULTS FOR TRAINING DATASET BEST PREDICTION RESULTS FOR TEST DATASET Fig. 3. Examined types of neural networks. Table 3 Performance indicators of trained neural networks – networks with a good response. Type of neural network Correlation coefficient Mean absolute error Maximum absolute error Ward Nets-3 hidden layers 13-18-13 0.8270 0.332 3.000 Ward Nets-3 hidden layers 13-18-20 0.8286 0.331 3.132 Ward Nets-3 hidden layers 13-24-13 0.8309 0.330 3.201 Standard Nets-3 hidden layers13-13-13 0.8355 0.333 3.000 Standard Nets-2 hidden layers 19-19-tanh-logist 0.8208 0.292 3.200 Standard Net-3 hidden layers 26-18-15-tanh-logist 0.7955 0.262 4.421 Standard Nets-3 hidden layers 30-20-18-tanh-logist 0.8199 0.280 3.000 Standard Nets-3 hidden layers 18-15-15-TR 0.8480 0.306 3.000 Standard Nets-3 hidden layers 20-15-13-tanh-logist-TR 0.7881 0.288 4.100 Standard Nets-2 hidden layers 19-24-tanh 0.8402 0.334 3.027 Standard Nets-1 hidden layer 38 0.8378 0.328 3.000 General Regression Net-1 hidden layer 2200tanh-AD 0.8554 0.311 3.000 General Regression Net-1 hidden layer 2380logist-AD 0.8491 0.322 3.000 General Regression Net-1hidden layer 2480tanh-AD 0.8554 0.311 3.000 General Regression Net-1hidden layer 2550-AD 0.8501 0.317 3.000 General Regression Net-1hidden layer 3500tanh-AD 0.8581 0.308 3.000 General Regression Net-1hidden layer 3500tanh-IT 0.8811 0.285 3.328 General Regression Net-1hidden layer 5500logist-AD 0.8469 0.323 3.001 General Regression Net-1hidden layer 5500logist-IT 0.8830 0.278 3.432 Jump Connection Nets-2 hidden layers 19-19 0.8374 0.326 3.000 Jordan–Elman Nets-2 hidden layers 38-38-tanh-logist 0.8293 0.334 3.000 5968 I. Ištoka Otković et al. / Expert Systems with Applications 40 (2013) 5965–5974 between measuring points depicted on the ordinate (Figs. 4–7). In Fig. 4, different combinations of input parameter values (1379 combinations) is depicted on the abscissa, and the ordinate shows the corresponding modeled mean traveling time for every set of in- put data. Traveling time obtained from the General Regression Net- work (Table 3) is compared to results of the VISSIM simulations for 1379 different combinations of values of input data used for neural network learning – training data set (Fig. 4). Besides the software testing of the generalization ability, a two step independent estimate of neural networks is also designed. A set of 25 combinations of new input parameters was used for a comparison of results of generalization of all trained networks. Traveling time prediction provided by the neural network was compared with the traveling time obtained from the VISSIM model. A part of results obtained from Ward Net and Standard Net neural network in the first step of an independent validation is shown in the Fig. 5 and those from the General Regression Network is pre- sented in the Fig. 6. From graphs in Figs. 5 and 6, it is noticeable that the General Regression Networks usually give better generalization results (Fig. 6), than Ward and Standard Net neural network types (Fig. 5). The final validation is done on the basis of data received from 64 new combinations of input parameters, for networks which have shown the best results in the trial validation (Fig. 7). Comparison and assessment are made according to the criterion of minimal mean absolute error. Prediction error is an absolute value of Fig. 4. VISSIM simulations vs. neural network GR 5500logist IT – training data set. Fig. 5. Traveling time prediction for a part of examined neural network. I. Ištoka Otković et al. / Expert Systems with Applications 40 (2013) 5965–5974 5969 difference between neural network prediction results and the expected value obtained by VISSIM simulation. The General Regression Network of the iterative type with a one hidden layer, 5500 neurons in that layer and logistic function of activation, with the approximate configuration depicted in the Fig. 8, had the best correlation with the mean traveling time ob- tained by the VISSIM simulation model. The same network has achieved the best generalization result in both steps of the inde- pendent estimate of neural networks, as it is described in detail and presented in the dissertation (Ištoka Otković, 2011), and there- fore it was chosen for traveling time predictions (replacement for the VISSIM) in the program calibration process (Fig. 2). Schematic overview of the configuration of the chosen neural network is shown in the Fig. 8. 3.3. Traffic microsimulation model calibration program The mean traveling time measured in the field, for the calibra- tion data set, is 21.8 s (Table 1). The mean traveling time obtained by VISSIM simulation, with default values of input parameters, is Fig. 6. Test prediction for adaptive and iterative type of General Regression Networks. 18 19 20 21 22 23 24 25 0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 Sets of input dana VISSIM GR 5500logist IT SN 13-13-13 TR GR 3500tanh AD Fig. 7. Prediction results of the best neural networks on the validation set of data. 5970 I. Ištoka Otković et al. / Expert Systems with Applications 40 (2013) 5965–5974 20.3 s. The aim of the calibration procedure is to find such combi- nation of values of input parameters, which will approximate the mean measured traveling time. The calibration computer program for the analyzed problem is written in MATLAB programming language. The algorithm, a de- tailed description of the program and the calibration program itself are available in Dissertation (Ištoka Otković, 2011). The basic steps of the program are shown in Fig. 2. For every combination of values of input parameters (Table 2) the program calls the prediction function given by the neural network as a subroutine. A practical problem, which occurred at this research stage, was the connection of neural network prediction function and the MATLAB program. Fig. 8. Scheme of artificial neural network for traveling time prediction. I. Ištoka Otković et al. / Expert Systems with Applications 40 (2013) 5965–5974 5971 Since this was a large output file, manual programming of the pre- diction function in MATLAB format was a time-consuming job. Therefore a special MATLB program, which used the output predic- tion function of the trained neural network in MATLAB format (Ištoka Otković, 2011), had been designed. According to the criterion set in the expression (1), the program generated a set of potentially optimal combinations of input parameter values, and the final selection of the optimal combina- tion was done by application of the original microsimulation traffic model (VISSIM), analysis of traveling time and queue parameters for the existing and new sets of measured data. 3.4. Analysis of program calibration results Analysis of the output file of the calibration program of poten- tially optimal input parameters combinations makes it obvious that there is a certain number of combinations with the maximum values of input parameters P5, P6 and P7 (Table 2). Maximum val- ues of those input parameters influence the modeled capacity of the intersection. Results of the simulation in the VISSIM show that the VISSIM does not manage to generate the overall traffic load in the designated time, which indicates that the capacity of the mod- eled section is smaller than the real one. Generally, the neural net- works gave the correct setting, but realistically, combinations of input parameters P5, P6 and P7 with a maximum value or a value Fig. 9. Output data-file of calibration program co close to the maximum, in the combination, significantly reduce the capacity in comparison to the real one and in this case they are not applicable. The output file analysis procedure is described in detail in the dissertation, but when the combination of input parameter with maximum values (P5, P6, P7) is left out, matching of the results of the output file and VISSIM simulations (80% of the results has less than 5% difference), is seen in Fig. 9. The final selection of values of model input parameters, which will make the best approximation of local traffic conditions (cali- brated model), was made through the second step of calibration, by the comparison of modeling results to measured values of trav- eling time and queue parameters at the examined roundabout. 3.5. Queue parameters During the second step of calibration a set of potentially opti- mal combinations of input parameters is analyzed in the context of VISSIM simulation values obtained for queue parameters – max- imum queue length and number of stopping at the entrance into the roundabout. When the results of the modeling in the VISSIM for 80 combinations of values of input parameters are compared with results of traffic indicators measured in the field according to expression (2), 13 combinations of input parameters, which sat- isfy the criterion set for three examined traffic indicators, are ob- tained. In the real traffic conditions, 21.8 s mean traveling time, maximum queue length of 26 m and the number of stopping at the entrance of 89 are measured (Table 1). T MOD � T MEAS T MEAS � � � � � � � � and Q maxMOD � Q maxMEAS Q maxMEAS � � � � � � � � and STOPMOD � STOPMEAS STOPMEAS � � � � � � � � 6 5% ð2Þ where: TMOD is the mean value of modeled traveling time between measurement points, and TMEAS is the mean value of measured trav- eling time between measurement points, and QmaxMOD mean value of modeled maximum queue at the entrance, and QmaxMEAS is the mean value of measured maximum queue at the entrance, and STOPMOD is the mean value of the modeled number of stopping at the entrance, and STOPMEAS is the mean value of the measured num- ber of stopping at the entrance. In Table 4 those combination values of input parameters that were the closest to the measured values in the field are shaded. mpared with the VISSIM simulation results. Table 4 Calibration by the other traffic indicator (1st set of measured data). Input parameters Neural network VISSIM P1 P2 P3 P4 P5 P6 P7 P8 T T Qmax STOP 6 4 164 8 2.4 2 3.3 40 21.267 20.8 25 86 6 4 172 8 2.4 1.9 3.4 40 21.303 20.8 25 90 6 4 172 8 2.4 1.9 3.5 40 21.295 20.9 25 89 6 4 172 8 2.5 1.9 3.3 40 21.295 20.8 25 86 6 4 172 12 2.4 1.9 3.4 40 21.271 20.8 25 90 6 4 172 12 2.4 1.9 3.5 40 21.264 20.9 25 89 6 4 172 12 2.5 1.9 3.3 40 21.283 20.8 25 86 6 4 173 8 2.4 1.9 3.4 40 21.298 21.1 25 85 6 4 173 12 2.4 1.9 3.4 40 21.298 21.1 25 85 6 4 175 8 2.5 1.9 3.5 40 21.262 21.0 25 85 6 4 176 8 2.4 1.8 3.5 40 21.275 21.4 27 88 6 4 176 8 2.5 1.8 3.4 40 21.273 20.9 27 90 6 4 176 12 2.5 1.8 3.4 40 21.273 20.9 27 90 5972 I. Ištoka Otković et al. / Expert Systems with Applications 40 (2013) 5965–5974 Two bolded combinations of input parameters, which present the best result, entered the validation procedure. The value of simu- lated time of 21.4 s was the best result, and, for the second combi- nation, one out of two practically the same combinations was selected. VISSIM, unlike neural networks, provides the same values of traffic indicators (P4 = 8 and P4 = 12), thus one of two combina- tions that differ only by the fourth input parameter P4, was selected. 4. Validation of the calibrated model The validation procedure is comprised of two steps and it in- volves a comparison of simulation results obtained by an uncali- brated and a calibrated VISSIM microsimulation model with traffic indicators measured in the field. The first validation was done for two new sets of data measured at the same roundabout where the calibration is done (roundabout I). The second valida- tion, done on the set of data measured at the second roundabout (roundabout II), gives an insight into the dilemma whether the cal- ibration process is tied to the one particular roundabout or it may be considered in a broader context. 4.1. First validation The first validation was performed at the same roundabout at which calibration procedure was carried out. The first validation data set of traffic parameters was measured on Wednesday, March 3rd 2010, between 4 and 5 p.m., and the second one was gathered on Wednesday, July 14th 2010, between 2 and 3 p.m. Table 5 Comparison of traffic indicators – first validation. Input parameters of VISSIM model P1 P2 P3 P4 P5 P6 Measured values First validation data set Second validation data set Values obtained by VISSIM model simulations Default values of input parameters – first validation measured data set 5 2 250 0 2 2 Default values of input parameters – second validation measured data set 5 2 250 0 2 2 Calibrated model – first validation measured data set 6 4 176 8 2.4 1.8 6 4 173 8 2.4 1.9 Calibrated model – second validation measured data set 6 4 176 8 2.4 1.8 6 4 173 8 2.4 1.9 In Dissertation (Ištoka Otković, 2011), data on traffic volume and traffic distribution in the roundabout for all performed count- ing and the measured traffic parameters are presented and a single measured passing time between the measurement points for each vehicle in the reference period for all counting was given. It is evident from the Table 5 that default values of input param- eters have provided traffic parameters which did not meet one of the three indicators, i.e. maximum queue differs for 28% from the measured values. The first combination of calibrated model input parameters satisfied all three criteria with an error smaller than 5% for the first measured data set. Both parameter combinations, obtained by calibration procedure (shaded), satisfied the limit set by expression (2) approaching the values measured in the field for the second data set. 4.2. Second validation The second validation was performed at the new location, at the second observed roundabout (roundabout II). Data were collected and traffic parameters measured, on Wednesday morning, July 14th 2010, between 8 and 9 a.m. A graphic presentation of the main roundabout project and measured data of traffic volume and traffic distribution, given in Dissertation (Ištoka Otković, 2011), served as the background for the second validation. VISSIM simulation values for input parameters default values and the two best parameter combinations were compared with measured values of traffic parameters and presented in the Table 6. Default values of input parameters provided simulation values of traveling time within specified limits, but the queue parameter Traffic indicators P7 P8 T Qmax STOP 19.9 21 61 18.1 15.5 54 3 40 20.3 15 60 3 40 17.6 23 50 3.5 40 19.8 22 58 3.4 40 20.2 15 63 3.5 40 17.6 15 52 3.4 40 17.8 15 53 Table 6 Comparison of traffic indicators – second validation, second roundabout. Input parameters of VISSIM model Traffic indicators P1 P2 P3 P4 P5 P6 P7 P8 T Qmax STOP Measured values 13.3 23 56 Values obtained by VISSIM model simulations Default values of input parameters 5 2 250 0 2 2 3 40 13.1 27 50 Calibrated model – third validation measured data set 6 4 176 8 2.4 1.8 3.5 40 13.1 22 58 6 4 173 8 2.4 1.9 3.4 40 13.1 22 54 I. Ištoka Otković et al. / Expert Systems with Applications 40 (2013) 5965–5974 5973 results were not within the expected range. The simulation results showed 26% greater maximum queue value and 10.7% lower value for the number of stopping at the roundabout entrance. Combina- tions of input parameters, obtained by calibration procedure, were within limits set by expression (2) and are shaded in the table. By analyzing and comparing the results of values of traffic parameters shown in Tables 4–6, the combination of input parameters, that satisfied all the observed indicators within the pre-set statistical limits, according to expression (2), can be ob- served. The first combination, obtained by calibration procedure, achieved the optimal combination of values of input parameters for single-line roundabouts in local traffic conditions shown in Table 7. 5. Discussion In the dissertation (Ištoka Otković, 2011) several calibration methods were analyzed using computer software calibrations based on the following: – Prediction of a neural network for a traffic indicator – traveling time between measuring points; – Prediction of a neural network for three traffic indicators – time of travel between measuring points, maximum length of a queue and number of stopping at the entrance into a roundabout; – Three independent neural networks which give three indepen- dent predictions for the examined traffic indicators. This paper describes in details the method, which provided the best result. This method is based on the prediction of one neural network for the traveling time between measuring points. Results presented in this paper clearly show that a neural net- work is not an alternative to a microsimulation model. The neural network achieved accurate results of traveling time prediction, but in this research no neural network gave good results for queue parameter prediction. This paper does not implement neural net- works in the prediction of other traffic indicators obtained by means of microsimulation model. A database of 1379 combinations of values of the model input parameters and the simulation traveling time results made in the VISSIM is used for neural network learning. Such a large database is used in order to investigate the real neural networks response and in order to prevent the network overtraining or lack of vari- Table 7 Optimal values of VISSIM model input parameters. Optimal values of VISSIM model input parameters P1 P2 P3 P4 P5 P6 P7 P8 6 4 176 8 2.4 1.8 3.5 40 ability in the network database to be the causes of possible bad results. It is presumed that such a large database is not necessary for achieving optimal results of neural network learning. The calibration computer program calls the neural network pre- diction as a subroutine and each combination of values of input parameters that meet the set statistical criteria – expression (1), is entered in the output file. Upon the analysis of the calibration program output file in VIS- SIM, 80 potentially optimal combinations of input parameters, which satisfy the criterion (1) set for the traveling time, are ob- tained. In the second step, 80 combinations were analyzed by applying the VISSIM, and in the context of the second traffic indi- cator, in order to choose combinations of values of input parame- ters which provide good simulation results for both traveling time and queue parameters. 13 combinations, which satisfy the condition set in accordance with expression (2), were obtained. Two combinations, which were the best in approximating values measured in the field on the calibration data set, were selected out of those thirteen combinations. The validation procedure was done in two steps, and traveling time and queue parameters, obtained by calibrated and uncali- brated traffic model, were compared to values measured in the field. In the first validation, a new data set, measured at the same roundabout for which the model calibration was done, was com- pared (Table 5). From Table 5 it is obvious that the calibrated mod- el gives simulation results for traveling time between measuring points and queue parameters closer to values measured in the field, than the uncalibrated model with default values of input parameters of the model. The second validation is done at the other roundabout (Table 6). Results, obtained by the comparison of modeling results and traffic indicators measured in the field, show that better results are reached with the model calibrated on the second roundabout of the same type in the local traffic network than with the uncali- brated model. The results in Table 6 show that the calibrated microsimulation model is applicable to the one-lane roundabouts in a local condition and that it is not necessarily related to the loca- tion of the intersection where the calibration is made. Such a con- clusion should not be taken too seriously, given that the research was based on two one-lane roundabouts. Possibility of generalized application of the obtained values of input parameters to other types of roundabouts or intersections in a local traffic network in general should be additionally explored. However, such an opinion is not unfounded, because most of calibrated input parameters are related to the behavior (psychology) of driver, which is to some ex- tent territorially and culturally conditioned. The combination of model input parameters, which approxi- mates local traffic conditions the best, is selected by the analysis of achieved results, presented in Tables 4–6. Optimal combination of values of model input parameters (Table 7) represents the cali- brated microsimulation traffic model for one-lane roundabouts in the examined local network. 5974 I. Ištoka Otković et al. / Expert Systems with Applications 40 (2013) 5965–5974 6. Conclusion Within this paper, a new calibration method encompassing the application of a neural network for the prediction of the results of simulations of microsimulation traffic model within the computer calibration program is analyzed. Traveling time and queue parameters are the traffic indicators which are analyzed in the process of calibration and validation of the model, because they are easily measurable in real traffic condi- tions. The microsimulation model selected for the study of the applicability of neural networks in the calibration procedure is the VISSIM, and two urban roundabouts were used as the basis of the experiment. Results have shown that a neural network is applicable in the process of calibration of the examined microsimulation model. Basic steps of the calibration method by means of neural net- work application include: – A minimum of two sets of measured data (the first for the model calibration and the second for the validation), which are: vehicle and pedestrian traffic load, traffic distribution, mean traveling time between two chosen measuring points easy to enter into the model layout, queue parameters at one of the entrances into the roundabout; – Creating the VISSIM simulation database (variations of values of input parameters) for neural network learning; – Selection of neural network (General Regression Net, Iterative type, with a logistic activation function gave the best response in this study) and obtaining the function of prediction for the program calibration; – Program calibration (program designed in MATLAB); – Checking the results of the program calibration output file in the VISSIM model for the mean traveling time and queue parameters as well as checking the best result in the validation procedure; – Validation procedure involves comparison of simulated and measured data for the second set of data measured in the field; – Combination of input parameters, that gives the simulation results which best approximate the measured data, is opti- mal = calibrated VISSIM. – The described basic steps of calibration are applicable, not only in the VISSIM, but in other microsimulation models. The question about the optimal size of database for microsimu- lation modeling of neural network learning and the question about a possibility of finding a configuration of neural network which would provide a better response than the chosen General Regres- sion Network both remain unanswered, since these answers re- quire further research. Regarding the optimisation issues, the question is whether the obtained optimum is local or absolute minimum and how good should possible local minimum approximate the absolute one. In practical terms, the comparison of the measured field results with simulation results obtained with calibrated and default values of input parameters, gives a basic insight into the performance of the calibration procedure. References Castro-Neto, M., Jeong, Y. S., Jeong, M. K., & Han, L. D. (2009). Online-SVR for short- term traffic flow prediction under typical and atypical traffic conditions. Expert Systems with Applications, 36, 6164–6173. Cheu, R. L., Jin, X., Ng, K. C., Ng, Y. L., & Srinivasan, D. (1998). Calibration of FRESIM for a Singapore expressway using genetic algorithm. Journal of Transportation Engineering, 124(6), 526–535. Cicu, F., Illotta, P. F., Bared, J., & Isebrands, H. (2011). VISSIM calibration of roundabouts traffic performance, In Transportation research board annual meeting 2011. Washington DC. Dia, H. (2001). An object-oriented neural network approach to short-term traffic forecasting. European Journal of Operational Research, 131, 253–261. Dorothy, P., Ambadipudi, R., & Kill, R. (2006). Development and validation of large- scale microscopic models. In Transportation research board 85th annual meeting 2006, Paper#06-2392. Washington DC. Ettema, D., Arentze, T., & Timmermans, H. (2011). Social influences on household location, mobility and activity choice in integrated microsimulation models. Transportation Research Part A, 45, 283–295. Fang, F. C., & Elefteriadou, L. (2005). Some guidelines for selecting microsimulation models for interchange traffic operational analysis. Journal of Transportation Engineering, 131(7), 535–543. Flötteröd, G., Chen, Y., & Nagel, K. (2012). Behavioral calibration and analysis of a large-scale travel microsimulation. Networks and Spatial Economics, 12(4), 481–502. Flötteröd, G., Bierlaire, M., & Nagel, K. (2011). Bayesian demand calibration for dynamic traffic simulations. Transportation Science, 45(4), 541–561. Ištoka Otković, I. (2011). Using neural networks in the process of calibrating the microsimulation models in the analysis and design of roundabouts in urban areas, Ph.D. thesis, University of Maribor, Faculty of Civil Engineering; . Kim, S.J. (2006). Simultaneous calibration of a microscopic traffic simulation model and OD matrix, Ph. dissertation, Texas A&M University, USA. Lawrence, S., Giles, C. L., & Tsoi, A. C. (1997). Lessons in neural network training: Overfitting may be harder than expected. In Proceedings of the fourteenth national conference on artificial intelligence, AAAI-97 (pp. 540–545). Menlo Park, California: AAAI Press. Olstam, J. J., & Tapani, A. (2004). Comparison of queue models, Swedish national road and transport research institute, Project VTI meddelande 960 A, Linköping, Sweden. Onieva, E., Milanés, V., Villagrá, J., Pérez, J., & Godoy, J. (2012). Genetic optimization of a vehicle fuzzy decision system for intersections. Expert Systems with Applications, 39, 13148–13157. Park, B., & Qi, H. (2005). Development and evaluation of simulation model calibration procedure. 84th annual meeting 2005, reprint CD-ROM. Washington, DC: Transportation Research Board. Park, B., Won, J., & Yun, I. (2006). Application of microscopic simulation model calibration and validation procedure: A case study of coordinated actuated signal system, Transportation Research Record, TRB, National Research Council, Washington, DC. Park, D., Rilett, L. R., & Han, G. (1999). Spectral basis neural networks for real-time travel time forecasting. Journal of Transportation Engineering, 125(6), 515–523. Park, K., & Kim, W. (2001). A systolic parallel simulation system for dynamic traffic assignment: SPSS-DTA. Expert Systems with Applications, 21, 217–227. Rickert, M., & Nagel, K. (2001). Dynamic traffic assignment on parallel computers in TRANSIMS. Future Generation Computer Systems, 17, 637–648. Stevanović, Z., & Martin, P. T. (2008). Assessment of the suitability of microsimulation as a tool for the evaluation of macroscopically optimized traffic signal timings. Journal of Transportation Engineering, 134, 59–67. Transportation Research Bord (2000). Highway Capacity Manual, Transportation Research Board, National Research Council, Washington, DC, USA. http://refhub.elsevier.com/S0957-4174(13)00289-3/h0005 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0005 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0005 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0010 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0010 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0010 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0015 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0015 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0020 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0020 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0020 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0025 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0025 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0025 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0065 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0065 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0065 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0030 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0030 http://dkum.uni-mb.si/Iskanje.php?type=napredno&stl0=Avtor&niz0=Irena+I%C5%A1toka+Otkovi%C4%87 http://dkum.uni-mb.si/Iskanje.php?type=napredno&stl0=Avtor&niz0=Irena+I%C5%A1toka+Otkovi%C4%87 http://dkum.uni-mb.si/Iskanje.php?type=napredno&stl0=Avtor&niz0=Irena+I%C5%A1toka+Otkovi%C4%87 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0035 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0035 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0035 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0035 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0040 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0040 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0040 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0045 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0045 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0045 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0070 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0070 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0050 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0050 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0055 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0055 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0060 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0060 http://refhub.elsevier.com/S0957-4174(13)00289-3/h0060 Calibration of microsimulation traffic model using neural network approach 1 Introduction 2 Data gathering 3 Calibration of microsimulation model using neural network approach 3.1 Input parameters of the model 3.2 Application of neural networks to traveling time prediction 3.3 Traffic microsimulation model calibration program 3.4 Analysis of program calibration results 3.5 Queue parameters 4 Validation of the calibrated model 4.1 First validation 4.2 Second validation 5 Discussion 6 Conclusion References