The validation of Expert System Traffic psychological assessment to Romanian Driving Schools Procedia - Social and Behavioral Sciences 30 (2011) 457 – 464 Available online at www.sciencedirect.com doi:10.1016/j.sbspro.2011.10.090 Procedia Social and Behavioral Sciences Procedia - Social and Behavioral Sciences 00 (2011) 000–000 www.elsevier.com/locate/procedia WCPCG-2011 The validation of Expert System Traffic psychological assessment to Romanian Driving Schools Mihai Aniţeia*, Mihaela Chraifb, Gernort Schuhfriedc, Markus Sommerd a Professor, PhD, University of Bucharest, Faculty of Psychology and Educational Sciences / Bd. M. Kogalniceanu 050107, Bucharest, Romania bPostdoctoral fellow, University of Bucharest, Faculty of Psychology and Educational Sciences, Bd. M. Kogalniceanu 050107, Bucharest, Romania c Schuhfried GmbH,Vienna, Austria dAsistent Professor, PhD, University of Vienna, Vienna, Austria Abstract Analyzing the multiple regression model for the composite criterion, the multiple correlation coefficient, evidence a high and statistically significant correlation between the predictors and the criterion (r=0.741, p<0.05). Also, the beta coefficients provide that the variables of the tests are predictors for the performances registered in traffic (p<0.05). This study based on the findings of the previous research highlight that the Romanian driving schools should improve the psychological assessment batteries with modern and validated instruments. The predictive regression validation model emphasizes the importance of using high performance statistical programs in choosing the psychological tests for evaluation. © 2011 Published by Elsevier Ltd. Keywords: criterion validation, psychological assessment battery, time reaction, tachistoscope test, driving behavior.Introduction 1. Theoretical framework In traffic psychology psychologists are mainly interested in predicting driving behavior using abilities tests (Schuhfried, Sommer, Aniţei & Chraif, 2010; Sommer, Schuhfried, Aniţei & Chraif, 2010) abilities and personality tests (Ulleberg & Rundmo, 2003; Herle, 2009) and autoperception of driving behaviour and risk tendencies (Dula & Ballard, 2003; Aniţei, Chraif, Niculae & Vancea, 2009). Many previous studies in traffic psychological assessment highlight the use of standardized tests in predictive validation of driving behavior (Ulleberg and Rundmo, 2003; Karner & Neuwirth 2001; Sommer, Arendasy, Olbrich & Schuhfried, 2004) cited by Risser, Chaloupka, Grundler, Sommer, Häusler & Kaufmann, 2008). Other researchers highlighted the objections a disadvantageous method because of low correlation coefficients obtained in recent validation studies as a consequence of the lack of correspondence between the generality of the predictor variable and the criterion measure (Sommer, Herle, Häusler, Risser, Schützhofer & Chaloupka, 2008). Thus, many authors, underline classical methods of statistical judgment formation (discriminatory analysis, regression analysis) could be vulnerable and offering less precision (Bortz, * Aniţei Mihai. Tel.: +040721232207 E-mail address: anitei_mihai@yahoo.com 1877-0428 © 2011 Published by Elsevier Ltd. Selection and/or peer-review under responsibility of the 2nd World Conference on Psychology, Counselling and Guidance. Open access under CC BY-NC-ND license. © 2011 Published by Elsevier Ltd. Selection and/or peer-review under responsibility of the 2nd World Conference on Psychology, Counselling and Guidance. Open access under CC BY-NC-ND license. http://creativecommons.org/licenses/by-nc-nd/3.0/ http://creativecommons.org/licenses/by-nc-nd/3.0/ 458 Mihai Anitei et al. / Procedia - Social and Behavioral Sciences 30 (2011) 457 – 464 Mihai Aniţei/ Procedia – Social and Behavioral Sciences 00 (2011) 000–000 2 1999; Brown and Wicker, 2000). Consequently, many methods of validation have been developed. Sommer & Häusler (2005) emphasized the artificial neural network validation as robust methods for pattern recognition tasks with little prerequisites to data characteristics (Bishop, 1995; Rojas, 2000; Warner & Misra, 1996). Risser, Chaloupka, Grundler, Sommer, Häusler & Kaufmann (2008) presented the validation of two tests batteries of the Expert System Traffic using standardized driving tests artificial neural network in calculating the criterion validity. Other methods of validation are based on simulations and animations. The traffic flow model is a discrete, stochastic, time step based microscopic model, with driver-vehicle-units as single entities. The model contains a psycho-physical car following a model for longitudinal vehicle movement and a rule-based algorithm for lateral movements (Fellendorf & Vortisch, 2001). The model is based on the continued work of (Wiedemann, 1974; Wiedemann, 1991). The authors emphasize that both microscopic calibration and macroscopic validation results show that simulation tools based on the psycho-physical car-following model can reproduce traffic flow very realistically under different real-world conditions. Therefore, it is possible but also necessary to adapt the model to the local traffic situation; at least national traffic regulations and driving styles must be taken into account. Adaptation of the model can be based on microscopic data gathered by probe vehicles equipped with electronic sensors or on macroscopic flow data, as it is normally available from measurement sites. Van Tongeren, Gietelink, De Schutter, & Verhaegen (2007) presented a microscopic traffic model for the validation of advanced driver assistance systems. The model describes single-lane traffic and is calibrated with data from a field operational test. The authors used the Monte Carlo simulation of single-lane traffic scenarios with application to cooperative adaptive cruise control system. Another model of validation using simulation is presented by (Pérez Arias, Kretz, Ehrhardt, Hengst, Vortisch & Hanebeck, 2009). This paper presents a novel framework for combining traffic simulations and extended range telepresence which create the impression of being present in a remote environment. The authors showed that real user's position data can thus be used for validation and calibration of models of pedestrian dynamics, while the user experiences a high degree of immersion by interacting with agents in realistic simulations. Starting from previous experimental studies regarding the validation of psychological assessments for driving schools in Romania using both paper and pencil tests and computer designed tests from Vienna Tests System (Aniţei, Mincu & Chraif, 2008; Anitei & Chraif, 2008; Schuhfried, Sommer, Aniţei & Chraif, 2010; Sommer, Schuhfried, Aniţei & Chraif, 2010) and from the validation methodology required at the Methodology Commission from the Romanian College of Psychologists we provide in this paper a model of criterion validation for the Expert System Traffic psychological assessment on Romanian population. 2. The objectives and hypothesis 2.1 The objective The objective is focused on the validation of the Expert System Traffic psychological assessment taking in into consideration the cross-cultural study (Aniţei, Schuhfried, Chraif, Giehl & Buzea, 2010) to predict the driving behavior of the potential Romanian drivers in traffic. 2.2 The hypothesis The independent variables of The Reaction Test, The Determination test, The Tachistoscope test, The Cognitrone test and The Adaptive Matrix test are predictor variables for driving performances in traffic. 459Mihai Anitei et al. / Procedia - Social and Behavioral Sciences 30 (2011) 457 – 464 Mihai Aniţei/ Procedia – Social and Behavioral Sciences 00 (2011) 000–000 3 3. The Method 3.1. The participants The participants were 352 male and female, aged between 18 and 71 years (M=38.15; S.D.= 11.74) from a few driving schools from in Romania, rural and urban areas, different levels of education. 3.2. The instruments  The Adaptive Matrices Test (figure 1) (Schuhfried, 2007). The test shows that the association between the test length and the measurement precision is optimized. This psychological test is a non-verbal test for assessing general intelligence as revealed in the ability to think inductively. The main area of application is organizational psychology, traffic psychology, aviation psychology, educational psychology. The items are somewhat similar to the classical matrices, but in contrast they are constructed on the basis of explicit psycho-logically-based principles involving detailed analysis of the cognitive processes used in solving problems of this type. The items were analyzed using the Rasch dichotomous probabilistic test model and the corresponding characteristic values were estimated for the items [26]. Figure 1. The Adaptative Matrices Test (Schuhfried, 2007)  The Reaction Test (figure 2) (Schuhfried, 2007). This test measures reaction time both as a simple choice and a multiple choice reaction. Yellow light stimulus modalities are available in the test battery, so that different stimulus constellations for the measurement of reaction time can be created. These can range from individual stimuli to simultaneous or sequentially presented stimulus combinations. Figure 2. Time reaction to yellow (Schuhfried, 2007) The use of a rest key and a reaction key makes it possible to distinguish between reaction and motor time. 460 Mihai Anitei et al. / Procedia - Social and Behavioral Sciences 30 (2011) 457 – 464 Mihai Aniţei/ Procedia – Social and Behavioral Sciences 00 (2011) 000–000 4  The Determination test (DT) (figure 3)(Schuhfried, 2007).The test is used to measure time reaction to different visual and auditory stimuli, stress tolerance and the associated ability to react. The test requires the respondent to use his cognitive skills to distinguish different colors and sounds, to memorize the relevant characteristics of stimulus configurations, response buttons and assignment rules and to select the relevant responses according to the assignment rules laid shown in the instructions and learned in the course during the test. Figure 3. The Determination test (Schuhfried, 2007) The difficulty of the DT arises from the need to sustain continuous, rapid and varying responses to rapidly changing stimuli.  Tachistoscopic test (ATAVT) (figure 4 ) [25] (Schuhfried, 2007). The ATAVT tests observational ability by presenting images of traffic situations. The items are constructed using an explicit, theory-led rationale which is based on detailed analysis of the cognitive processes involved in solving the test. Figure 4. The Tachistoscopic test (Schuhfried, 2007) The design of the ATAVT is based on the principles used in the well-known TAVTMB test. The ATAVT test is based on TAVTMB test but it takes into account recent research and findings on the perception of scenes and objects.  The Cognitrone test (figure 5) [25]. The test measure attention and concentration with the support given by the validity of the Rasch model. Numerous validation studies prove the construct and the criterion of validity. Depending on the test form, it can be applied in the field of attention assessment and concentration through the comparison of figures concerning their congruence. 461Mihai Anitei et al. / Procedia - Social and Behavioral Sciences 30 (2011) 457 – 464 Mihai Aniţei/ Procedia – Social and Behavioral Sciences 00 (2011) 000–000 5 Figure 5. The Cognitrone test (Schuhfried, 2007) The test can be applied in the following areas: traffic psychology, aviation psychology, clinical and health psychology, neuropsychology, organizational psychology, sport psychology.  The questionnaire for evaluating a driver’s performance (CEPCA 2008) The questionnaire was designed using as a model a description for evaluating the driver’s performances while driving on route. This evaluation description has been used in the driving school Ilioara during applying the test battery together with the CEPCA questionnaire. The questionnaire for the evaluation of the drivers’ performance while in traffic, named after the initials CEPCA as it has designed and validated as a converged model [27], [28] at the driving school Ilioara, has ten-item Likert scale from 1 (strong disagreement) to 5 (strong agreement). Applied as a test pilot questionnaire, the value of Cronbach Alpha obtained was of 0.736 for the 10 items and a correlation coefficient of 0.637 (p>0.01) by evaluating each individual examined with both questionnaires. Consequently, a pilot study was published previous to validation [23]. 4. The results and discussions After applying the psychological tests from the test battery Expert System Traffic psychological assessment, the data collected have been analyzed with SPPSS 17 programme. Table 1 shows the descriptive statistics of the independent and dependent variables. Table 1. Descriptive statistics, mean and standard deviation Variable Mean Standard deviation 1. Total performance in driving 63.81 12.14 2. TAHITO correct 46.27 11.26 3. TAHITO incorrect 31.93 13.47 4. DT omitted 23.29 15.21 5. DT correct 47.95 14.18 6. DT incorrect 27.16 10.66 7. TR reaction time 49.94 12.83 8. TR motor time 67.82 18.51 9. S.D. reaction time 41.12 15.27 10. S.D. motor time 38.61 18.35 11. Cognitrone 42.36 28.62 12. AMT 3388..1155 2266..7744 The correlation matrix in table 2 reveals the statistically significant correlations between the criteria and the predictor variables (the battery tests). Analyzing the multiple regression model for the composite criterion, the multiple correlation coefficient shows a high and statistically significant correlation between the predictors and the criterion (r=0.741, p<0.05). Also, the beta coefficients show that the variables of the tests are predictors for the performances registered in traffic (p<0.05). 462 Mihai Anitei et al. / Procedia - Social and Behavioral Sciences 30 (2011) 457 – 464 Mihai Aniţei/ Procedia – Social and Behavioral Sciences 00 (2011) 000–000 6 Table 2. The correlation matrix between the independent and dependent variables Variable 1 2. 3. 4. 5. 6. 7. 8. 9 10 11 12 1. Total performance in driving 1.00 2. tahito correct .48** 1.00 3. tahito incorrect -.36** -.47** 1.00 4. DT omitted -.31** -.48** .32** 1.0 5. DT correct .54** .53** -.18* -.44** 1.00 6. DT incorrect -.36** -.11 .29** .28** -.29* 1.0 7. TR reaction time .43** .10 .05 -.21** .11 .19* 1.0 8. TR motor time .47** .07 .26** -.22* .39** -.38** .28** 1.00 9. S.D. react. time .32** .08 .02 .05 .08 -.08 .38** .09 1.0 10. S.D. motor time .27** .07 .09 -.06 .29** .31** .08 .51** .14 1.0 11. Cognitron .37** .22** .08 -.28** .09 -0.10 -.07 .12 -.09 .05 1.0 12. AMT .28** .25** .24** -.23** .38** -.22** .05 .09 ..1122 .11 0.35** 1.00 *p<0.05 and **p<0.01 The correlation matrix (table 2) reveals the statistically significant correlations between the criteria and the predictors (independent variables of the psychological test). Thus, the total performances in traffic, has a statistically significant positive correlation with the following predictors: tahitoscop_corect (.48**), DT corect (.54**), reaction time (.43**), motor time (.47**), cognitrone (.37**) and AMT (.28**). The same criteria have a statistically significant negative correlation with the following independent variables: Dt omitted (-.31**), DT incorrect (-.36**) and tahito incorrect (-.36**). For the criterion total performances in traffic, the following regression model has been applied (table 3): Table 3. The multiple regression model for the dependent variable: total performances in traffic Dependent variable: total performances in traffic Standardized coefficients t p Independent variables β 1 Constant 22.17** 12.54 0.000 3. TAHITO correcte 0.21* 1.78 0.031 4. TAHITO incorrecte -0.42** -2.71 0.000 5. DT omitted -0.061 -0.47 0.482 6. DT correcte incorrect 0.72** 1.43 0.002 7. DT incorrecte correct -0.57* -1.38 0.042 8. TR reaction time 0.69** 1.72 0.003 9. TR motor time 0.54* 1.92 0.02 10. S.D reaction time 0.0049 0.69 0.35 11. S.D. motor time 0.0081 0.53 0.27 12. Cognitron 0.49* 1.378 0.031 13. AMT 00..3388** 11..5533 00..004411 F 7.629** 00..00000011 R 0.741 R2 0.549 R2 Adjusted 0.536 *p<0.05 and **p<0.01 463Mihai Anitei et al. / Procedia - Social and Behavioral Sciences 30 (2011) 457 – 464 Mihai Aniţei/ Procedia – Social and Behavioral Sciences 00 (2011) 000–000 7 As we can be observe in table 3 the regression model explains 54.9% of the variance (R square value). Also that the model is statistically significant (F=7.629; p=0.001) and the R square value is 0.549. The regression equation, obtained from the multiple regression model (table 3) is the following: total performances in traffic = 22.17+0.21*Tahit_correcte-0.42*tahito_incorrecte+ 0.72*DT_correcte- 0.57*Dt incorrect+0.69* TR reaction timet+0.54*TR_motor_time +0.49*Cognitron+0.38*AMT Figure 6. a)The The Histogram for Criteria; b) The scatterplot: predictors total performances in traffic Figure 6 a and b show that there is a strong positive correlation between the observed variables and the expected ones. The regression model has a predictive value for the chosen criteria: right curve, cross road. 5. Conclusions and recomandations The predictive regression validation model emphasizes the importance of using high performance statistical programs in choosing the psychological tests for evaluation. Thus, Expert System Traffic psychological assessment taking into consideration the cross-cultural study (Aniţei, Schuhfried, Chraif, Giehl & Buzea, 2010) to predict the driving behavior of the potential Romanian drivers in traffic was as a predictive model on a Romanian selected sample with age between 18 and 70 years, different levels of education. In addition to the studies previously validated (Schuhfried, Sommer, Aniţei & Chraif, 2010; Sommer, Schuhfried, Aniţei & Chraif, 2010), the tests of peripheral perception, composed time reaction, viziotest and the concentrated attention test have been removed in order to achieve a predictive validation for the test battery Expert System Traffic psychological assessment used in Europe and all over the world as well. Also the cost of the test battery has been taking into account. The negative correlation between these variables Dt_omitted (-.31**), DT_incorrect (-.36**) and tahito_incorrect (-.36**) highlights the fact that participants obtained low performances for the incorrect and omitted tasks, which takes to a higher level of performance in traffic. So, the test battery Expert System Traffic psychological assessment from the computerised psychological assessment company Vienna Test Sytem, standardized and validated in countries such as Austria, Germany, Saudi Arabia, it is also standardized and validated on Romanian population. Analyzing the statistical significance of the beta standardized coefficients (table 3), it can be observed that the independent variables explain the regression model. Thus, we can conclude that psychologists either apply and experimentally validate the psychological testing batteries or use validated and standardized psychological test batteries for the psychological assessment used in driving schools. This study based on the findings of the previous research show that the Romanian driving schools should improve the psychological assessment batteries with modern and validated instruments. The predictive regression validation model emphasizes the importance of using high performance statistical programs in choosing the psychological tests for evaluation. In addition to the results of the individual tests, the Expert System Traffic generates a highly valid overall assessment of the subject's driving-specific ability. Therefore, we can conclude that the Romanian psychologists should select and validate the psychological tests using performance criteria in experimental studies, before using it in driving schools as psychological assessments. 464 Mihai Anitei et al. / Procedia - Social and Behavioral Sciences 30 (2011) 457 – 464 Mihai Aniţei/ Procedia – Social and Behavioral Sciences 00 (2011) 000–000 8 Also, they have to take into consideration that the Standard Quality Driving Curriculum which needs to be kept continuously updated by considering the new developments that technology brings to vehicles and roads, all of which require the acquisition of new skills by drivers. References Aniţei, M., Mincu, C.L. & Chraif, M. (2008). Correlative study between the processing of stimuli in the peripheral perceptual field and speed and distance appreciation in the central perceptual field, The Journal of Human Resources [Revista Psihologia Resurselor Umane], Cluj- Napoca, no. 1. Anitei, M. & Chraif, M. (2008). Designing a test battery for predicting the driver’s behaviour, in the volume of the International Conference with international participation, “Titu Maiorescu” University, Ed. Univ."Titu Maiorescu". Aniţei, M., Schuhfried, G., Chraif, M., Giehl, M. & Buzea, E. (2010). The cross-cultural validation of the psychological tests, [Interkulturelle validierung von tests], in procedeengs of the 6th International Conference of Applied Psychology, Romania, Timişoara, 93-97. Aniţei, M., Chraif, M., Niculae, L. & Vancea, D. (2009). The gender influence in perception driving agresivity, in Proceedings of “Psychology and Society” [Influenţa genului în agresivitatea autopercepută a conducerii autovehicolului “Psihologie şi societate”- Sesiune anuală de comunicări ştiinţifice cu participare internaţională] Ed. Titu Maiorescu. Bishop, C.M. (1995). Neural networks for pattern recognition. Oxford, Oxford University Press, 1995. Bortz, J. (1999). Statistik für Sozialwissenschaftler [Statistics for social scientists]. Berlin, Springer. Brown, M.T. & Wicker, L.R. (2000). Discriminant analysis. In H. E. A. Tinsley & S. D. Brown (Eds.), Handbook of applied multivariate statistics and mathematical modelling, San Diego, CA: Academic Press, 209-234. Dula, C.S. & Ballard, M.E. (2003). Development and evaluation of a measure of dangerous, aggressive, negative emotional and risky driving. Journal of Applied Social Psychology, issue 33, volume 2, 263-282. Fellendorf, M. & Vortisch, P. (2001). Validation of the microscopic trans. model VISSIM in diferent real-world situations. In Proceedings of the Transportation Research Board, Washington, DC. Herle, M. (2009). Personality measurement in traffic psychological assessment. In: W.R. Nickel, G. Meinhard, I. Born (Hrsg). Proceedings zu Fit to drive. 4th International Traffic Expert Congress, 56-61. Bonn: Kirschbaum. Hornke, L.F., Kuppers, A. & Etzel, S. (2000). Konstruction und evaluation eines Adaptativen Matrizentest. Diagnostica, 46, 182-188. W.M.K. Trokim, Social research methods, http://www.socialresearchmethods.net Karner, T. & Neuwirth, W. (2001). Die Bedeutung der peripheren Wahrnehmung in der verkehrspsychologischen Untersuchung. Psychologie in Österreich, 20, 183-186. Pérez Arias, A., Kretz, T., Ehrhardt, P., Hengst, S., Vortisch, P. & Hanebeck, U.D. (2009). A Framework for Evaluating the VISSIM Traffic Simulation with Extended Range Telepresence. In: Proceedings of the 22nd Annual Conference on Computer Animation and Social Agents, 13-16. Pitariu, H. & Chraif, M. (2009). The validation of scales with behavioural anchors, The Psychology Journal, Ed. Academiei Române, issue 3/4. Risser, R., Chaloupka, Ch., Grundler, W., Sommer, M., Häusler, J. & Kaufmann, C. (2008). Using non linear methods to investigate the criterion validity of traffic-psychological test batteries. Accident Analysis & Prevention, Volume 40, Issue 1, 149-157. Rojas, R. (2000). Neuronal Networks, A systematic introduction, Heidelberg, Springer, 2000. Schuhfried, G., Sommer, M., Aniţei, M. & Chraif, M. (2010). The validation methodology of a psychological assessment battery for drivers’ schools in Romania. Romanian Journal of Applied Experimental Psychology, volume 1, issue 1, 5-14. Schuhfried, G. (2007). Handbook of the tests. Vienna Tests System, www.schuhfried.at Sommer, M., Schuhfried, G., Aniţei, M. & Chraif, M. (2010). A model of driving test battery validation. Journal of Psychology, 56, no. 3–4, 243– 250. Sommer, M., Arendasy, M., Olbrich, A. & Schuhfried, G. (2004). Qualitätsverbesserung in der verkehrspsychologischen Diagnostik mit neuronalen Netzen: Eine Pilotstudie. Zeitschrift für Verkehrssicherheit, 50, 193-198. Sommer, M., Herle, M., Häusler, J., Risser,R., Schützhofer, B. & Chaloupka, Ch. (2008). Cognitive and personality determinants of fitness to drive Transportation Research Part F. Traffic Psychology and Behaviour, Volume 11, Issue 5, 362-375. Sommer, M. & Häusler, J. (2005). Non-Linear Methods for the Identification of Drivers at Risk to Cause Accidents. In L. Dorn (ed), Driving Behaviour and Training. Volume II, 425-436, Hamphire, Ashgate. Ulleberg, P. & Rundmo, T. (2003). Personality, attitudes and risk perception as predictors of risky driving behaviour among young drivers. Safety science, 41, 427-43. Van Tongeren, R., Gietelink, O., De Schutter, B. & Verhaegen, M. (2007). “Traffic modelling validation of advanced driver assistance systems,” Proceedings of the 2007 IEEE Intelligent Vehicles Symposium, Vol IV, Istanbul, Turkey, 1246-1251. Warner, B. & Misra, M. (1996). Understanding neural networks as statistical tools, The American Statistician, Volume 50, 284-293. Wiedemann, R. (1974). Simulation des Straßenverkehrsflusses. Schriftenreihe des Instituts für Verkehrswesen der Universität Karlsruhe, Heft 8. Wiedemann, R. (1991). Modelling of RTI-Elements on multi-lane roads. In: Advanced Telematics in Road Transport edited by the Comission of the European Community, DG XIII, Brussels.