key: cord-0430991-ffy0abd6 authors: Campo-Arias, A.; Caamano-Rocha, L. E.; Pedrozo-Pupo, J. C. title: Spanish Version of the Attitude Towards COVID-19 Vaccines Scale: Reliability and Validity Assessment date: 2021-07-22 journal: nan DOI: 10.1101/2021.07.18.21260733 sha: 5f6688f081900f0c2e24e7f96ee04821d2df3709 doc_id: 430991 cord_uid: ffy0abd6 Background and purpose: The negative attitude to vaccines for coronavirus disease (COVID-19) has motivated the adaptation of instruments for this specific purpose. However, details of the reliability and validity of these scales are unknown. The study aimed to evaluate some indicators of the reliability and validity of the Spanish version of the Attitude towards COVID-19 Vaccines Scale. Methods: A validation study was carried out with 1,136 students of emerging age (18 and 29 years) from a Colombian university; 65.5% were female. Cronbach's alpha and McDonald's omega were calculated for reliability, and exploratory and confirmatory factor analyzes for validity. Additionally, the gender differential item functioning (DIF) was estimated with Kendall's tau b. Results: The Spanish version of Attitude towards COVID-19 Vaccines Scale showed high internal consistency (Cronbach's alpha of 0.94 and McDonald's omega of 0.95), a one-dimensional structure with acceptable goodness-of-fit indicators (CFI=0.94, TLI=0.91, and SRMR=0.04), and non-gender DIF (Kendall's tau b between 0.02 and 0.06). Conclusions: The Spanish version of the Attitude towards COVID-19 Vaccines Scale presents some appropriate reliability and validity indicators among university emerging adults. These findings should be explored in samples with other characteristics. Over the past two decades, the negative attitude movement towards a particular vaccine or national vaccination plan has grown worldwide (Dubé et al., 2015; Zaidi & Flores-Romo, 2020) . This general negative attitude to vaccines, at present, includes vaccines against coronavirus disease (García & Cerda, 2020; Harrison & Wu, 2020) . Several researchers have studied some variables related to aversion, distrust, hesitance, or negative attitude towards vaccination against COVID-19 (Fisher et al., 2020; Freeman et al., 2020; Paul et al., 2020) . These investigations showed that COVID-19 vaccine rejection varied widely, between 3% and 55% (Feleszko & Lewulis, 2021; Fisher et al., 2020; Freeman et al., 2020; Lin et al., 2021; Murphy et al., 2021; Paul et al., 2020; Pogue et al., 2020; Prati, 2020; Robinson et al., 2021; Salili et al., 2020; Williams et al., 2021) . These differences can be explained for three reasons: the first, various instruments to measure attitudes towards COVID-19 vaccines (Lin et al., 2021) . The second, limited information on the instruments' reliability and validity, and only Cronbach's alpha was reported for internal consistency (Freeman et al., 2021) . The third, because cultural, social, and political differences significantly affect the acceptance of COVID-19 vaccination (García & Cerda, 2020; Harrison & Wu, 2020) . Moreover, the fourth, the first investigations were carried out at different stages of developing the COVID-19 vaccines, as the vaccine went from being just a possibility to a closer reality (Lin et al., 2021) . The attitude towards COVID-19 vaccination has been evaluated with few questions or questionnaires (Feleszko & Lewulis, 2021; Fisher et al., 2015; Graffigna et al., 2020; Murphy et al., 2021; Prati, 2020; Robinson et al., 2021; Salili & Uysal, 2020) or type scale instruments (De Roos, 2020; Freeman et al., 2020) . There are available two scales for rating COVID-19 vaccine trust or negative attitude: The Attitude towards COVID-19 Vaccines Scale (8 items) (De Roos, 2020) and Oxford COVID-19 Vaccine Hesitancy Scale (15 items) (Freeman et al., 2021) . These scales were adaptations of other existing ones to measure general attitude towards vaccines, such as the 12-item Vaccination Attitudes Examination Scale (Martin & Petrie, 2017) and the 7-item Vaccine Conspiracy Beliefs Scale (Shapiro et al., 2016) . For the present study, the Attitude towards COVID-19 Vaccines Scale was chosen because it had good validity and reliability (De Roos, 2020) . The Attitude towards COVID-19 Vaccines Scale comprises eight items modified of the Parent Attitudes about Childhood Vaccines Scale (Opel et al., 2021) and Vaccine Hesitancy Scale (Larson et al, 2015) . In a sample of Danish adults between 18 and 71 years old, 73% women, the Attitude towards COVID-19 Vaccines Scale presented high internal consistency (Cronbach's alpha of 0.95), good inter-item correlations between 0.47 and 0.82, and acceptable correlation with an external measure of trust in government agencies (r=0.38). However, information on the instrument's dimensionality and the differential item functioning (DIF) for some interest variables was omitted (De Roos, 2020) . It is necessary to have valid and reliable indices and scales that guarantee precision measurements in clinical and population studies (Terwee et al., 2007) . These instruments should also measure the exact nuances of the construct in different contexts, populations, and languages since these measurements allow better comparisons around the world (Borsboom et al., 2004) . However, it should be borne in mind that these measurement scales frequently show significant variations in the validity and reliability indicators according to the population's characteristics (Keszei et al., 2010) . The current study aimed to evaluate the reliability and validity of the Spanish version of the Attitude towards COVID-19 Vaccines Scale. One-thousand one-hundred thirty-six students between 18 and 29 participated (M=22.0, SD=3.0). Five students who opened the questionnaire and did not sign the consent were unable to record their responses. The participants' characteristics (gender, marital status, career, income, residence, and COVID-19 infection history) are shown in Table 1 . The Spanish version of Attitude towards COVID-19 Vaccines Scale is an eight-item scale that offers five response options: 'totally disagree,' 'disagree,' 'unsure,' 'agree,' and 'totally agree.' These responses are scored directly from 0 to 4, except for item 7, which is scored in the reverse sense. Possible total scores are between 0 and 32. The higher the score, the more favourable attitude or greater acceptance. Table 2 presents the items of Attitude towards COVID-19 Vaccines Scale. A translation process was done from the original version in Danish to English and from English to Spanish. The back-translation process was carried out from Spanish to English only in a satisfactory way, given the simplicity of the sentences that easily allowed to achieve equivalence (Ramada-Rodilla et al., 2013) . In this process, the original versions of the scales from which the items were taken and modified were reviewed (Larson et al., 2015; Opel et al., 2015) . The authors preferred 'COVID-19' to 'Corona' because of Colombia's generalized name for the Spanish version. In the same way, it was preferred to use vaccines in the plural, since in Colombia, several vaccines are known and what is projected for herd immunization is the application of several of them. The PSS-10-C is a ten-item instrument quantifying perceived stress during the last month related to the COVID-19 epidemic. The tool offers five response options that are rated from zero to four. The total scores can be observed between 0 and 40; the higher the score, the greater the perceived stress. The PSS-10-C showed internal consistency of 0.85 in a previous online study of Colombian adults (Campo-Arias et al., 2020) . The Fear of COVID-5 is a five-item tool that quantifies attitude toward COVID-19. The scale offers four response options that are rated from zero to three. Total scores are between 0 and 15; higher scores indicate high fear. The Fear of COVID-5 showed internal consistency of 0.75 in Colombian adults who participated in online research (Cassiani-Miranda et al., 2021) . An invitation was sent from one of the researchers' institutional mail to the mail consigned in the university's admissions system. The email explained the study's objectives and reported the participants' anonymity and confidential handling information. Also, informed consent was required for a Google Form (c) designed questionnaire to be deployed. All responses were mandatory to avoid missing values. Two emails were sent with an interval of one week. In the second, those who had not done it were remembered and invited to participate. A second email a week can improve the response rate (Arroyo & Finkel, 2019) . The questionnaire link was active for potential participants between January 18th and February 5th, 2021. As suggested by the American Psychological Association, two internal consistency coefficients were calculated, Cronbach's alpha (Cronbach, 1951) and McDonald's omega, a better estimator when the principle of tau equivalence necessary for the calculation of Cronbach's alpha is violated (McDonald, 1970) . It is recommended that these coefficients show values between 0.70 and 0.95 (Keszei et al., 2010; Terwee et al., 2007) . These coefficients were calculated with Jamovi version 1.2.27. The EFA started with the calculation of the Bartlett test, for which a significant p-value was expected (Bartlett, 1950) and the Kaiser-Meyer-Olkin (KMO) coefficient, which should be above 0.60 (Kaiser, 1974) . Both acceptable tests indicate the existence of a latent factor in the set of items. Besides, communalities and loadings were observed. In the CFA were calculated Satorra-Bentler chi-square (and chi-square/df ratio), Root Mean Square Error of Approximation (RMSEA) and 90% confidence interval (90%CI), Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), and Standardized Mean Square Residual (SMSR). Under ideal conditions, observing the chi-square is expected to show a probability greater than 0.05 (Hu & Bentler, 1999) . Due to the high sensitivity of the indicator, the chi-square/df ratio is often preferred. Authors consider an acceptable value less than 5.0 (Wheaton et al., 1977) , and others prefer it below (Hair et al., 2006) . RMSEA and SMSR values lower than 0.06 are accepted, and CFI and TLI values greater than 0.89 (Hu & Bentler, 1999) . Generally, it is accepted that the data fit the proposed model if three or more indicators with recommended values are observed (Hu & Bentler, 1999) . EFA and CFA were carried out with Jamovi version 1.2.27 and STATA 13.0. The gender DIF of the items was quantified with the correlation for dichotomous variables tau-b (Kendall, 1938) . Non-DIF is accepted if the observed correlations are less than 0.20 and indicate that the response patterns are free of bias according to the variable taken as a reference (Hambleton, 2006) . This test was computed with STATA 13.0. Total scores were compared according to the students' careers. The health sciences students were taken as a reference against the rest of the careers. It is expected that health science students showed less negative attitude to COVID-19 vaccination and, consequently, showed lower scores than students in other majors. This comparison can be taken as another validity test called hypothesis or nomological (Borsboom et al., 2004) . The comparison was made with the Student's t-test (Student, 1908) . Finally, correlations were calculated between the total scores on the PSS-C-10, Fear of COVID-5, and Spanish version of Attitude towards COVID-19 Vaccines Scale as potential indicators of convergence or divergence nomological validity of the values of the correlation (Borsboom et al., 2004) . It was considered that if the correlations between the scales were observed below 0.30, it would indicate some divergence or independence, and the values between 0.30 and 0.60 would be an indicator of nomological validity (Hambleton, 2006) . The possibility of convergence, correlations higher than 0.60, was excluded because this is only possible when comparing scores of scales measuring the same construct (Borsboom et al., 2004) . For this, the Spearman correlation test was used (Spearman, 1904) . It was calculated using STATA 13.0. The project received an endorsement from an institutional research ethics board of a Colombian State university (Act 002 of the ordinary meeting of March 26th, 2020). All participants consented voluntarily (online informed consent). According to the Colombian norm (Ministerio de Salud de . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted July 22, 2021. ; https://doi.org/10. 1101 /2021 .07.18.21260733 doi: medRxiv preprint Colombia, 1993 , this project is considered a minimal risk for the participants and followed the guidelines enshrined in Helsinki's Declaration (World Medical Association, 2018) . The Spanish version of Attitude towards COVID-19 Vaccines Scale scores were between 0 and 32 (M=18.7, SD=7.2, Me=19, and IQR=14-24). The data presented a skewed distribution. Item 7 showed the lowest mean and correlation with the total score. Mean, standard deviation, and corrected correlation item-total score are summarized in Table 3 . Inter-item Pearson's correlations were above 0.63, except item 7 ('I am concerned about the possible side effects of the COVID-19 vaccines') correlated with the others between 0.25 and 0.43. Furthermore, items 2 ('I have faith in the COVID-19 vaccines') and 4 ('I would recommend the COVID-19 vaccines'), and 5 ('I think it is important that the people around me are vaccinated with the COVID-19 vaccines'), and 6 ('I think it is important that I get vaccinated with the COVID-19 vaccines') showed high correlations, above 0.80, suggesting collinearity. Table 5 Table 3 . The exploratory factor analysis showed that the items grouped a latent factor, 111.3, df=28, and KMO=0.93 . The one-dimensional structure was observed, an Eigenvalue of 5.64 that explained 70.5% of the variance. The communality values were between 0.164 and 0.794, and loading was between 0.405 and 0.891. Item 7 ('I am concerned about the possible side effects of the COVID-19 vaccines') presented the lowest values. See Table 5 . Confirmatory factor analysis was performed for the eight items and for the version without item 7, which showed low commonality, loading, and correlations with the other items. Items 2 ('I have faith in the COVID-19 vaccines') and 4 ('I would recommend the COVID-19 vaccines'), and 5 ('I think it is important that the people around me are vaccinated with the COVID-19 vaccines') and 6 ('I . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted July 22, 2021. ; https://doi.org/10.1101/2021.07.18.21260733 doi: medRxiv preprint think it is important that I get vaccinated with the COVID-19 vaccines') showed high correlations, above 0.80, suggesting collinearity. Versions of five were tested with the omission of one of the collinear items. The model for items 1, 3, 4, 6, and 8 improved in the RMSEA statistic. Evaluation of these items' content suggested that item 8 ('I rely on the information provided about the COVID-19 vaccines') probably explained the uncomfortable values for chi-square and RMSEA. The version with items 1, 3, 4, and 6 presented the best performance. The goodness-of-fit indicators for the four versions are detailed in Table 6 . Gender DIF According to gender, the Spanish version of Attitude towards COVID-19 Vaccines Scale showed similar scores among females (M=18.4, SD=7.2) and males (M=19.1, SD=7.2) (Levene's test for equality for variances F=0.03, a p-value of 0.88, and t=1.47, df=1,134, a p-value of 0.14). Besides, nongender DIF was observed (tau b Kendall between 0.02 and 0.06). Review Table 5 . Other statistics Non-health science students reported high distrust or negative attitude to COVID-19 vaccines, they got lower scores (M=18.0, SD=7.0) than health science students (M=21.7, SD=7.4) (Levene's test for equality for variances F=0.61, p-value of 0.44, and t=6.55, df=1,134, p-value<0.001). The Fear of COVID-5 scale presented scores between 0 and 35 (M=3.1, SD=3.0, Me=3, and IQR=2-6). The data presented a skewed distribution. This tool presented high internal consistency (Cronbach's alpha of 0.82). Furthermore, the PSS-C scores were between 0 and 40 (M=18.5, SD=6.4, Me=19, and IQR=14-22). The data presented a skewed distribution. This scale showed high internal consistency (Cronbach's alpha of 0.85). This study observed that the 8-item Spanish version of the Attitude towards COVID-19 Vaccines Scale presents a one-dimensional structure with high internal consistency in Colombian university emerging adults. However, the goodness-of-fit indicators are better for a four-item version. The 8-item Spanish version of Attitude towards COVID-19 Vaccines Scale scores was substantially different in health sciences students and students from other majors. Furthermore, it was found that the scale does not present gender DIF. The present study's findings are similar to those observed in the only previous investigation that applied the 8-item Spanish version of the Attitude towards COVID-19 Vaccines Scale for another purpose than reliability and validity assessment. Nevertheless, the author reported high internal consistency, acceptable inter-item correlations, and nomological validity with government trust and non-convergence with other measurements (De Roos, 2020). On the other hand, the author did not report the dimensionality of the scale; but, it can be inferred that internal consistency of 0.95 indirectly . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted July 22, 2021. ; https://doi.org/10.1101/2021.07.18.21260733 doi: medRxiv preprint indicates one-dimensionality because it is unlikely that an eight-item two-dimensional scale will show unusually high internal consistency. These internal consistency values are commonly observed for scales of more than fifteen items or when there is an evident redundancy of items or multiple collinearities (Keszei et al., 2010) . These similarities between the first and current study, especially internal consistency and interitem correlations, are encouraging indicators for a recently introduced instrument. It is common to observe very contradictory findings in psychometric assessments, even more so during the scale's application in a culturally different population (Keszei et al., 2010; Ramada-Rodilla et al., 2017; Terwee et al., 2007) . It is essential to consider possible explanations for the poor performance of item 7 ('I am concerned about the possible side effects of the COVID-19 vaccines'). The item has a reversed score and uses the adjective 'concerned.' It is often argued that these problems can be induced by items graded in a reverse way, negative sentences, disqualifying adjectives, or emotions and verbs imply psychological distress (Liu & Keusch, 2017; Suárez-Álvarez et al., 2018) . Negative attitude to COVID-19 vaccination showed an increasing trend during 2020 (Lin et al., 2021; Robinson et al., 2021) . Therefore, it is necessary to have a practical, valid and reliable instrument for routine measurement of the attitude towards this vaccine due to the negative public health implications. A sufficient number of vaccinated people is necessary to reach a herd immunity (Dubé et al., 2015; Zaidi & Flores-Romo, 2020) . Negative attitude to vaccines requires a better-articulated strategy to reduce inaccurate, contradictory, malicious, or conspiratorial information (García & Cerda, 2020). Health education must be at the service of different groups and weigh the cultural, social, political differences, etcetera, possibly mediating the attitude towards vaccines in general and the vaccine against COVID-19 in particular (Paul et al., 2020) . Colombia is a country with approximately 51 million inhabitants that, by July 19th, 2021, can certify more than 4,6 million cases of COVID-19, more than 116,000 fatal results, and a rate of approximately 2,250 deaths per million inhabitants (Ministerio de Salud y de la Protección Social de Colombia, 2021). Massive negative attitude to COVID-19 vaccination would have even more devastating effects on the most vulnerable populations promoting social inequalities and inequities that show that the COVID-19 pandemic is more of a syndemic (Horton, 2020) . Community health workers, general and specialized physicians, nurses and nursing assistants, occupational therapists, and social workers can help and not miss an opportunity to educate about vaccination benefits. It is also necessary to create synergies with other sectors such as those responsible for the mass media to provide health education based on truthful information (Paul et al., 2020) . The infodemic represents in itself a stressor and a danger for the success of some actions in public health (Ahmed, 2020; Patel et al., 2020) . Good indicators of validity and reliability of health measurements are needed (Keszei et al., 2010; Terwee et al., 2007) . Precise measurements guarantee the actual magnitude of the problems and verify the impact of the actions aimed at a solution (Terwee et al., 2007) . For instruments translated into Spanish from other languages, generally from English, it is necessary to preserve the equivalence of meaning that allows later cross-cultural comparisons (Ramada-Rodilla et al., 2017) . The CFA was not reported for the Danish version or other scales used to measure attitudes towards COVID-19 vaccination in studies published to date. The present study has the strength of showing several indicators of validity and reliability of the 8-item Spanish version of the Attitude towards COVID-19 Vaccines Scale among a large sample of emerging adults. Additionally, based on CFA, the current could verify the adjustment of versions of seven, five, and four items. The use of these shorter versions is at the investigators' discretion and can be used in particular contexts (Keszei et al., 2010; Terwee et al., 2007) . Nevertheless, no significant correlation was seen with measurements for COVID-19-related perceived stress and fear of COVID-19. These findings are not worrisome because the associations between variables depend more on the population's characteristics and, in the present case, on the measurement (Keszei et al., 2010) . Moreover, the convergence of the 8-item Spanish version of Attitude towards COVID-19 Vaccines Scale with another scale adapted for negative attitude to COVID-19 vaccination remains to be studied. In the same way, it should be considered that emerging university adults represented the sample. This particularity is not a problem because psychometric evaluation studies do not require a representative sample (Keszei et al., 2010; Terwee et al., 2007) . These considerations should not discourage using a scale since the instrument's validity depends more on the measurement's needs or objectives (Borsboom et al., 2004) . Additionally, they confirm the need for reliability and validity assessment in different cultures (Ramada-Rodilla et al., 2017) . The 8-item Spanish version of the Attitude towards COVID-19 Vaccines Scale is a onedimensional instrument, high internal consistency, acceptable nomological validity, reliability, and validity in emerging adults of a Colombian university. However, versions of seven, five, and four items may be helpful depending on the study's objectives. These findings should be taken as preliminary and invite corroboration of these validity and reliability indicators of the 8-item Spanish version of the Attitude Scale towards COVID-19 Vaccines in Colombia and other Spanish-speaking countries worldwide. Similarly, it should be studied the performance of versions in English and other languages. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted July 22, 2021. ; https://doi.org/10.1101 /2021 doi: medRxiv preprint vaccine rejection-An international overview. Vaccines, 9(1), 44. https://doi.org/10.3390/vaccines9010044 Fisher, K. A., Bloomstone, S. J., Walder, J., Crawford, S., Fouayzi, H., & Mazor, K. M. (2020) . Attitudes toward a potential SARS-CoV-2 vaccine: A survey of US adults. Annal of Internal Medicine, 173 (12) . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted July 22, 2021. ; https://doi.org/10.1101 /2021 .07.18.21260733 doi: medRxiv preprint Williams L., Flowers P., McLeod J., Young D., Rollins L., & The CATALYST Project Team (2021 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted July 22, 2021. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted July 22, 2021. ; https://doi.org/10.1101 https://doi.org/10. /2021 Managing News Overload (MNO): The COVID-19 infodemic Encuestas por Internet y nuevos procedimientos muestrales Test of significance in factor analysis The concept of validity Pandemic-Related Perceived Stress Scale of COVID-19: An exploration of online psychometric performance Adaptation and psychometric evaluation of the Fear of COVID-19 Scale in the general Colombian population Coefficient alpha and the internal structure of tests Who should tell you to vaccinate? The provider of information in the attitude towards the Corona vaccine: Need for conformity as a moderator? (master's thesis) Vaccine hesitancy, vaccine refusal and the antivaccine movement: influence, impact and implications Flattening the curve of COVID Relationship between citizens' health engagement and intention to take the covid-19 vaccine in Italy: A mediation analysis. Vaccines Multivariate data analysis Good practices for identifying differential item functioning Vaccine confidence in the time of COVID-19 Offline: COVID-19 is not a pandemic Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives An index of factorial simplicity A new measure of rank correlation Introduction to health measurement scales Measuring vaccine hesitancy: the development of a survey tool Confidence and receptivity for COVID-19 Vaccines: A rapid systematic review. Vaccines Effects of scale direction on response style of ordinal rating scales Understanding the dimensions of anti-vaccination attitudes: The Vaccination Attitudes Examination (VAX) Scale. Annual of Behavioral Medicine 1970.tb00432.x Ministerio de Salud de Colombia. Resolución 008430 por la cual se establecen las normas científicas, técnicas y administrativas para la investigación en salud Ministerio de Salud y la Protección Social de Colombia Psychological characteristics associated with COVID-19 vaccine hesitancy and resistance in Ireland and the United Kingdom Development of a survey to identify vaccine-hesitant parents: the parent attitudes about childhood vaccines survey Infodemic" COVID 19: More pandemic than the Virus Attitudes towards vaccines and intention to vaccinate against COVID-19: Implications for public health communications. The Lancet Regional Health-Europe, 100012 Influences on attitudes regarding potential COVID-19 vaccination in the United States Intention to receive a vaccine against SARS-CoV-2 in Italy and its association with trust., worry and beliefs about the origin of the virus Cross-cultural adaptation and health questionnaires validation: Revision and methodological recommendations International estimates of intended uptake and refusal of COVID-19 vaccines: A rapid systematic review and meta-analysis of large nationally representative samples COVID-19 vaccine hesitancy is associated with beliefs on the origin of the novel coronavirus in the UK and Turkey Validation of the vaccine conspiracy beliefs scale General intelligence objectively determined and measured The probable error of a mean Using reversed items in Likert scales: A questionable practice Quality criteria were proposed for measurement properties of health status questionnaires Assessing reliability and stability in panel models