Validating Culture-and Gender-Specific Constructs: A Mixed-Method Approach to Advance Assessment Procedures in Cross-Cultural Settings Validating Culture- and Gender-Specific Constructs: A Mixed-Method Approach to Advance Assessment Procedures in Cross-Cultural Settings John H. Hitchcock Caliber, an ICF International Company Sreeroopa Sarkar Bonnie K. Nastasi Gary Burkholder Walden University Kris Varjas Georgia State University Asoka Jayasena University of Peradeniya, Sri Lanka An earlier version of this work was presented in a symposium conducted at the 2005 annual meeting of the American Education Research Association. Funding for the early phases of this work was provided through grants to the third author from the Soci- ety for the Study of School Psychology and the University at Albany. The authors ac- knowledge Bill Disch for his support with statistical analysis. Address correspondence to: John Hitchcock, Caliber, an ICF International Com- pany, 10530 Rosehaven Street, Suite 400, Fairfax, VA 22030 (E-mail: JHitchcock@ icfcaliber.com). [Haworth co-indexing entry note]: “Validating Culture- and Gender-Specific Constructs: A Mixed-Method Approach to Advance Assessment Procedures in Cross-Cultural Settings.” Hitchcock, John H. et al. Co-pub- lished simultaneously in Journal of Applied School Psychology (The Haworth Press, Inc.) Vol. 22, No. 2, 2006, pp. 13-33; and: Multicultural Issues in School Psychology (ed: Bonnie K. Nastasi) The Haworth Press, Inc., 2006, pp. 13-33. Single or multiple copies of this article are available for a fee from The Haworth Doc- ument Delivery Service [1-800-HAWORTH, 9:00 a.m. - 5:00 p.m. (EST). E-mail address: docdelivery@ haworthpress.com]. Available online at http://japps.haworthpress.com  2006 by The Haworth Press, Inc. All rights reserved. doi:10.1300/J370v22n02_02 13 http://japps.haworthpress.com SUMMARY. Despite on-going calls for developing cultural compe- tency among mental health practitioners, few assessment instruments consider cultural variation in psychological constructs. To meet the challenge of developing measures for minority and international stu- dents, it is necessary to account for the influence culture may have on the latent constructs that form a given instrument. What complicates matters further is that individual factors (e.g., gender) within a culture necessi- tate additional refinement of factor structures on which such instruments are based. The current work endeavors to address these concerns by demonstrating a mixed-methods approach utilized to assess construct val- idation within a specific culture; and in turn develop culturally-specific in- struments. Qualitative methods were used to inform the development of a structured self-report by gaining detailed knowledge of the target culture and creating items grounded in interview and observational data. Factor analysis techniques and triangulation with qualitative analyses validated these findings. Previous work (Sarkar, 2003) suggested a number of gen- der-specific perceptions of mental health constructs within the target culture and these were investigated using additional mixed-method anal- yses. This article demonstrates an emerging mixed-method technique for developing culturally sound assessment tools, offers guidance on how to incorporate the overall approach in assessment, and provides a basis for thinking critically about the use of existing instruments when working with diverse populations. doi:10.1300/J370v22n02_02 [Article copies available for a fee from The Haworth Document Delivery Service: 1-800-HAWORTH. E-mail address: Web- site: © 2006 by The Haworth Press, Inc. All rights reserved.] KEYWORDS. Assessment, environment, gender, culture, validation Researchers, policy makers, professional organizations, and mental health practitioners have repeatedly called attention to the lack of cul- turally appropriate instruments for ethnic minorities, and the need to consider cultural factors in mental health programming (American Psy- chological Association [APA], 1990, 2003; Hall & Okazaki, 2002; Tanaka, Ebreo, Linn, & Morera, 1998; United States Department of Health and Human Services [USDHHS], 1999, 2001). An example eas- ily demonstrates why such calls are made. Egeland, Hostetter and Eshleman (1983) describe how culture can influence assessment 14 MULTICULTURAL ISSUES IN SCHOOL PSYCHOLOGY http://www.HaworthPress.com when describing manic behaviors of Old Order Amish suffering from bipolar disorder. Suppose a client presents some of the height- ened psychomotor activities associated with the disorder, but otherwise engages in such behaviors as using pay phones, dresses as a typical American might, is sexually active with more than one partner and uses machinery. Absent knowledge of Amish cultural norms, such behavior might easily be viewed as typical and a critical diagnosis could be missed. In this example, even cursory knowledge of the culture reveals just how suspect these behaviors might be, and such recognition would facilitate proper diagnosis. Unfortunately, however, dealing with cul- tural factors is almost never this straightforward because cultural varia- tion extends into a myriad number of groups and related effects can be subtle (see Castillo, 1997). Recognizing the complexities of culture represents a critical compo- nent of school psychology practice (Rogers et al., 1999); indeed, an em- pirical base supports the idea that cultural factors play an important role in influencing mental health (APA 2003; Nastasi, Moore, & Varjas, 2004). One explanation is that mental health constructs can be viewed differently by individuals from varied cultures, and perception is influ- enced by different agents of socialization (e.g., parents, school, and peers) that shape people’s beliefs, motives, attitudes, and behavior via socialization practices and expectations. Other variables such as gender can influence how mental health constructs are formed (Brannon, 2002; Sarkar, 2003). Clear and well-replicated gender differences in mental illnesses like depression, anxiety or schizophrenia have been reviewed by mental health researchers (Romans, 1998). In addition, researchers have shown that physiological, social, psychological, and environmental factors each predict gender differences in terms of mental health constructs (Anshell, Porter, & Quek, 1998; Bird & Harris, 1990; Block & Robins, 1993; Knox, Funk, Elliott, & Bush, 1998; Ptacek, Smith, & Dodge, 1994). Broader socio-cultural factors also play an important role in in- fluencing mental health of individuals through the process of socializa- tion. For instance, socialization practices within a particular culture influence the process of gender role socialization, and researchers have shown how this socialization process has made women more vulnerable to mental health problems (Das & Kemp, 1997; Das Dasgupta, 1996). Through its various agents of socialization, culture fosters development of gender role attitudes, beliefs/stereotypes, and gender-specific behav- iors which contribute to the gender differences in mental health con- structs (Sarkar, 2003). Hitchcock et al. 15 In view of the findings of gender-role socialization, investigating the gender differences in mental health constructs in the context of culture is important in providing appropriate services (Baxter, 1998; USDHHS, 1999; 2001). However, there is a dearth of research exploring the role of culture and particularly gender as a cultural variable influencing men- tal health of individuals; this is especially true in school psychology (Henning-Stout & Brown-Cheatem, 1999). In culturally diverse set- tings, it is therefore important to explore if there is variation in the defi- nition of mental health constructs. Because many U.S. public school systems represent multicultural contexts (Rogers et al., 1999), the meth- ods described below have clear applications for practice. What complicates matters is that most psychological instruments do not adequately address the influence of culture on functioning, especially for ethnic minority groups (APA, 2003; Padilla, 2001; USDHHS, 1999, 2001). Some literature suggests that members of a cultural majority tend to be unaware of cultural influences (Sue, Bingham, Porche-Burke & Vasquez, 1999); not surprisingly, researchers often generate instru- ments with little regard for these factors (Rogler, 1999). Failure to ad- dress cultural differences in assessment may therefore lead to problems with construct validity and subsequent efforts to develop interventions based on such assessment. The objective of this article is, therefore, to address the shortcom- ings in typical assessment in multicultural settings by demonstrating a mixed-methods approach (see Tashakkori & Teddlie, 2003) for devel- oping and validating culturally specific instruments. The approach combines the use of ethnographic and factor analytic methods to de- velop and test psychological instruments for students located in Sri Lanka. Some findings presented here will not have immediate relevance outside of Sri Lanka, although the methods can be readily applied to fu- ture studies concerned with mental health assessment. The overall approach entailed several stages: (a) a two-year ethnog- raphy conducted in the target culture for the purpose of understanding self-concept issues of adolescents, (b) developing and administering a series of culture-specific instruments that further assess these issues, and (c) demonstrating a mixed-method approach for identifying and vali- dating psychological constructs that are specific to the target culture. An overview of this process can be found in Hitchcock and colleagues (2005). The primary purpose of the earlier publication was to illustrate the methodology with a specific focus on the substantive area of self-con- cept (as defined by Harter, 1999) and concerns for adolescents in Sri Lanka. This paper extends the previous work through reanalysis of quali- tative data (that have not been previously published) and quantitative 16 MULTICULTURAL ISSUES IN SCHOOL PSYCHOLOGY analysis of a different dataset. The earlier work focused on students’ per- ceptions of how their parents valued various competencies and behav- iors; this article focuses on student perceptions of their competencies, and identifying gender-specific differences of these perceptions. FOUNDATIONS The work described herein employed the Ecological-Developmental Model of Mental Health (EDMMH; Nastasi et al., 2005) as the concep- tual framework for investigating individual and cultural factors related to mental health. The EDMMH has its foundations in psychology and an- thropology (Nastasi & DeZolt, 1994; Nastasi, Varjas, Sarkar, Jayasena, 1998) and has been used to guide the development of culture-specific definitions of key constructs through the integration of cultural factors related to the experiences of participants. An ecological approach to hu- man development involves scientific study of a ‘progressive, mutual ac- commodation’ between an individual and the environment, in view of the social, cultural and historical contexts (Bronfenbrenner, 1989; 1999). Reflected in the EDMMH is the conceptualization of individual mental health as an interaction among the personal factors–self-perceptions of culturally valued competencies, personal resources, and personal vul- nerability, with the cultural factors–cultural norms, competencies val- ued within the culture, social-cultural stressors and resources, and socialization agents and practices. The model promotes the idea of de- veloping a sense of normative cultural values and any conflicts in values among groups within a context. This allows for an assessment of devi- ant behavior given cultural norms and stressors derived from lack of re- sources and competing group needs, both of which are representative of mental health concerns. EDMHH can help assess what might be cultur- ally deviant, but more importantly, it is a framework that is consistent with well-established conceptions of risk and protective factors related to mental health/illness of individuals and prevalence within communi- ties (e.g., Elias & Branden, 1988), models of stress and coping (e.g., Lazarus & Folkman, 1984) and personal-social competence (Nastasi & DeZolt, 1994). It also is consistent with recent developments in the field of positive psychology (Frydenberg, 2002; Snyder & Lopez, 2002). QUALITATIVE METHODOLOGY AND CONTEXT The research involved the combined use of qualitative (e.g., inter- views, focus groups) and quantitative (rating scales) data collection Hitchcock et al. 17 methods to assess the contextual environment of adolescents in Sri Lanka. The study was conducted within the municipality of Kandy, the second largest city in the country with a population of approximately 100,000. Its schools draw students from the urban community and sur- rounding villages and the socioeconomic status of community members ranges from lower to upper class (Nastasi et al., 2004).The qualitative study was formative in nature and involved the assessment of the mental health needs of adolescents (see Nastasi et al., 2004). This study was conducted with the long-term goal of developing school-based mental health programs in that country. The findings from that work were used to develop and evaluate a culture-specific mental health assess- ment tool and intervention program for promoting mental health among school-age children in Sri Lanka. Sri Lanka has experienced advances in education, health and general standard of living (United Nations Development Programme, 2002; The World Bank in Sri Lanka, 2003). However, the last twenty years of Sri Lankan history has been marked by internal tension resulting from high unemployment rates, civil war, ethnic tension and the youth insur- rection (1985-1991; Gunaratna, 1990). These factors have contributed to an unstable economy with dwindling per capita income as well as po- litical unrest and mental health problems among the youth. In an effort to gather data on the mental health of students, researchers conducted group interviews with adolescents (33 groups) and teachers (18 groups), and individual interviews with principals from 18 schools to explore the following constructs: (a) cultural definition of mental health/personal- social competence and adjustment difficulties, (b) cultural mechanisms for socialization and development of mental health/personal-social com- petence, (c) social stressors as viewed by the adolescents, (d) mecha- nisms/personal resources for coping with those and other everyday life stressors, and (e) existing resources within the schools and community for securing mental health services. The interviews focused on gather- ing a wide range of data with regard to the definition of key constructs. Interviews were therefore conducted in an open-ended, semi-structured format (Nastasi, Varjas, Sarkar & Jayasena, 1998.) Interview questions were generated in collaboration with an educa- tional sociologist/teacher educator and a child psychiatrist in Sri Lanka who had experience in schools (Nastasi et al., 1998). Additionally, questions were asked regarding more specific mental health related is- sues (e.g., stressors the youth face with regard to academics, types of ac- ademic adjustment difficulties that are prevalent among adolescents, family problems and related adjustment difficulties in the children, and 18 MULTICULTURAL ISSUES IN SCHOOL PSYCHOLOGY the ways in which children and adolescents cope with family problems). Data were coded to identify and define the culture-specific individual and cultural constructs. Efforts were made to understand the educa- tional and psychological concerns of students through their own per- spective, as well as those of school professionals. Specific issues noted were high rates of mental health concerns such as substance abuse among peers and parents, stress associated with dating and teacher/par- ent interest in limiting interaction between the genders, and suicide. A variety of educational concerns associated with a high stakes examina- tion system and limited school resources also were noted. Finally, the data reflected gender differences in several mental health constructs. For example, girls discussed sexual harassment, lack of freedom com- pared to their male peers and differential expectations for women; whereas boys expressed concern about violence, corruption, and unem- ployment. [See Nastasi et al., 1998 for full report of methods and initial results.] In view of these findings, a re-analysis of qualitative data was performed to explore the gender differences in the definition of mental health constructs. These analyses formed the basis for a key approach that can combine quantitative and qualitative findings–ethnographic surveys. QUANTITATIVE METHODOLOGY Ethnographically informed instruments are designed to reflect the experiences and constructs relevant to the target population (Schensul & LeCompte, 1999), thus yielding culture-specific measures. From the qualitative data, we developed self-report measures designed to as- sess adolescents’ perceived competencies and perceived value of the competencies from their own perspective and that of parents, teachers, and peers. The instruments used the Likert-scale (3- point; 1 = not at all, 2 = some, 3 = a lot) response format. On the perceived competencies measure (the focus of this paper), adolescents were asked to rate them- selves on a set of culturally defined competencies. Scale and item con- struction were designed to reflect the full variation of data across age, gender, ethnicity, and ecological context. After piloting, back translat- ing (e.g., English Æ Sinhalese Æ English, to insure accuracy of mean- ing) and refining via consultation with local experts with knowledge of the target culture, the instrument was administered in written form to students (n = 611; 315 males, 296 females), grades 7-12, ages 12-19, Hitchcock et al. 19 across six schools which represented the range of the student population in terms of ethnicity, religion, and socioeconomic status. For further discussion of instrument development, see Hitchcock and colleagues (2005). The strategy used for the quantitative analysis included a principal components analysis (PCA) of the items in the survey instrument. Promax (oblique) rotation was used to account for an expected correla- tion among components. Items loading less than .30 on any scale, as well as complex items (those whose factor loadings across scales dif- fered by less than .20) were removed. Cronbach’s alpha, a measure of internal consistency, was computed for all scales for the total sample and by gender. Multivariate analysis of variance (MANOVA) was used to compare first the scale scores resulting from the factor analysis by gender, then the individual items on any scale that demonstrated signifi- cant differences between boys and girls. FINDINGS FROM QUALITATIVE ANALYSES Formative qualitative data were collected by asking students and teachers to describe characteristics of socially-acceptable and unaccept- able behavior patterns for boys and girls. Analysis of these data provided evidence of (1) socially-acceptable “suitable” behavior, (2) socially-un- acceptable “unsuitable” behavior and (3) behavior that reflected atten- tion to the personal and interpersonal needs of adolescents (see Table 1). Overall, gender differences in how unsuitable behavior is defined were noted. That is, both males and females recognized a broader unsuitable behavior construct but differed when defining it. In terms of suitable be- havior, both genders defined it the same way but differences were noted in the degree of expected compliance, with girls generally reporting that they were expected to engage in socially-acceptable behaviors more so than boys. In the area of personal/interpersonal needs, minor gender dif- ferences were reflected in the restricted movement and lack of inde- pendence and freedom of girls. Following is an overview of each construct. Suitable Behavior In group interviews, both male and female adolescents described suit- able behavior to include “good” behavior and obedience (see Table 1). For example, suitable behavior entails following school rules and lis- 20 MULTICULTURAL ISSUES IN SCHOOL PSYCHOLOGY tening to adults. Both male and female students in Sri Lanka argued that respect for elders is a major attribute of culturally valued suitable behaviors. Suitable behavior also was characterized as conducting one- self well, such as by being well disciplined, humble, loyal, and trustwor- thy. Respondents from both genders revealed that culturally acceptable suitable behavior also included being performance-oriented (e.g., per- forms well and assumes leadership in extracurricular and competitive activities). As previously noted, there were differences between boys and girls in terms of suitable behavior that focused on degree of compliance ex- pectations. That is, females were expected to exhibit more of these be- haviors than boys. Female students described a girl showing suitable behavior as one who listens to what parents and teachers say, and ob- serves the rules and regulations. Examples of such behaviors might be trying to avoid troubling others, giving up a seat to a teacher when they are on the bus, and observing customs and tradition of the country. Girls indicated that such an individual talks nicely with others, sets a good ex- ample for others, performs community services, and loves the country. Male students also indicated that such a person obeys the law and does not harm others, but focused more on avoiding disruptive behavior, as opposed to following a broad set of expectations. Hitchcock et al. 21 TABLE 1. Definition of Suitable and Unsuitable Behavior with Gender Differ- ences Highlighted Cross-Gender Definitions of Suitable Behavior Well behaved, Obedient, Disciplined, Humble, Respect elders, Helpful, Loyal, Trustworthy, Supportive, Guides others (e.g., to correct path), Makes effort for best performance in academics Male-Specific Reponses Female-Specific Responses Obey laws of the country, Leader, Possesses the ability to address and solve problems, Avoids trouble Follows rules, Talks and behaves nicely with people, Loves the country–protects it, Performs community services, Sets example for others, Treats and loves everyone equally–irrespective of ethnic origin Cross-Gender Definitions of Unsuitable Behavior Ill behaved (e.g., rude, tease girls), Abusive (e.g., use abusive/foul languages), Disruptive (e.g., interrupt others' work), Noncompliance, School truancy, Neglects responsibilities and/or obligations, Not helpful (e.g., do not help friends in trouble) Male-Specific Reponses Female-Specific Responses Throws tantrum, Pouts, Runs away from home, Betrays friends, Aggressive (e.g., fights, argues; Abusive), Abuses alcohol, drugs, Steals or robs, Joins the gang, carries weapons Does not follow etiquette, Lies, Stubborn, Interfering, Disruptive, Mistreats others, Slanders, Does not love the country, Having romantic relationship with boys, Acts like a boy Unsuitable Behavior Both males and females referred to unsuitable behavior as being “ill behaved” (e.g., gets drunk and misbehaves with girls), aggressive (e.g., fights with other people), abusive (e.g., uses foul languages), and/or dis- ruptive (e.g., disrupts the classroom). Noncompliance (e.g., refuses to comply with rules; does not obey elders) and school truancy were other attributes of unsuitable behavior among adolescents. Overall, descrip- tions of unsuitable behavior appeared to be readily identifiable in inter- view responses. Despite this finding, the qualitative data indicated important gender differences in how the construct is defined. Unsuit- able behavior among Sri Lankan girls was described in terms of dishon- esty (e.g., lying to parents), stubbornness, not following etiquette, and engaging in romantic relationships or love affairs. Furthermore, girls did not approve of cross-gender behavior. They indicated that “girls who act or speak like boys or dress like boys” have behavioral prob- lems. Males described the construct in different terms. They focused on behaviors such as delinquency (e.g., substance abuse, stealing, running away from home, joining gangs) and aggression (e.g., fighting, quarrel- ing with adults or peers, assaulting others). Personal/Interpersonal Needs Students provided information about personal and interpersonal needs during their interviews. Although few gender differences were found in terms of this construct, it is does provide an interesting perspective of some of the pressures adolescents report. Respondents indicated that poverty limits their access to basic necessities of life such as proper food, clothing, housing and education. They also indicated that family support, both material and emotional, was very important for them to thrive. Meanwhile, several respondents, particularly girls, indicated that many mothers leave to work in the Middle East (e.g., Saudi Arabia, Jordan, Kuwait, Qatar, United Arab Emirates) for financial reasons. They find jobs as household workers or maids and send money to their families in Sri Lanka, leaving children to assume several domestic re- sponsibilities without maternal support. Respondents were vocal about their academic needs in terms of in- structional support, guidance, and emotional support that they received from parents, teachers, peers, relatives, and private tutors. They particu- larly emphasized the importance of effective teaching in the classroom and the support from teachers to meet their academic needs (recall Sri 22 MULTICULTURAL ISSUES IN SCHOOL PSYCHOLOGY Lanka uses a high stakes examination system). They also described the lack of mental health services in school. They indicated that such ser- vices were only available through psychiatric centers at hospitals, which are often not readily accessible. Furthermore, mental illness is stigmatized in the culture and this is thought to reduce the likelihood that services will be accessed. Respondents spoke about their need for extracurricular activities. In particular, they expressed the need for recreation and complained about lack of recreational opportunities due to academic pressure from par- ents. Sample quotes include: [Our] worst stress is tuition [private tutoring; additional instruc- tion outside of the school context] . . . here all girls get tuition. We finish school at 2 p.m. and go home and grab a snack and go to tui- tion [meaning class] . . . we need other things like sports. But we don’t have the time. [Female respondent] [There is] so much competition. We don’t have lots of time to do extracurricular activities. There is not time to do things kids are supposed to do. No hobbies. We do not have time to do such things . . . we have little time, we get tired from going to classes. [Male respondent] Students expressed a desire to go out with their friends, party, go on va- cations, and watch sports and television. Interview data from adoles- cents and adults indicated that indulging in recreational activities is typically not permitted by parents because of the strong emphasis on ac- ademic preparation. Several students mentioned interest in interacting with opposite-sex friends; meanwhile, Sri Lankan society does not encourage free interac- tion of males and females. For example, boys and girls mentioned that they were not allowed by their parents to interact with the opposite gen- der or have any relationship with them. Many students thought that this practice needs to be changed. They also indicated the importance of ro- mantic relationships during adolescence and how their parents, relatives and the society did not approve of such relationships. Female respon- dents also indicated the lack of freedom and independence for girls in Sri Lankan society. They suggested that girls enjoyed much less free- dom and independence in contrast with the boys. Their activities were restricted and supervised by parents and other elders. The girls ex- pressed the need for more freedom and removal of restrictions that Hitchcock et al. 23 would allow them to function independently and would promote self-efficacy and self-confidence in girls. Although adults emphasized the importance of suitable and unsuitable behaviors as critical to definitions of culturally-valued competencies, ad- olescents also emphasized behaviors that related to their personal and in- terpersonal needs. For this reason, when developing the perceived competence instrument, we included items related to effective engage- ment in recreational activities and interpersonal relationships (i.e., re- flecting a more “well-rounded” adolescent or, as described in the culture, the “all-rounder”). FINDINGS FROM QUANTITATIVE ANALYSES Principal Components Analysis (PCA). Recall that qualitative data were used to develop surveys. Analysis of survey responses yielded an initial scale structure that was congruent with the qualitative findings. Three factors were identified. Unsuitable Behavior (socially unaccept- able behavior) [a = .94; a = .95 (Female); a = .93 (Male)] consisted of fifteen items (see Table 2). These items described behaviors viewed to be inappropriate within this culture (e.g., drug use, stealing, and fight- ing). Suitable Behavior (socially acceptable behavior) [a = .76; a = .72 (Female); a = .78 (Male)] consisted of thirteen items. This scale de- scribed desirable adolescent behaviors within this culture, including studying, following school rules, and remaining clean in appearance. The final factor, Personal/Interpersonal Needs (i.e., behaviors/compe- tencies related to fulfilling personal and social needs of Sri Lankan ado- lescents) [a = .72; a = .67 (Female); a = .76 (Male)], consisted of nine items that described behaviors/competencies for meeting these needs. These needs included spending the day with friends and going to par- ties. These factor structures were expected both because of the qualita- tive analyses but also because they have been previously established with a similar survey (student perceptions of the degree to which par- ents value the behaviors) using the same sample (Hitchcock et al., 2005). Note, however, that the previous work did not examine the data for gender differences. Table 2 contains factor loadings for males and females as well as for the total sample. MANOVA analyses were performed to test for gender differences. In the first MANOVA, gender was used as the independent variable, and the three scale scores were used as dependent variables. The overall MANOVA was significant, Wilks Lamdba = .96, F(3, 606) = 8.77, p < 24 MULTICULTURAL ISSUES IN SCHOOL PSYCHOLOGY TABLE 2. Factor Loadings Resulting from Principal Components Analysis (PCA) and Differences of Scale Scores and Individual Items by Gender Scale/Item Factor Loadings Overall Male Female Overall Female Male Mean (SE) Mean(SE) Mean(SE) Suitable Behaviors Cronbach Alpha(a) .94 .95 .93 2.75 (.23) 2.70 (.25) 2.79 (19)*** I am honest. .62 .59 .61 2.81 (.41) 2.75 (.45) 2.87 (.35)*** I obey school and classroom rules. .60 .54 .60 2.87 (.35) 2.80 (.42) 2.92 (.26)*** When somebody shows me a mistake, I like to accept it and correct myself. .58 .45 .64 2.84 (.38) 2.77 (.44) 2.90 (.31)*** I like to look after others who are sick or hurt. .57 .54 .53 2.73 (.47) 2.64 (.53) 2.83 (.39)*** I study regularly. .55 .53 .55 2.58 (.51) 2.53 (.53) 2.64 (.49)** I pay attention to the studies in the school. .55 .49 .60 2.94 (.24) 2.92 (.27) 2.95 (.21) I persevere even when faced with a difficult task. .48 .48 .63 2.66 (.52) 2.60 (.56) 2.72 (.47)** I help the poor through good works. .47 .45 .44 2.62 (.49) 2.53 (.51) 2.70 (.46)*** I interact well with my teachers. .47 .48 .46 2.85 (.37) 2.83 (.40) 2.88 (.35) I move with respectable peers. .45 .31 .55 2.72 (.53) 2.68 (.56) 2.75 (.51) I follow rules and expectations according to the situation. .44 .41 .44 2.83 (.39) 2.80 (.42) 2.85 (.37) I observe customs and traditions of the country. .43 .52 .43 2.71 (.47) 2.67 (.49) 2.75 (.43)* I am clever (intelligent). .42 .47 .37 2.63 (.50) 2.67 (.50) 2.60 (.50) Unsuitable Behaviors Cronbach Alpha (a) .76 .72 .78 1.06 (.22) 1.08 (.23) 1.05 (.21) I use drugs. .91 .92 .91 1.03 (.21) 1.03 (.23) 1.02 (.20) I drink alcohol. .91 .94 .89 1.03 (.23) 1.04 (.24) 1.03 (.23) I steal. .90 .94 .86 1.03 (.23) 1.04 (.24) 1.03 (.23) I carry weapons. .85 .93 .75 1.04 (.26) 1.06 (.27) 1.04 (.25) I smoke cigarettes. .85 .89 .81 1.03 (.23) 1.03 (.25) 1.03 (.22) I persuade others to join a gang. .82 .84 .81 1.03 (.24) 1.03 (.25) 1.03 (.24)2 5 TABLE 2 (continued) Scale/Item Factor Loadings Overall Male Female Overall Female Male Mean (SE) Mean(SE) Mean(SE) Unsuitable Behaviors Cronbach Alpha(a) .76 .72 .78 1.06 (.22) 1.08 (.23) 1.05 (.21) I persuade others to engage in bad habits. .79 .75 .83 1.04 (.25) 1.04 (.25) 1.03 (.24) I insult others. .73 .86 .59 1.09 (.32) 1.12 (.35) 1.05 (.29) I use profane language .71 .77 .68 1.09 (.33) 1.13 (.28) 1.05 (.27) I scold or criticize teachers. .72 .80 .65 1.09 (.33) 1.13 (.38) 1.06 (.27) I interrupt others’ work. .72 .74 .70 1.09 (.23) 1.10 (.23) 1.07 (.23) I associate with bad peers. .69 .86 .57 1.08 (.31) 1.14 (.37) 1.04 (.23) I am a member of a gang. .70 .71 .71 1.07 (.32) 1.06 (.28) 1.07 (.35) I waste time. .60 .57 .64 1.10 (.37) 1.12 (.40) 1.08 (.34) I don’t carry out my responsibilities. .45 .53 .36 1.17 (.46) 1.17 (.46) 1.17 (.29) Personal/Interpersonal Needs Cronbach Alpha (a) .72 .67 .76 2.50 (.34) 2.48 (.36) 2.52 (.31) I like to spend the day with friends. .69 .65 .71 2.42 (.63) 2.38 (.65) 2.46 (.62) I like to have fun with others. .62 .61 .64 2.56 (.59) 2.56 (.59) 2.56 (.60) I listen to others’ problems. .61 .48 .70 2.21 (.76) 2.19 (.74) 2.22 (.77) I like to listen to music or sing. .53 .50 .55 2.70 (.49) 2.65 (.51) 2.75 (.46) I like to go on trips. .53 .54 .53 2.64 (.53) 2.67 (.53) 2.61 (.53) I help others to solve their problems. .49 .41 .53 2.55 (.57) 2.52 (.60) 2.58 (.55) I am sensitive to others’ feelings and needs. .48 .40 .53 2.41 (.59) 2.39 (.59) 2.43 (.60) I like to go to parties. .49 .49 .49 2.40 (.58) 2.42 (.61) 2.39 (.55) I safeguard others’ secrets. .40 .41 .40 2.61 (.64) 2.54 (.66) 2.67 (.63) Note: * p < .05; **p < .01; ***p < .001 2 6 .001. The follow-up analysis indicated that one of the scales, Suitable Behaviors, was different by gender [F(3,606) = 25.3, p < .001], with girls scoring higher than boys. The other two factors were not statisti- cally significant at the .05 level (Unsuitable Behaviors–F(3,606) = 3.07, p = .08; Personal/Interpersonal Needs–F(3,606) = 2.45, p = .12). Subsequent analyses were performed to examine the individual items in the Suitable Behaviors factor by gender. The overall MANOVA, as ex- pected, was statistically significant (Wilks Lamdba = .90, F(13, 582) = 4.80, p < .001). Table 2 contains more detailed information of the analy- sis, including the items in each factor, the factor loading on the individ- ual factor, the means and standard errors for each factor and item by gender, and significant differences (determined by the MANOVA) by gender. To summarize, results of the factor analyses provided construct validation of perceived competencies consistent with cultural concepts reflected in the qualitative data. A key finding is that the suitable behav- ior construct differs by gender; in general, girls reported more suitable behaviors. Examination of the factor loadings for boys and for girls indicated that there might be differences in factor structure. Such differences can illuminate ways in which boys and girls interpret items within a con- struct differently. Tests for factor (construct) invariance using structural equation modeling can be used to help locate such statistical differences that can then be compared to qualitative findings (see Byrne, 1994). Hy- potheses are developed that test the equivalence of factor loadings across groups (levels of gender, in this case). While the details of the analysis are beyond the scope of this paper, we conducted preliminary analyses using this technique.1 Table 3 contains a summary of the anal- yses. The strategy is to initially test the equivalence of the factor struc- ture across groups; then, successively restrictive equality constraints are imposed on the models. For example, the constraints of interest here are the factors loadings. If item factor loadings are constrained to be equal across models, and the change in the degrees of freedom and chi-square are examined, a statistically significant chi-square statistic suggests dif- ferences in structure. The results of Table 3 show that, while Suitable Behaviors and Personal & Interpersonal Needs factor items load equiv- alently for boys and girls, there are differences in the factor structure of Unsuitable Behaviors. This finding is supported by the qualitative data which indicated that several behaviors such as joining gangs, robbing, carrying weapons, as- saulting people, alcohol and drug abuse were described only by male re- spondents and viewed as only relevant to males. This occurrence may Hitchcock et al. 27 be explained by the cultural emphasis on gender-appropriate behavior which is comparatively more rigid and less permissive for women. Some caution is needed to avoid over-interpretation. The assessment tool used for quantitative data collection was a self-report instrument and the female respondents may have included only the socially desir- able responses to conform to the cultural expectations. These dif- ferences in definition and interpretation of unsuitable behavior do, how- ever, require further examination. DISCUSSION When dealing with populations with distinct cultural variables, re- searchers typically employ preexisting clinical or research instruments developed for mainstream American children and adolescents, in some instances with minimal modifications (e.g., language translation) and without re-validation. This is problematic because of the likelihood that such instruments will miss nuanced issues important to a target popula- tion. The work described here provides an alternative and represents theoretical and methodological approaches for investigating the role of individual and cultural factors in mental health of a distinct culture, par- 28 MULTICULTURAL ISSUES IN SCHOOL PSYCHOLOGY TABLE 3. Constrained Models CFA Tests Summary of Analyses Model c2 Df Dc2 Ddf CFI RMSEA Full, 3 Factor Model, no constraints 4246.637 1252 .964 .06 Full, 3 Factor Model, Factor loadings, variance, covariances constrained to be equal 4493.456 1326 246.82 74* .962 .06 Full, 3 Factor Model, Factor loadings only for all three factors constrained 4334.142 1286 87.51 34* .963 .06 Unsuitable Behaviors only constrained 4306.790 1266 60.15 14* .963 .06 Suitable Behaviors only constrained 4265.249 1264 18.62 12 .964 .06 P & IP Needs only constrained 4255.381 1260 8.744 8 .964 .06 * p < .05 Note. Models were compared to the full models with no constraints. The results suggest (and are consistent with exploratory analysis) that structures for Unsuitable Behaviors are different for boys and girls. ticularly the combined use of ethnographic and factor analytic tech- niques. As a result of this effort, a culturally specific instrument that measures self-concept issues in Sri Lanka was developed and yielded insights into the culture. There is compelling data that show girls report that they endeavor to follow societal expectations for culturally-defined suitable behavior when compared to boys. This could be attributed to the cultural emphasis on a traditional gender role behavior that encour- ages submissiveness, tolerance and a caring attitude in women (Das Dasgupta, 1996). The findings dealing with girls’ perceptions of suitable and unsuit- able behavior may prove useful when engaged in future work in the cul- ture. Should a girl appear to not be following expectations, we can now recognize that follow-up may be important. The girl may simply be un- usually independent, or she might be reacting to some difficult life cir- cumstances. Whatever the case, these analyses provide a basis for recognizing the behavior as unusual and follow-up may be warranted. This work also suggests a need to revisit the unsuitable behavior con- struct in light of gender-specificity. As noted above, the items that form the factor structure for the construct load differently by gender. This is a subtle difference from the MANOVA results indicating differences in suitable behavior. In this latter construct, the data do not suggest the genders define the construct differently; girls simply scored higher than boys. For Unsuitable Behavior, however, Table 2 shows the factor load- ings for some items were higher for girls than boys (e.g., I carry weap- ons, I insult others, I scold or criticize others, I associate with bad peers, and I don’t carry out my responsibilities). This suggests greater variability in how males responded to these items and the issue can be further explored in future studies. The Personal/Interpersonal Needs construct appeared in the factor analysis, but this was not gender-spe- cific. These findings were congruent with qualitative data; there was no reason to suspect there would be gender differences on this factor. The identification of this third factor can also inform future intervention work. The data suggest that Sri Lankan students recognize a need for recreational time with friends, yet this is generally not allowed by par- ents and educators. Communicating this finding to stakeholders may alone be beneficial as allowing for more recreation may reduce stress among adolescents (a long-term goal of this research program). More generally, the psychological constructs presented here have been vali- dated by qualitative and factor analytic data analyses, so researchers can proceed with confidence using this instrument for wider research proj- ects within the country. Hitchcock et al. 29 One limitation of the present study is that our data did not completely capture indicators of socioeconomic status (SES); qualitative work did not highlight this as an important factor. Ancillary analyses (not shown) including mother and father education as SES indicators as covariates did not change any of the conclusions made here, and gender did not in- teract with SES variables. However, the results of those analyses sug- gested that SES indicators may offer important ways to understand elements of students’ self-concept in Sri Lanka; this issue should be ex- plored in future research. We opened with a discussion as to why having knowledge of a target culture facilitates assessment. A specific example was provided using Old Order Amish and the data here further demonstrate the point. Now suppose that, in Sri Lanka, an interventionist meets a girl who reports she is rejecting behavior expectations imposed upon her by parents by actively seeking out recreational activity in lieu of studying and engag- ing in cross-gender behavior (e.g., acting or dressing like a boy). In the United States this behavior would not be considered unusual and possi- bly even be encouraged; in Sri Lanka, such behavior would be non-nor- mative and a caregiver would do well to pay closer attention to the girl’s circumstances. This raises the issue of whether it is better to change the local culture to facilitate such independence or to get the girls to con- form. This represents a complex debate, but the importance of interven- ing on the girl’s behalf at some level would be clear and the caregiver should not dismiss the behavior. The point is that, as with the case of the Old Order Amish, knowledge of cultural variation would solicit very different behaviors among practitioners. In other words, detailed knowledge of a culture facilitates assessment and the development of sensitive interventions, which is very much in-line with calls for devel- oping cultural competency. As a side note, we have confidence in these findings because there is cross-method consistency (i.e., triangulation) that supports their validity. In-depth qualitative findings, obtained from smaller groups, were consistent with the results of the various quantita- tive analyses described above, using a large sample. Finally, these findings should not be of interest only to those with in- terests in mixed-methods and cross-cultural work; there are implica- tions here for school psychologists. Psychologists are being pushed to develop cultural competencies and the methods described above sup- port this endeavor. When working in a multicultural setting, the qualita- tive procedures can serve as models for service planning and identifying relevant cultural issues, and the development of a survey from this in- formation can be used to quantify such information in the event a large 30 MULTICULTURAL ISSUES IN SCHOOL PSYCHOLOGY enough sample warrants the additional effort. These skills can help school psychologists understand the idiosyncratic needs of a local cul- ture, develop nuanced assessment skills and in turn develop highly tar- geted interventions. NOTE 1. More detailed findings can be obtained from the first author. REFERENCES American Psychological Association (1990). Guidelines for providers of psychologi- cal services to ethnic, linguistic, and culturally diverse populations. Washington, DC: Author. American Psychological Association (2003). Guidelines on multicultural education, training, research, practice, and organizational change for psychologists. American Psychologist, 58, 377-402. Anshel, M. H., Porter, A., & Quek, J. (1998). Coping with acute stress in sport as a function of gender: An exploratory study. Journal of Sport Behavior, 21 (4), 363-376. Baxter, J. (1998). Culture and Women’s Mental Health: International Perspectives and Issues for Aotearoa/New Zealand. In Sarah Romans (ed.) Folding Back the Shad- ows: A Perspective on Women’s Mental Health. Dunedin, NZ: University of Otago Press. Bird, G. W., & Harris, R. L. (1990). A comparison of role strain and coping strategies by gender and family structure among early adolescents. Journal of Early Adoles- cents, 10, 141-158. Block, J., & Robins, R. W. (1993). A longitudinal study of consistency and change in self-esteem from early adolescence to early adulthood. Child Development, 64, 909-923. Brannon, L. (2002). Gender: Psychological Perspectives (3rd ed.). Boston: Allyn and Bacon. Bronfenbrenner, U. (1989). Ecological systems theory. In R. Vasta (Ed.), Annals of Child Development (Vol. 6, pp. 187-249). Greenwich, CT: JAI Press. Bronfenbrenner, U. (1999). Environments in developmental perspective: Theoretical and operational models. In S. L. Friedman & T. D. Wachs (Eds.), Measuring envi- ronment across the life span: Emerging methods and concepts (pp. 3-30).Washing- ton, DC: American Psychological Association. Byrne, B.M. (1994). Structural equation modeling with EQS and EQS/Windows: Basic concepts, applications and programming. Thousand Oaks, CA: Sage. Castillo, R.J. (1997). Culture and Mental Illness: A Client-Centered Approach. Pacific Grove, CA: Brooks-Cole. Das, A. K., & Kemp, S. F. (1997). Between two worlds: Counseling South Asian Americans. Journal of Multicultural Counseling and Development, 25 (1), 23-33. Das Dasgupta, S. (1996). Feminist consciousness in women-centered Hindi films. Journal of Popular Culture, 30 (1), 173-189. Hitchcock et al. 31 Egeland, J.A., Hostetter, A.M., & Eshleman, S.K. (1983). Amish study III: The impact of cultural factors on diagnosis of bipolar illness. American Journal of Psychiatry, 140, 67-71. Elias, M. J. & Branden, L. R. (1988). Primary prevention of behavioral and emotional problems in school-aged populations. School Psychology Review, 17, 581-592. Frydenberg, E. (2002). (Ed.). Beyond coping: Meeting goals, visions, and challenges. Oxford, UK: Oxford University Press. Gunaratna, R. (1990). Sri Lanka: A Lost Revolution. Kandy, Sri Lanka: Institute of Fundamental Studies. Hall, G.C.N. & Okazaki, S. (2002). (Eds.). Asian American Psychology: The science of lives in context. Washington, DC: APA. Harter, S. (1999). The construction of the self: A developmental perspective. New York: Guilford. Henning-Stout, M. & Brown-Cheathem, M.A. (1999). School psychology in a diverse world: Considerations for practice, research, and teaching. In C.R. Reynolds & T.B. Gutkin (Eds.), The handbook of school psychology, 3rd edition (pp. 1041-1055). New York: John Wiley & Sons Inc. Hitchcock, J. H., Nastasi, B. K., Dai, D. Y., Newman, J., Jayasena, A., Bernstein-Moore, R., Sarkar, S., & Varjas, K. (2005). Illustrating a mixed-method approach for vali- dating culturally specific constructs. Journal of School Psychology, 43, 259-278. Knox, M., Funk, J., Elliott, R., & Bush, E. G. (1998). Adolescents’ possible selves and their relationship to global self-esteem. Sex Roles, 39 (1/2), 61-80. Lazarus, R. S., & Folkman, S. (1984). Stress, appraisal, and coping. New York: Springer. Nastasi, B. K., & DeZolt, D. M. (1994). School interventions for children of alcoholics. New York: Guilford. Nastasi, B. K., Varjas, K., Sarkar, S., & Jayasena, A. (1998). Participatory model of mental health programming: Lessons learned from work in a developing country. School Psychology Review, 27, 260-276. Nastasi, B. K., Varjas, K., Jayasena, A., Bernstein Moore, R., Sarkar, S., Hitchcock, J., & Burkholder, G. (2005). Sri Lanka mental health promotion project. Unpublished manuscript. Minneapolis, MN: Walden University School of Psychology. Nastasi, B.K., Moore, R. B., & Varjas, K. M. (2004). School-Based Mental Health Ser- vices: Creating Comprehensive and Culturally Specific Programs. Washington, DC: American Psychological Association. Padilla, A.M. (2001). Issues in culturally appropriate assessment. On L.A. Suzuki, J.G. Ponterotto, & P.J. Meller (Eds.), Handbook of multicultural assessment, 2nd edi- tion, (pp. 5-27). San Francisco: Jossey-Bass. Ptacek, J. T., Smith, R. E., & Dodge, K. L. (1994). Gender differences in coping with stress: When stressors and appraisal do not differ. Personality and Social Psychol- ogy Bulletin, 20, 421-430. Rogers, M. R., Ingraham, C. L., Bursztyn, A., Cajigas-Segredo, N., Esquivel, G., Hess, R. S., & Nahari, S. G., & Lopez, E. C. (1999). Best practices in providing psycho- logical services to racially, ethnically, culturally, and linguistically diverse individ- uals in the schools. School Psychology International, 20, 243-264. Rogler, L.H. (1999). Methodological sources of cultural insensitivity in mental health research. American Psychologist, 54, 424-433. 32 MULTICULTURAL ISSUES IN SCHOOL PSYCHOLOGY Romans, S. E. (1998). Undertaking research with women. In S. E. Romans (Ed.), Fold- ing Back the Shadows: A Perspective on Women’s Mental Health, (pp. 23-34). Dunedin, NZ: University of Otago Press. Sarkar, S. (2003). Gender as a cultural factor influencing mental health among the ad- olescent students in India and Sri Lanka: A cross-cultural study. Unpublished doc- toral dissertation. University at Albany, State University of New York, Albany. Schensul, J. J., & LeCompte, M. D. (1999). Ethnographer’s Toolkit (Volumes 1-7). Walnut Creek, CA: AltaMira. Snyder, C. R., & Lopez, S. J. (2002). (Eds.). Handbook of positive psychology. Oxford, UK: Oxford University. Sue, D. W., Bingham, R. P., Porche-Burke, L., & Vasquez, M. (1999). The diversifi- cation of psychology: A multicultural revolution. American Psychologist, 54, 1061-1069. Tanaka, J. S., Ebreo, A., Linn, N., & Morera, O. F. (1998). Research methods: The con- struct validity of self-identity and its psychological implications. In L. C. Lee & N. W. S. Zane (Eds.), Handbook of Asian American Psychology (pp. 21-82). Thousand Oaks, CA: Sage. Tashakkori, A. & Teddlie, C. (2003). Handbook of mixed methods in social and behav- ioral research. Sage: Thousand Oaks, CA: Sage. The World Bank in Sri Lanka (April 2003). The World Development Indicator Data- base. Retrieved April 16, 2003, from the World Bank in Sri Lanka website; http://lnweb18.worldbank.org/sar/sa.nsf/srilanka United Nations Development Programme (2002). Human Development Indicators, 2002. Retrieved on April 17, 2003. http://hdr.undp.org/reports/global/2002/en/indicator/ indicator.cfm?File=index.html. U.S. Department of Health and Human Services (1999). Mental health: A report of the Surgeon General. Rockville, MD: U.S. Department of Health and Human Services, Substance Abuse and Mental Health Administration, Center for Mental Health Ser- vices, National Institutes of Health, National Institute of Mental Health. U.S. Department of Health and Human Services (2001). Mental health: Culture, race, and ethnicity–A supplement to mental health: A report of the Surgeon General. Rockville, MD: U.S. Department of Health and Human Services, Substance Abuse and Mental Health Administration, Center for Mental Health Services. doi:10.1300/J370v22n02_02 Hitchcock et al. 33 http://lnweb18.worldbank.org/sar/sa.nsf/srilanka http://hdr.undp.org/reports/global/2002/en/indicator/