Microsoft Word - 04김성혜, 전유아.doc 61 English Teaching, Vol. 69, No. 1, Spring 2014 Test-taking Strategies of L2 Adolescent Learners: Three Multiple-choice Items and L2 Proficiency Sung Hye Kim (Korea Institute for Curriculum and Evaluation) Yuah V. Chon * (Hanyang University) Kim, Sung Hye, & Chon, Yuah V. (2014). Test-taking strategies of L2 adolescent learners: Three multiple-choice items and L2 proficiency. English Teaching, 69(1), 61-90. The demand for test-taking has become an inevitable element of Korean secondary EFL learners’ academic lives in which learners strive to excel in the high-stakes exam- oriented milieu. However, current knowledge of practitioners, such as those of teachers and administrators, may be insufficient for ascertaining if the learners actually undergo the test-taking process they anticipate and assess what they aim to test. In contrast to the product-oriented view where the primary interest is on the outcome (i.e., scores or stanine levels), the purpose of the present study was to explore 165 Korean high school learners’ test-taking process via questionnaires with proficiency and item type as the grouping variables. There was avid use of conventional reading strategies among the high proficiency learners and test management strategies among the lower proficiency learners. Results present implications for teaching learners not only to become test- wise, but also to work consistently towards building reading skills and adopt long-term learning strategies. Key words: multiple-choice, reading strategies, test-taking strategies, item type 1. INTRODUCTION In line with the cognitive perspective of learning during the 1970s (Mayer, 1992), the process-oriented view of language learning has generated interest in the investigation of learner strategies and language learning strategies (O’Malley & Chamot, 1990; Oxford, 1990). In reading, the focus has largely been on the types of reading strategies and their * Sung Hye Kim: First author; Yuah V. Chon: Corresponding author Book Centre교보문고 KYOBO 62 Sung Hye Kim · Yuah V. Chon relationships with reading proficiency (Hosenfeld, 1977; O’Malley & Chamot, 1990; Purpura, 1998), the first language’s impact on second language use (Nevo, 1989), and strategy-based instruction in language classrooms (Dreyer & Nel, 2003; Tian, 2000). However, the strategy research has been less extended to research for test-taking strategies in spite of the interactive nature of the language skills. For instance, some students even with substantial knowledge of a foreign/second language are unable to perform well in tests, and Cohen (1998) asserts that this is due to success in language tests that depend both on sufficient linguistic and strategic competence. Regarding testing situations, test-taking strategies for multiple-choice items are of primary importance in the present study since they have shown to be most susceptible to multiple-choice items (Chou, 2013; Cohen, 1998, 2006; Cohen & Upton, 2006). Also, multiple-choice items have extensively been used in second/foreign language programs to assess different aspects of language since they are regarded as highly reliable, convenient in scoring, efficient, and economical. In the Korean context, multiple-choice questions are the dominant forms of the high- stakes College Scholastic Ability Test (CSAT) so that research on the test-taking strategies can provide insights for test developers about how test-takers approach tasks. That is, if the actual test-taking process does not coincide with test developers’ (e.g., teacher-assessors) anticipation, there is a reason to doubt whether the testing tasks measured what they intended to measure. That said, the primary interest of the present study is on test-taking strategies, being essential issues in test development and validation, and illustrates the process that learners undergo to answer multiple-choice items. However, Anderson, Bachman, Perkins, and Cohen (1991), and Rupp, Ferne, and Choi (2006) have noted the scarcity of research in this area in spite of the common testing procedure used in educational contexts. Another emerging interest of the study would be to examine how different item types and learners’ L2 proficiency may prompt the use of different test-taking strategies. Most teachers and practitioners may not be aware of the different test-taking strategies of Korean adolescent learners, which remain covert due to the cognitive-oriented nature of the test- taking process. Another reason why the process-oriented view of test-taking has not received ample attention may be due to teachers being more involved in teaching English content (e.g., grammar, vocabulary) so that the development or the decisions for selecting strategies has been relegated to students. Book Centre교보문고 KYOBO Test-taking Strategies of L2 Adolescent Learners: Three Multiple-choice Items and L2 Proficiency 63 2. BACKGROUND 2.1. Test-taking Strategies for Reading Comprehension One of the main implications of a purpose for reading is that it guides readers in the selection of their strategies, and the range of skills they draw on. This is particularly pertinent for multiple-choice reading comprehension tests, because they present a context in which readers apply a variety of strategies that are unique and different from those utilized in a non-testing context. In fact, previous research demonstrates that readers adjust their strategies and engage in the type of comprehension process that most suit their purpose for reading (Alderson, 2000). In the testing context, debates about test-taking revolve around differing views on their classification. However, in general, test-taking strategies have been classified into three distinct categories: (1) language learner strategies, (2) test management strategies, and (3) test wiseness strategies (Cohen, 2006; Cohen & Upton, 2006). Language learner strategies refer to “the ways that respondents operationalize their basic skills of listening, speaking, reading and writing, as well as the related skills of vocabulary learning, grammar, and translation” (Cohen, 2006, p. 308). Users of language learner strategies draw from their reading strategies, such as skimming and scanning, memorizing text information, making educated guesses, and looking for definitions, examples, main ideas, or details of meaning in the passage. These language learner strategies, which are similar to the cognitive strategies primarily require cognitive processing to answer test questions. The second category of test-taking strategies are test management strategies which are used for responding meaningfully to test items. For example, test-takers read the questions before going to the reading passage and directly look for answers from the text. Learners may also reread information for clarification or confirmation, select options through elimination of other options, consider or postpone dealing with a question, find answers in the text through some vocabulary or synonyms, or change answers after completing other questions. Although test management strategies are considered a type of test-taking strategy, they require learners to plan, monitor, and coordinate the target language input to search for the best way to answer questions. Test management strategies thus share common features of metacognitive strategies. Test wiseness strategies, according to Cohen (2006), refer to “strategies for using knowledge of test formats and other peripheral information to answer test items without going through the expected linguistic and cognitive processes” (p. 308). This may include looking for answers in chronological order in the text or taking advantage of clues appearing in other questions or options, or choosing the longest option in a multiple-choice question without knowing its meaning. Book Centre교보문고 KYOBO 64 Sung Hye Kim · Yuah V. Chon All in all, test-taking strategies (i.e., language learner strategies, test management strategies, test wiseness strategies) have been largely investigated as part of test validation, establishing whether test takers used the expected strategies to answer test items (e.g., Cohen & Upton, 2006; Dougals & Hegelheimer, 2005; Lee & Ku, 2005; Storey, 1997). In our study, in addition to reading strategies, we collectively refer to test management and test wiseness strategies as “test-taking strategies.” Incidentally, the literature also suggests how employment of test-taking strategies may have relationships with features of the test items and learner variables. The following section reviews this. 2.2. Test-taking Strategies, Test Items, and Learner Variables The main purpose of responding to multiple-choice (MC) questions about reading passages is, undoubtedly, to answer them correctly, and so test-takers select their strategies accordingly to optimize their chances for success. That is, since testing provides a unique purpose for reading, it impacts the strategies that test-takers draw on when responding to questions, which are mediated by characteristics of the test input such as text type and question type. In relation to the focus of the present study, there is particular interest in studies that research learners’ test-taking strategies of reading in relation to MC item types and learners’ proficiency. Studies have demonstrated how item type (Alderson, 2000; Anderson, Bachman, Perkins, & Cohen, 1991; Cohen & Upton, 2006; Dollerup, Glahn, & Rosenberg Hansen, 1982; Lee & Ku, 2005; Rupp, Ferne, & Choi, 2006), as well as test-taking strategies for different languages (L1 and L2; Nevo, 1989) may have an impact on the test-taking process. Rupp, Ferne, and Choi (2006) demonstrate that item type, in particular MC items, have an influence on the response process, but that the quality and intensity of the reading comprehension process that test-takers engage in can vary considerably across items. According to their proposition, an analysis of the structure and content of MC questions on any reading comprehension test will typically reveal that very different levels of reading comprehension are assessed with different items. In a process-oriented, verbal report study related to describing the reading and test- taking strategies that test takers used with different item types on the reading section of the LanguEdge Courseware (ETS, 2002), materials developed to familiarize prospective respondents with the New TOEFL, Cohen and Upton (2006) investigated the strategies used to respond to more traditional single-selection MC formats (i.e., basic comprehension and inferencing questions) versus the new selected-response (multiple selection, drag-and- drop) reading to learn items. The findings indicated that as a whole, the reading section of the New TOEFL does, in fact, call for the use of academic reading skills for passage comprehension—at least for respondents whose language proficiency was sufficiently Book Centre교보문고 KYOBO Test-taking Strategies of L2 Adolescent Learners: Three Multiple-choice Items and L2 Proficiency 65 advanced so that they not only took the test successfully, but could also tell the researchers how they did it as well. When analyzing data on test-taking strategies employed by three students with different reading abilities when answering MC questions on the test, Lee and Ku (2005) found for three item types (i.e., understanding main ideas, understanding direct statements, and drawing inferences) that the test-takers were seen to use the strategy on ‘understanding information from the text’ more than any other strategies. The researchers interpret this to be an indication of the test-takers’ engagement of in-depth reading and inferencing processes rather than surface-structure-based reading processes. Also, the test-takers were found to use more strategies in the item types of “understanding main ideas” and “drawing inferences” than in those of “understanding direct statements.” This implies that they depended on strategies concerning discourse-intersentential relationship more frequently than sentence level information. Subject variables researched with regard to test-taking strategies have been learners’ proficiency (Carrell & Grabe, 2002; Gordon, 1987; Lee & Ku, 2005; Phakiti, 2003; Purpura, 1997, 1998, 1999; Taguchi, 2001; Yoshida-Morise, 1998; Yoshizawa, 2002) and vocabulary knowledge (Carrell & Grabe, 2002). With regard to the proficiency level of test-takers and their reported use of strategies in test-taking and their performance on the L2 tests, Purpura (1997, 1998) had test takers from 17 language centers in Spain, Turkey, and the Czech Republic answer a cognitive and metacognitive strategy questionnaire (based on the work of O’Malley & Chamot, 1990; Oxford, 1990), then take a standardized language test. When Purpura (1997, 1998) examined the relationships between strategy use and L2 test performance with high- and low-proficiency test takers, the researcher found the use of monitoring, self-evaluating, and self-testing serving as significantly stronger indicators of metacognitive strategy use for the low-proficiency group than for the high- proficiency group. In addition, it was found that high- and low-proficiency test takers, while often using the same strategies or clusters of strategies, experienced differing results when using them. Another study related to test-taking strategies and respondents’ proficiency levels involved an investigation into the relationship between the use of cognitive and metacognitive strategies on an EFL reading test and success on the test (Phakiti, 2003). The students, who were enrolled in a fundamental English course at a Thai university, took a reading achievement test (which included MC questions), followed by a cognitive– metacognitive questionnaire on what they had been thinking while responding to test items. The results suggested that the use of cognitive and metacognitive strategies had a weak but positive relationship to the reading test performance, with the metacognitive strategies reportedly playing a more significant role. In addition, the highly successful test takers reported significantly higher metacognitive strategy use than the moderately successful Book Centre교보문고 KYOBO 66 Sung Hye Kim · Yuah V. Chon ones, who, in turn, reported higher use of these strategies than the unsuccessful test takers. In their study aiming to provide construct validity of the reading comprehension section of CSAT, Lee and Ku (2005) generally found the low level learner exploiting as many test- taking strategies at his disposal as the other two readers. They note that this was in contrast to how they had expected the high-achieving test-takers to be more frequent in the use of strategies. However, they also note how the low level test-taker was not able to apply a group of strategies that are most effective in arriving at the correct response. The studies as a whole provide insight into how given test formats (i.e., MC) and item types may affect learner responses, and how these may interact with proficiency and other contextual factors. However, despite the considerable literature on learner strategies in L1 and L2 learning (Macaro, 2001), and the importance that has been placed on types of reading strategies and their relationship with reading proficiency (Grabe, 2009; Hosenfeld, 1977; O’Malley & Chamot, 1990), comparatively little research has focused on how EFL high school students apply strategies to process different MC reading items. This type of information is important because it is expected to provide information on construct validity for item writers, test developers, or teacher-assessors (Cohen & Upton, 2006; Lee & Ku, 2005). Based on previous findings, the following research questions were posed, in particular for separate interest in three types of MC questions, which respectively required skills for inferring knowledge (i.e., fill-in-the-blank), comprehensive understanding (i.e., reordering paragraphs), and factual understanding (i.e., information not true of the passage). The items have been chosen for their different testing construct, and evident problem- orientedness of the items for test-takers. To conclude, there was also analysis of how the learners’ English L2 proficiency was attributable to test-taking strategies and learning strategies that the learners had been using to improve L2 reading. 1. To what extent were the repertoire of reading and test-taking strategies employed by the learners in solving MC items? 2. How is the employment of reading and test-taking strategies different according to item type and learners’ English proficiency? 3. How do the subcategories of reading, test-taking and learning strategies for L2 reading predict the learners’ English proficiency? 3. METHODS 3.1. Participants The participants were recruited from the Seoul and Incheon area so that this study may Book Centre교보문고 KYOBO Test-taking Strategies of L2 Adolescent Learners: Three Multiple-choice Items and L2 Proficiency 67 be more representative of learners in the municipal areas. All the learners were in their sophomore years (n = 166) and there was an equal proportion of learners for different gender. Table 1 lists the distribution of the participants for location and type of schools. Information on student stanine levels with a mock version of their CSAT indicated that there was almost an equal proportion of learners from each of the levels between 1-5 (i.e., level 1 = 19.1%, level 2 = 15.3%, level 3 = 16.0%, level 4 = 17.6%, and level 5 = 16.8%), but there were a smaller portion of learners at the lower stanine levels (i.e., level 6 = 6.1%, level 7 = 5.3%, level 8 = 3.8%, and level 9 = 0%). When calculated, the mean score of 71.01/100 from the mock CSAT indicated that the learners were on average a group of high-intermediate learners. We attribute this to the learner populations from the “Autonomous High Schools (ja-youl-hyung-kong-lip-go/sa-lip-go)” whose students are considered above average. In fact, 43.37% (n = 72) of the learners were from these types of school so that data on their achievement level on the CSAT is not surprising. In addition to the learners that participated in the questionnaires, there were four learners that were also recruited for semi-structured interviews in order to gain a more detailed view of the test- taking process. There were two male and female students, and they were recruited on the basis of different stanine levels. With assistance from the teachers, the student-interviewees were chosen so that there were two students (i.e., one male and one female student) from stanine level 1 (i.e., higher proficiency), while the other two students (i.e. one male and one female student) had reached level 5 (i.e., lower proficiency) on the mock CSAT according to their teachers. However, the interview protocols were utilized more for triangulation to the quantitative analysis. TABLE 1 High School and Gender of Learners High Schools Gender Location Total Male Female KD High Sch. (Auto) 23 0 Seoul 23 PM Girls’ High Sch. 0 34 Seoul 34 HS High Sch. (Auto) 0 19 Incheon 19 SH Girls’ High Sch. (Auto) 0 30 Seoul 30 BK High Sch. 60 0 Incheon 60 Total 83 83 166 Note. Auto: Autonomous Public/ Private High Schools Book Centre교보문고 KYOBO 68 Sung Hye Kim · Yuah V. Chon 3.2. Instruments and Materials 3.2.1. Reading and test-taking strategies questionnaire Considering our interest in the reading and test-taking strategies of MC English items, we developed a questionnaire for the strategies. For the purpose of trying to distinguish the use of strategies for different test items, we concentrated mainly on three items which were each presented with subsequent common 15 questionnaire items on strategies, totaling 45 items. The taxonomy of reading and test-taking strategies was retrieved from Cohen and Upton’s (2006) coding scheme previously developed from their verbal report study. Their data collected from 32 students, representing four language groups consisted mainly of Asian test-takers (i.e., Chinese, Japanese, and Korean) so that the strategy scheme that we adopted from their study can be deemed appropriate for Korean high school learners. As Cohen and Upton (2006) categorize, we devised the strategy questionnaire by selecting the subcategories of Reading Strategies (RS), and Test-taking Strategies. Test-taking strategies were further categorized as Test management (TM) and Test wiseness strategies (TW). Items 1-8 were RS, items 9-12 and 15-1/15-2/15-3 were TM strategies, and items 13-14 were TW strategies. The TM strategies 15-1/15-2/15-3 were designed differently according to the testing construct of each item (See Appendix for strategy items). The reading and test-taking strategies were presented in the questionnaire so that learners could rate 7-point Likert scales on the extent to which they had used the strategies, subsequently after the learners had solved each of the test items. The questionnaire was designed so that the learners could attend to answering questions about the RS, TM, and TW strategies almost immediately after they had solved an item. While we were concerned about the cognitive load that would arise when participants are asked to solve a test item and report on their metacognitive knowledge of the strategies they employ for solving a reading item, we made principled decisions in limiting the number of test items to three. That is, with the three items, the number of strategies that the learner had to mark was 45 since there were 15 statements to read and check after having solved a test item. We considered the number of items to be appropriate since the time and cognitive processing involved in solving the test items had to be considered. We eventually wanted to make sure that we were not creating a fatigue effect (Dörnyei, 2010), which in some cases may lead to insincere responses. The reliability analysis for internal consistency of the items with Cronbach’s α for the questionnaire results occurred at a moderate level for each set of questions in Item 1 (RS: α = .80, TM: α = .70, TW: α = .66), Item 2 (RS: α = .83, TM: α = .70, TW α = .80), and Item 3 (RS: α = .80, TM: α = .71, TW: α = .69). Book Centre교보문고 KYOBO Test-taking Strategies of L2 Adolescent Learners: Three Multiple-choice Items and L2 Proficiency 69 For the incorporation of test items, MC questions were retrieved from the Level 2 reading section of NEAT (National English Ability Test) at http://www.neat.re.kr. The items were selected since they are illustrative of the archetypal forms of MC reading comprehension test items, and also related to the current Korean CSAT (See Appendix for the reading test items). We chose three different types of items according to how they are related to testing different constructs of reading: Inferring knowledge (“fill-in-the-blank”), Comprehensive Understanding (“reordering paragraphs”), and Factual Understanding (“information not true of the passage”). Items 1 and 2 were chosen so that they were of moderate length (i.e., more or less than 160 words), while item 3 was deliberately chosen for more reading (i.e., 266 words). The items were selected based on the different types of demands that they would make on the students’ level of cognition. That is, the “fill-in-the- blank” requires test-takers to infer the missing information based on a given passage. “Reordering paragraphs” requires the test-taker to be able to rearrange given paragraphs, and “Factual Understanding” would require learners to be able to understand the detailed information stated in a given reading passage. The features of the items are summarized in Table 2. TABLE 2 Information on Reading Comprehension Test Items Item 1 Item 2 Item 3 Skill required Inferring knowledge Comprehensive understanding Factual understanding Item type Fill-in-the-blank Reordering paragraphs Information not true of the passage Topic/ Theme Longevity of art How creative ideas take form Getting accepted for a job Length of passage 152 words 166 words 266 words 3.2.2. Retrospective semi-structured interviews In order to triangulate the results of the questionnaire on the test-taking process, four learners were recruited from the group that had completed the questionnaires. The learners were asked to report on their test-taking process by trying to recall their problem-solving process. After their general description of the process, the learners were asked if there had been execution of strategies that they had consciously used for the three items. Book Centre교보문고 KYOBO 70 Sung Hye Kim · Yuah V. Chon 3.3. Procedure The strategy questionnaire was distributed to the students at a time of the semester when they were not burdened with school work in the Fall of 2012. While kept anonymous in the questionnaire, the learners were also asked to provide their scores and stanine levels of the Mock CSAT in addition to reporting their use of reading and test-taking strategies. The learners were also asked to report on the learning strategies (i.e., what kind of resources they used to practice L2 reading) that they usually used to improve their English reading comprehension skills for later analysis. The learners were given sufficient time to solve each of the test items, and check on the extent to which they had used the strategies. Regarding stimulated recall interviews, the learners were recruited for the interviews on the basis of their stanine levels (i.e., two students from level 1 and two others from level 5). The students were interviewed in Korean by one of the researchers of the study on the same day subsequently after the students had completed the questionnaires. The interviews lasted no more than 10 minutes for each student. 3.4. Data Analysis The data obtained via the questionnaire were analyzed with SPSS 19.0. There was calculation of descriptive statistics for examining the mean scores of the reading and test- taking strategies. To examine how the strategies were employed by the learners for the three test items and learners’ English proficiency, three-way (3*3*3) mixed ANOVA was conducted with proficiency groups and item type as between-group variables, and strategies as the within-subject variable. The alpha level for analysis was set at .05 unless otherwise stated. The three subtypes of strategies were RS, TM, and TW strategies. The three item types were “fill-in-the-blank,” “reordering paragraphs,” and “information not true of the passage.” The learners’ proficiency was divided into three groups according to the learners’ stanine levels. That is, those learners who had scored levels 1-3 were categorized as the “high” proficiency, levels 4-6 as the “mid” proficiency, and levels 7-9 as the “low” proficiency. As a final analysis and to analyze the related variables comprehensively, there was examination of how the reading strategies, test-taking strategies, and learning strategies for L2 reading had contributed to the learners’ English proficiency (i.e., CSAT mock exam scores). This analysis was conducted to ascertain which of the strategies may deserve more attention for improving the learners’ English proficiency. Book Centre교보문고 KYOBO Test-taking Strategies of L2 Adolescent Learners: Three Multiple-choice Items and L2 Proficiency 71 4. RESULTS AND DISCUSSION 4.1. Reading and Test-taking Strategies for Multiple-choice Questions This section, in dealing with research question one, reports on the reading strategies and test-taking strategies employed by the learners (See Table 3). A preliminary examination of the strategies as a whole indicated that the learners had most often utilized RS, that is, RS2 Reads a portion of the passage rapidly looking for specific information, RS8 Infers the meaning of new words by using the external context (neighboring words/sentences), and RS1 Reads the whole passage carefully which ranked 1-3 respectively. TABLE 3 Mean and Rank of Reading Strategies (N = 165) M Rank of TOTAL Strategies SD RS1 Reads the whole passage carefully. 4.19 [RS3] 3 1.34 RS2 Reads a portion of the passage rapidly looking for specific information. 4.61 [RS1] 1 1.29 RS3 Repeats, paraphrases, or translates (words, phrases, or sentences) to aid or improve understanding. 3.76 [RS6] 10 1.33 RS4 Identifies/Marks an unknown word or phrase 2.74 [RS8] 15 1.31 RS5 Adjusts comprehension of the passage as more is read 3.94 [RS4] 7 1.39 RS6 Identifies the key words of the passage as reading. 3.26 [RS7] 14 1.42 RS7 Looks for the sentence that conveys the main idea. 3.77 [RS5] 9 1.44 RS8 Infers the meaning of new words by using the external context (neighboring words/sentences) 4.23 [RS2] 2 1.49 Note. RS: Reading Strategies; [ ]: Rank within strategy type In fact, regarding RS8 (i.e., word attack skill; Nuttall, 1996), the strategy would have been employed when learners need to decode unknown words or phrases. For instance, in item 3, learners may have had a lexical gap problem due to the figurative meaning for out of the running (i.e., no longer being considered, or eliminated from a contest) (see Appendix for the passage). However, those learners that are successful in inferring the meaning of the new phrase may have been able to figure out from “Amy had an interview with the company she’d always wanted to work for, but was crushed” that Amy had not made it to the job. The interviews, in fact enabled the researchers to ascertain what the learners did when they had lexical gap problems. Between students of different proficiency (i.e., stanine Book Centre교보문고 KYOBO 72 Sung Hye Kim · Yuah V. Chon levels 1 and 5), the higher proficiency learner reported that the words had not been difficult anyway so that there was not much of a problem. The learner reported that sometimes skipping words had not caused him much problem. On the other hand, the lower proficiency learner clearly recognized that it was vocabulary that had caused problems for him. The following interview protocol of the lower proficiency learner from item 1 is presented below, translated from Korean: Researcher: So what was difficult about the test item? Learner (male, lower): The problem were the words….I knew the words that are useless for solving the problem. It seemed that I did not know the important words that are needed for finding the answer. Researcher: What kinds of words do you mean? Learner: Words such as adjectives…refuge, longevity, circumstances, permanently….They seem to be the words you need to know. The interview shows that the learner was clearly having lexical problems from the way he could not distinguish part of speech. He referred to the unknown words (i.e., refuge, longevity, circumstances, permanently) as adjectives. In fact, the student-interviewee later also reports that it is important for him to be able to infer the meaning of new words by using the external context, but he ends up with the wrong answer (② artistic tradition is based on culture) which shows that his execution of the strategy has been unsuccessful. According to Nation (2001), guessing from context will work only when more than 95% of the words are known. However, it can be understood from the learner’s proficiency that he will have probably known less than this pre-conditional level to be able to decode the given text successfully. Simultaneously, it was also reading strategies, as in, RS4 Identifies/Marks an unknown word or phrase, and RS6 Identifies the key words of the passage as reading that had been least useful for the learners. We suspect that some learners had not needed to physically underline the words or phrases, but another explanation for this may be that the test takers, when under time pressure, will use strategies they think are most efficient to finding the correct response. As seen among learners in Purpura’s (1999) study, this connects to how the learners employed a product-oriented attitude where they wanted to answer the items quickly and efficiently rather than spend time trying to comprehend or understand test input of the multiple choice item. As a whole, the pattern of RS indicates that reading for tests will also entail a similar process to the learners’ usual reading process where TM or TW did not necessarily rank high on the total rank of the strategies (See Table 4). Regarding test management strategies (TM), the item specific strategies as in 15-1. Insert option(s) in the blank, and considers the new sentence in context, 15-2. Considers the passage after having rearranged the passage, and 15-3. Consider each of the options against what is stated in the passage were most frequently used for the respective items, Book Centre교보문고 KYOBO Test-taking Strategies of L2 Adolescent Learners: Three Multiple-choice Items and L2 Proficiency 73 that is, item 1 (“fill-in-the-blank”), item 2 (“reordering paragraphs”), and item 3 (“finding information not true of the passage”). For item 2, the strategies reported in the interviews indicated that one of the learners at the high proficiency level was generally relying on their prediction and background knowledge to foresee what should follow. The female student reported that she had used her schema in figuring out the outline of the story (i.e., picks an oyster off the bottom � paddles to shore � opens the shell � finds nothing but an oyster inside � “He’s wasting a lot of time. You’re right.” � Pearls are rare; a diver must open many oysters before finding one.) TABLE 4 Test Management and Test Wiseness Strategies (N = 165) M Rank of TOTAL Strategies SD TM9 Goes back/Rereads the question or passage for clarification. 3.95 [TM3] 6 1.33 TM10 Considers the options before reading the passage. 4.02 [TM2] 5 1.45 TM11 Considers the options and selects preliminary option(s) 3.46 [TM5] 13 1.42 TM12 Makes an educated guess using background knowledge 3.61 [TM4] 11 1.48 TM15-1/15-2/15-3 See Note (below) 4.12 [TM1] 4 1.42 TW13 Uses the process of elimination 3.81 [TW1] 8 1.34 TW14 Selects the option because it appears to have a word or phrase from the passage in it/ when some words, such as, key words help in selecting the answer 3.49 [TW2] 12 1.36 Note: TM: Test Management Strategies; TW: Test Wiseness Strategies; [ ]: Rank within strategy type; 15-1: Insert option(s) in the blank, and considers the new sentence in context; 15-2: Considers the passage after having rearranged the passage; 15-3: Considers each of the options against what is stated in the passage. Similarly, TM 15-3 (i.e., Consider each of the options against what is stated in the passage) would have helped learners most efficiently find the option that is not consistent with the information in the passage. That is, by considering the options, the learner would have been able to see that “ She was hired by the company without any interview ④ process” is untrue of the passage since the passage states that “Amy was interviewed and was finally in her dream job” (see Appendix for the passage). As previously mentioned, test wiseness strategies (TW) are those that are employed by test takers to answer items by using knowledge of test formats and other peripheral information without going through the expected linguistic and cognitive processes (Cohen, 2006). For TW strategies as indicated in Table 4, the mean values demonstrate that TW13 Book Centre교보문고 KYOBO 74 Sung Hye Kim · Yuah V. Chon Uses the process of elimination was a more useful strategy than TW14 Selects the option because it appears to have a word or phrase from the passage in it/ when some words, such as, key words help in selecting the answer. However, simply the descriptive figures do not provide whether this was related to any learner variables or item type. For a more detailed analysis, the following section will examine how the use of reading and test-taking strategies transpired according to learners’ proficiency and item type. 4.2. Relationship Between Test-taking Strategies, Test Items, and L2 Proficiency This section reports on the second research question which focuses on investigating how the reading and test-taking strategies were employed according to item type and learners’ English proficiency groups. Our analysis was conducted by first looking at the largest interaction and then working backwards, since main effects for single variables may not retain their importance in the face of interactions (Larson-Hall, 2010). The largest interaction, which is the three-way interaction between “item type,” “strategy type,” and the “three proficiency groups” was non-significant with Greenhouse-Geisser (F(6.069, 333.775) = 1.784, p = .101). However, Greenhouse-Geisser indicated significant two-way interactions respectively for “strategy type” and “three proficiency groups” (F(3.570, 196.346) = 2.677, p = .039), and “item type” and “strategy type” (F(3.034, 333.775) = 7.480, p = .000). The detailed comparisons are described in the subsequent sections. 4.2.1. Strategy type and proficiency groups The descriptive statistics for “strategy type” and “three proficiency groups” were observed as in Table 5. The “high” proficiency learners most frequently used strategies in the order of RS, TW, and TM. Both the “mid” and “low” proficiency learners favored the use of TM while the “mid” proficiency learners were more frequent in their use of RS. As a whole, it was the more proficient test takers that were keener on using the rudimentary RS while the less proficient learners were more conspicuously noted for using metacognitive strategies (e.g., TM), probably to make up for their perceived incomplete knowledge of English as found in previous studies (Purpura, 1997, 1998). A more detailed analysis via mixed ANOVA indicated significant differences between the use of three strategies and proficiency groups (F(3.570, 196.346) = 2.677, p = .039). Multiple comparisons indicated that the high proficiency learners (p = .018) had employed more RS than TM, and the low proficiency learners had employed more TM rather than RS (p = .020) (See Table 6). The explanation for the results is that the high proficiency learners preferred the employment of their conventional RS, and it was deemed necessary to see Book Centre교보문고 KYOBO Test-taking Strategies of L2 Adolescent Learners: Three Multiple-choice Items and L2 Proficiency 75 which of the strategies they favored to adopt. The descriptive statistics on the “high” proficiency learners’ strategies indicated that RS2 Reads a portion of the passage rapidly looking for specific information had been most frequently used whereas RS4 Identifies/Marks an unknown word or phrase had been least used. In fact, the use of RS2 Reads a portion of the passage rapidly looking for specific information was statistically different between groups (F(2,126) = 5.101, p = .007) with the higher proficiency learners more frequently using the strategy than the low proficiency learners (i.e., Mean: high = 4.81, mid = 4.60, low = 3.58). There were statistical differences between the high and low (p = .005), and between the mid and low proficiency groups (p = .030), but not between the high and mid proficiency groups (p = 1.000). TABLE 5 Proficiency and Test-taking Strategies Proficiency Strategy M 95% Confidence Interval Lower Bound Upper Bound High RS 3.913 (1) 3.728 4.201 TM 3.730 (3) 3.191 3.736 TW 3.772 (2) 3.702 4.272 Mid RS 3.906 (2) 3.830 4.362 TM 4.070 (1) 3.523 4.136 TW 3.893 (3) 3.623 4.264 Low RS 2.962 (3) 2.715 3.793 TM 3.370 (1) 2.445 3.685 TW 3.212 (2) 2.576 3.874 Note. Numbers in ( ) indicate the rank for the mean of strategies; RS: Reading Strategies; TM: Test Management Strategies; TW: Test Wiseness Strategies TABLE 6 Post-hoc tests for Proficiency and Test-taking Strategies Proficiency (I) Strategy (J) Strategy Mean Difference (I-J) Sig. 95% Confidence Interval for Difference Lower Bound Upper Bound High RS TM .183 * .018 .032 .334 TW .141 .179 -.065 .348 TM TW -.042 .654 -.228 .143 Mid RS TM -.163 .059 -.333 .007 TW .014 .906 -.219 .246 TM TW .177 .096 -.032 .386 Low RS TM -.408 * .020 -.751 -.064 TW -.250 .294 -.720 .220 TM TW .158 .461 -.265 .580 *p < .05 Book Centre교보문고 KYOBO 76 Sung Hye Kim · Yuah V. Chon In contrast, the “low” proficiency learners had most popularly deployed TM10 Considers the options before reading the passage (M = 4.06, SD = 1.60), and the least for TM9 Goes back/Rereads the question or passage for clarification (M = 2.88, SD = 1.27). Scrutiny of other TM indicated that TM11 Considers the options and selects preliminary option(s) had been used more often by the mid proficiency learners than the high proficiency learners (p = .025). The pattern, as seen previously (Table 6), indicates that the more proficient learners may display a sense of confidence by relying on their strategic deployment of reading skills whereas the low proficiency learners will try to find ways to overcome their linguistic deficiency by using test taking tactics. In parallel, the learners were also reluctant to go back to the questions or the passage to check on their comprehension or the correctness of their response. Similarly, Lee and Ku (2005) found in their study that the “low” proficiency learner exploited as many test-taking strategies as he could, but that the execution of strategies was found not to be effective for obtaining the correct response. All in all, the findings on the strategies and proficiency demonstrate a relationship between learners’ proficiency and their preferences for different types of strategies (Phakiti, 2003; Purpura, 1997, 1998, 1999). Hereafter, how the use of strategies differed for the separate test items will be presented. 4.2.2. Item type and strategy type Regarding the significant interaction between “item type” and “strategy type,” Table 7 indicates how some strategies were more constructively employed according to item type. The RS were most frequently utilized for item 3 (i.e., not true of the passage) (M = 3.643) whereas TM (M = 3.824) and TW strategies (M = 3.925) were most often deployed for item 1 (i.e., fill-in-the-blank). Not surprisingly, it was RS2 Reads a portion of the passage rapidly looking for specific information (M = 4.58, SD = 1.604) that indicated to be most commonly used in item 3. The item will have entailed test-takers to do some careful reading when figuring out the mismatching piece of information. On the other hand, item 1 prompted the most frequent use of TM9 Goes back/Rereads the question or passage for clarification (M = 4.19, SD = 1.636). That is, to solve a “fill-in-the-blank” item, learners are likely to reread the passage in their attempt to find the best contextually matching word, phrase, or sentence. Regarding the use of TW strategies for item 1, two strategies were employed most frequently to a similar extent by the learners (i.e., TW13 Uses the process of elimination: M = 3.88, SD = 1.775; TW14 Selects the option because it appears to have a word or phrase from the passage in it: M = 3.86, SD = 1.682). For information on inferential statistics with mixed ANOVA, significant differences were noted only within TW strategies for different items (F(3.034, 333.775) = 7.480, p = .000) Book Centre교보문고 KYOBO Test-taking Strategies of L2 Adolescent Learners: Three Multiple-choice Items and L2 Proficiency 77 (See Table 8). Multiple comparisons indicated that the TW strategies had been used more often in item 1 (i.e., fill-in-the-blank) compared to item 2 (i.e., reordering the passage) (p = .000), and that the strategies had been employed more often in item 3 (i.e., not true of the passage) over item 2 (p = .000). TABLE 7 Test-taking Strategies and Item Types Strategy Items M 95% Confidence Interval for Difference Lower Bound Upper Bound RS Fill-in- the-blank 3.565 (3) 3.349 3.781 Reordering the passage 3.573 (2) 3.330 3.816 Not true of the passage 3.643 (1) 3.400 3.887 TM Fill-in- the-blank 3.824 (1) 3.564 4.083 Reordering the passage 3.637 (3) 3.369 3.906 Not true of the passage 3.708 (2) 3.421 3.995 TW Fill-in-the-blank 3.925 (1) 3.621 4.229 Reordering the passage 3.148 (3) 2.797 3.500 Not true of the passage 3.803 (2) 3.475 4.132 Note. Numbers in ( ) indicate the rank for the frequency of strategies TABLE 8 Post-hoc tests for Item Type and Test-taking Strategies Strategy (I) Items (J) Items Mean Difference (I-J) Sig. 95% Confidence Interval for Difference Lower Bound Upper Bound Reading strategies 1 2 -.007 .932 -.179 .165 3 -.078 .376 -.251 .096 2 3 -.070 .374 -.227 .086 Test management strategies 1 2 .187 .092 -.031 .405 3 .116 .342 -.125 .356 2 3 -.071 .449 -.256 .114 Test wiseness strategies 1 2 .776 ** .000 .397 1.156 3 .121 .473 -.213 .455 2 3 -.655 ** .000 -.960 -.351 Note. Item 1: fill-in-the-blank; Item 2: reordering the passage; Item 3: not true of the passage In order to examine which of the TW strategies were contributing to this effect, one-way repeated-measures ANOVA was conducted for the individual TW strategies. For TW13 Uses the process of elimination, there was a difference (F(2, 328) = 11.706, p = .000) in the use of strategies where item 1 (i.e., fill-in-the-blank; M = 3.87) had more often prompted Book Centre교보문고 KYOBO 78 Sung Hye Kim · Yuah V. Chon the use of the strategy over item 2 (i.e., reordering the passage; M = 3.38) at a significant level (p = .011); item 3 (i.e., not true of the passage; M = 4.18) had also prompted more use of the strategy over item 2 (p = .000). The way TW13 was utilized underscores the importance that test takers attribute to the process of elimination so as to narrow down the target option, particularly in item 3. Another examination involved the use of TW14 Selects the option because it appears to have a word or phrase from the passage in it which resulted in significant differences between the items (F(2, 326) = 27.343, p = .000). Multiple comparisons indicated the strategy to be used more often in item 1 (i.e., fill-in-the-blank; M = 3.86) over item 2 (M = 2.95) at a significant level (p = .000); item 3 (i.e., not true of the passage; M = 3.65) also prompted more use of the strategy over item 2 (p = .000). The execution of TW14 illustrates how learners were trying to be test smart by spotting words in the passage and the options that may relate to each other, noticeably in item 1. The following interview protocol illustrates the use of both TW13 and TW14, and shows how the lower proficiency learner is unsuccessful: Researcher: When you were attending to the problem, what kind of an elimination process do you go through? Learner (male, lower ): I delete [an option] when it doesn’t fit in with the general idea of the passage. Researcher: How do you do this? Learner: In option number 4, I deleted the option because it has the word “philosophy” that I don’t know. In option number 3, conducting art does not seem to be related to human creativity, so I had to delete it. There has to be the word “test” but it’s not in the option. Option number 1 translates as “the best art stands the test of time,” but from what I read there is reference to “time” but it’s not what the passage is about so that I had to delete it. As such, the interview protocol demonstrates that he wrongly chose “② artistic tradition is based on culture” as the correct response. Although he had evidently used the strategy of elimination, the outcome illustrates that he has failed to select the target option. For instance, the learner could have inferred the main idea of the passage by noticing the sentence, such as “the artist’s work continues to have something new to say,” but he most critically does not understand the meaning of “test” in the option. This accords with the type of behavior found in previous research (Cohen & Upton, 2006) where students use a strategy of matching material from the passage with those in the item stem and in the alternatives, and prefer this surface-structure reading of the test items to one that calls for more in-depth reading and inferencing. The results as a whole allowed us to see that in spite of the most frequent use of strategies for item 3 (M = 3.900) when calculated separately, the total mean for the Book Centre교보문고 KYOBO Test-taking Strategies of L2 Adolescent Learners: Three Multiple-choice Items and L2 Proficiency 79 individual strategies indicated that it was item 1 (i.e., fill-in-the-blank) that had necessitated the learners to use a variety of strategies (i.e., TM and TW strategies) for finding the problematic missing sentence. That is, the mean value was highest for the previously mentioned TW13 Uses the process of elimination, and the test taker would have been successful if the learner had been able to eliminate options and find that the passage is an exposition on how art has been able to last for a long time (i.e., ① The best art stands the test of time). 4.3. Strategies of Reading, Test-taking and Learning as Predictors of L2 Proficiency For a more comprehensive view of the learners’ test-taking behavior, the third research question concerned how variables (predictors), that is, the reading strategies, test-taking strategies, and learning strategies contributed to explaining L2 proficiency. The learning strategies had been reported by the learners as a part of the procedure for responding to the questionnaire where the learners were asked on the different types of learning strategies they usually used to improve their English reading skills. The choices were ① School Lesson, Exercise ② Book, ③ Private Education, Miscellaneous, and ④ ⑤ Don’t study. Using the enter method with multiple regression, a significant model emerged: F(7, 91) = 3.067, p = .006. Table 9 presents information for the predictors entered into the model. The results indicated RS (p = .008), TM (p = .032), and the learning strategies with using exercise books (p = .004) and private education (p = .018) to be significant for predicting the learners’ English proficiency scores, which in the study had been assessed by the proficiency test (i.e., CSAT). In fact, when compared to those who had relied mainly on school lessons for reading, the unstandardized coefficients (B) indicated that scores had increased by 19 points for learners studying with exercise books and more than 13 points for those that had reported on attending private institutes. Regarding the use of strategies, the unstandardized coefficients (B) indicated that a rise of a level in the use of a RS is related to a gain of approximately 12 points on the CSAT. In general, this indicates that the conscious employment of RS is a contributing factor for obtaining higher scores on the CSAT, potentially leading to improved L2 proficiency. Book Centre교보문고 KYOBO 80 Sung Hye Kim · Yuah V. Chon TABLE 9 Reading, Test-taking and Learning Strategies as Predictors of L2 Proficiency Note. *p < .05, **p < .01; RS Total: Total mean for Reading strategies, TM Total: Total mean for Test management strategies, TW Total: Total mean for Test wiseness strategies In comparison, those learners who had reported using TM strategies seem to have experienced a fall in their proficiency test (i.e., a drop of 10.852 points). We attribute this to how the use of the strategies per se had not been sufficient to compensate for the L2 learners’ linguistic deficiency. As seen in Lee and Ku’s (2005) study, the use of strategies, particularly among the low-proficiency learners did not necessarily lead to the successful use of strategies, which may imply that the learners depended on strategies concerning discourse-intersentential relationship more frequently than sentence level information. Nevertheless, the fall in scores compelled us to conduct a detailed analysis for the individual items in order to seek which of the TM strategies in the items had been rather detrimental to the test taking process. For all items, the regression models resulted to be significant (item 1: F(5, 108) = 7.243, p = .000; item 2: F(5,108) = 3.338, p = .008; item 3: F(5,109) = 2.366, p = .044) for explaining the relationship between TW strategies and L2 proficiency. For item 1 (i.e., fill-in-the-blank), we found that those strategies that are related to considering the options, such as Considers the options before reading the passage (p = .000) or Considers the options and selects preliminary option(s) (p = .012) had not been helpful for gaining higher scores or in finding the target option since their unstandardized coefficients (B) recorded negative values (-4.829, -3.635) for each of the strategies. In comparison, it was the TM strategy as in Goes back/Rereads the question/passage for clarification (i.e., rise of 4.418 in proficiency) that is expected to benefit the learners’ problem-solving process. For item 2 (i.e., reordering paragraphs), it was the item specific TW strategy as in Consider the passage after having rearranged the passage that contributed significantly (p = .032) with unstandardized coefficients (B) of 2.957 indicating a positive effect on the Unstandardized Coefficients Standardized Coefficients t Sig. B Std. Error Beta (Constant) 54.490 11.125 4.898** .000 RS Total 12.331 4.544 .489 2.714** .008 TM Total -10.852 4.973 -.474 -2.182* .032 TW Total .351 3.341 .017 .105 .917 School vs. Exercise Book 18.937 6.325 .320 2.994** .004 School vs. Private Ed 13.477 5.578 .267 2.416* .018 School vs. Miscellaneous 1.730 22.714 .007 .076 .939 School vs. Don’t study separately 8.115 8.900 .103 .912 .364 Book Centre교보문고 KYOBO Test-taking Strategies of L2 Adolescent Learners: Three Multiple-choice Items and L2 Proficiency 81 performance of CSAT. In a similar way, the learners who had been using the item specific strategy Consider each of the options against what is stated in the passage (p = .003, unstandardized coefficients (B) = 4.091) as in item 3 (i.e., information not true of the passage) seemed to have performed better on the CSAT, reflecting improved L2 performance. 5. CONCLUSION Reading research on the process-oriented view of the test-taking process of Korean EFL high school learners yielded results that inform us on the strategic actions that learners adopt to obtain correct responses for MC items. A collective examination of the reading and test-taking strategies revealed that the learners’ test-taking process much resembled a general reading process, such as, Reading a portion of the passage rapidly to look for specific information, followed by strategies for solving a lexical gap problem, such as in Inferring the meaning of new words by using the external context. Interest in the proficiency level of learners with respect to the use of strategies produced significant results demonstrating a relationship between learners’ proficiency and their preferences for different types of strategies (Phakiti, 2003; Purpura, 1997, 1998, 1999). It was with the high proficiency learners that we saw efficient use of strategies, and this is likely when they do not experience the same type of difficulties that the low proficiency learners go through, for instance, with the case of lexical gap problems that were recognized in the interviews. Seemingly to overcome their deficiencies, the less proficient learners were more conspicuously noted for using metacognitive strategies, that is, TM. The use of TM as a whole illustrates that these may be a separate repertoire of strategies that learners will deploy when they find that their usual reading strategies are insufficient to help them decode the reading passage and obtain the correct response (Cohen & Upton, 2006). Analysis also indicated that the use of strategies was associated with particular items (Alderson, 2000; Anderson, Bachman, Perkins, & Cohen, 1991; Cohen & Upton, 2006; Dollerup, Glahn, & Rosenberg Hansen, 1982; Lee & Ku, 2005; Rupp, Ferne, & Choi, 2006). For the three items, RS (i.e., Reads a portion of the passage rapidly looking for specific information) were most frequently utilized in item 3 (i.e., not true of the passage) whereas TM and TW strategies were most often deployed for item 1 (i.e., fill-in-the-blank). For item 1, the most common TM was Goes back/Rereads the question or passage for clarification whereas the most common TW were Uses the process of elimination and Selects the option because it appears to have a word or phrase from the passage in it. This evidences how the “fill-in-the-blank” type of items are problematic for the learners, Book Centre교보문고 KYOBO 82 Sung Hye Kim · Yuah V. Chon compelling them to use an array of strategies perceived to them as being successful. Concurrently, we also found that TW Uses the process of elimination was a critical strategy for item 3 where learners had to find the option that did not corresponded with the given passage, and TW Selects the option because it appears to have a word or phrase from the passage in it was seen as an important strategy for the learners in item 1, which tested them on the ability to infer the missing sentence based on their understanding of the given reading passage. Multiple regression analysis with the predictors and English proficiency resulted in the use of RS, TM, and the learning strategies with using exercise books and private education to be the significant predictors for learners’ English proficiency. In fact, it was only with the selective choice of TM for specific items that benefited the learners’ English proficiency. For instance, in item 1, Considering the options before reading the passage or Considering the options and selecting preliminary option(s) were rather detrimental to the outcome. We can deduce that TW and any other kinds of test-taking strategies will be effective only when learners are facilitated with the threshold level of linguistic knowledge (e.g., vocabulary, grammar) that can be boosted for improved outcome by the use of reading and test-taking strategies. All in all, the study proposes implications for L2 reading and reading tests. First, our study was able to foresee that to perform well on tests of L2 reading, some of the preconditions for this would be to see that the learners are able to adopt basic reading strategies and practice them on a regular basis. As seen in the regression model, the utility of the reading strategies cannot be overlooked since this also appeared to have positive effects on L2 proficiency. In a similar vein, reading researchers have also shown that an integrated comprehension of a text relies heavily on fluid, accurate, and efficient application of bottom-up processes (e.g., Aebersold & Field, 1997; Anderson & Pearson, 1984; Carrell, 1984; Rumelhart, 1980; Stanovich, 1980). Teachers and learners will need to see that solving MC items of reading is a matter concerned with basic reading skills rather than the development on a variety of test-taking strategies, such as for test management or test wiseness. Another implication from the study is that learners will need to be eclectic in the execution of TM strategies (e.g., Considering the options before reading the passage) since some may rather be detrimental to the problem-solving process as we saw with Item 1 (i.e., fill-in-the-blank). Third, the problems of the low proficiency learners demonstrated in the interviews that some learners will need to be trained to meet words in context, preferably in dealing with a variety of topics. For instance, we saw how the learners were having problems with the polysemy “test” in the sentence “the best art stands the test of time.” Last but not least, the results also suggest how the extra time spent at the private institute, when based on self-regulation, may have a contributing effect on the development of L2 Book Centre교보문고 KYOBO Test-taking Strategies of L2 Adolescent Learners: Three Multiple-choice Items and L2 Proficiency 83 proficiency. Inevitably, it seems that time well spent, for instance, at a private institute with utilization of exercise books for MC items will potentially have a positive effect for improving learners’ L2 proficiency. Nevertheless, it will be long-term employment of learning strategies for reading (e.g., extensive reading; Day & Bamford, 2009) that can lead to sustaining positive outcomes on an L2 reading test. REFERENCES Aebersold, J., & Field, M. (1997). From reader to reading teacher: Issues and strategies for second language classrooms. Cambridge: Cambridge University Press. Alderson, J. C. (2000). Assessing reading. New York: Cambridge University Press. Anderson, N. J., Bachman, L., Perkins, K., & Cohen, A. (1991). An exploratory study into the construct validity of a reading comprehension test: Triangulation of data sources. Language Testing, 8(1), 41-66. Anderson, R., & Pearson, P. (1984). A schema-theoretic view of basic processes in reading comprehension. In, P. D. Pearson (Ed.), Handbook of reading research (pp. 255- 291). New York: Longman. Carrell, P. L. (1984). Schema theory and ESL reading: Classroom implications and applications. Modern Language Journal, 68, 332-343. Carrell, P. L., & Grabe, W. (2002). Reading. In N. Schmitt (Ed.), An introduction to applied linguistics (pp. 233-250). London: Arnold. Chou, M.-H. (2013). Strategy use for reading English for general and specific academic purposes in testing and nontesting contexts. Reading Research Quarterly, 48(2), 175-197. Cohen, A. D. (1998). Strategies and processes in test taking and SLA. In L. F. Bachman & A. D. Cohen (Eds.), Interfaces between second language acquisition and language testing research (pp. 90-111). New York: Cambridge University Press. Cohen, A. D. (2006). The coming of age of research on test-taking strategies. Language Assessment Quarterly, 3(4), 307-331. Cohen, A. D., & Upton, T. A. (2006). Strategies in responding to the new TOEFL reading tasks (Monograph No. 33). Princeton, NJ: ETS. Retrieved June 16, 2012, from www.ets.org/Media/Research/pdf/RR-06-06.pdf. Day, R. R., & Bamford, J. (2009). Extensive reading in the second language classroom (9 th ed.). Cambridge: Cambridge University Press. Dollerup, C., Glahn, E., & Rosenberg Hansen, C. (1982). Reading strategies and test- solving techniques in an EFL-reading comprehension test—A preliminary report. Journal of Applied Language Study, 1(1), 93-99. Book Centre교보문고 KYOBO 84 Sung Hye Kim · Yuah V. Chon Dörnyei, Z. (2010). Questionnaires in second language research: Construction, administration, and processing (2 nd ed.). London: Routledge. Douglas, D., & Hegelheimer, V. (2005). Cognitive processes and use of knowledge in performing new TOEFL listening tasks (2nd Interim Report to Educational Testing Service). Ames, IA: Iowa State University. Dreyer, C., & Nel, C. (2003). Teaching reading strategies and reading comprehension within a technology-enhanced learning environment. System, 31(3), 349-365. ETS. (2002). LanguEdge Courseware score interpretation guide. Princeton, NJ: Author. Gordon, C. (1987). The effect of testing method on achievement in reading comprehension tests in English as a foreign language. Unpublished master’s thesis. Tel-Aviv University, Tel-Aviv. Grabe, W. (2009). Reading in a second language: Moving from theory to practice. New York: Cambridge University Press. Hosenfeld, C. (1977). A preliminary investigation of the reading strategies of successful and nonsuccessful second language learners. System, 5(2), 110-123. Larson-Hall, J. (2010). A guide to doing statistics in second language research using SPSS. New York: Routledge. Lee, J.-W., & Ku, M. (2005). Yeongeo Ilgi Siheom-Suhaeng Jeonryak Sayonge Gawnhan Sarye Yeongu (A case study on the use of English reading test-taking strategies). Yeongeo KyoyukYeongu (English Language Teaching), 17(2), 179-201. Macaro, E. (2001). Learning strategies in foreign and second language classrooms. London: Continuum. Mayer, R. E. (1992). Cognition and instruction: Their historic meeting within educational psychology. Journal of Educational Psychology, 84, 402-412. Nation, I. S. P. (2001). Learning vocabulary in another language. New York: Cambridge University Press. Nevo, N. (1989). Test-taking strategies on a multiple-choice test of reading comprehension. Language Testing, 6(2), 199-215. Nuttall, C. (1996). Teaching reading skills in a foreign Language. Oxford: Heinemann. O’Malley, J. M., & Chamot, A. U. (1990). Learning strategies in second language acquisition. Cambridge, UK: Cambridge University Press. Oxford, R. L. (1990). Language learning strategies: What every teacher should know. New York: Newbury House/Harper & Row. Phakiti, A. (2003). A closer look at the relationship of cognitive and metacognitive strategy use to EFL reading achievement test performance. Language Testing, 20(1), 26-56. Purpura, J. E. (1997). An analysis of the relationships between test-takers’ cognitive and metacognitive strategy use and second language test performance. Language Learning, 47(2), 289-325. Book Centre교보문고 KYOBO Test-taking Strategies of L2 Adolescent Learners: Three Multiple-choice Items and L2 Proficiency 85 Purpura, J. E. (1998). Investigating the effects of strategy use and second language test performance with high- and low-ability test-takers: A structural equation modelling approach. Language Testing,15(3), 333-379. Purpura, J. E. (1999). Learner strategy use and performance on language tests: A structural equation modeling approach. Cambridge, UK: Cambridge University Press. Rumelhart, D. (1980). Schemata: The building blocks of cognition. In R. J. Spiro, B. C. Bruce, & B. W. Brewer (Eds.), Theoretical issues in reading comprehension. (pp. 33-58). NJ: Erlbaum. Rupp, A. A., Ferne, T., & Choi, H. (2006). How assessing reading comprehension with multiple-choice questions shapes the construct: A cognitive processing perspective. Language Testing, 23(4), 441–474. Stanovich, K. (1980). Toward an interactive-compensatory model of individual differences in the development of reading fluency. Reading Research Quarterly, 16, 32-71. Storey, P. (1997). Examining the test-taking process: A cognitive perspective on the discourse cloze test. Language Testing, 14(2), 214-231. Taguchi, N. (2001). L2 learners’ strategic mental processes during a listening test. JALT Journal, 23(2), 176-201. Tian, S. (2000). TOEFL reading comprehension: Strategies used by Taiwanese students with coaching-school training. Unpublished doctoral dissertation. Columbia University, New York. Yoshida-Morise, Y. (1998). The use of communication strategies in language proficiency interviews. In R. Young & A. W. He (Eds.), Talking and testing: Discourse approaches to the assessment of oral proficiency (pp. 205-238). Amsterdam: John Benjamins. Yoshizawa, K. (2002). Relationships among strategy use, foreign language aptitude, and second language proficiency: A structural equation modeling approach. Unpublished doctoral dissertation. Temple University, Philadelphia. Book Centre교보문고 KYOBO 86 Sung Hye Kim · Yuah V. Chon APPENDIX ▣ Item 1 (“fill-in-the-blank”) ※ 다음 문항의 정답은 무엇입니까? Sometimes art can be a refuge from life, and in extreme cases it is a second chance at life. One familiar way to describe the relative longevity of art and life is to say that what makes great art great is that it remains eternally young, while we don’t. It is a common place observation that Henri Matisse, Pablo Picasso or Vincent van Gogh becomes like a new artist every time his work is shown because, as time passes, circumstances change, generations change, but the artist’s work continues to have something new to say. Although a Matisse painting is a finite and finished object, it remains in flux and permanently fresh, affecting and being affected by other art over time. As the critic John Russel simply phrased it, "What we have seen this week will not look the same when we see it again next week." In other words, you might say _____________. *flux: 끊임없는 변화 1. Which one best fits in the blank? ① the best art stands the test of time ② artistic tradition is based on culture ③ art is the product of human creativity ④ art reflects the philosophy of a society ※ 1 번 문항 풀이에 적용한 읽기 전략의 사용 정도를 표시하여 주세요. 읽기 전략 전혀 아니다 대체로 아니다 약간 아니다 보통 이다 약간 그렇다 대체로 그렇다 매우 그렇다 1. 전체 지문을 주의 깊게 읽었 다. 1 2 3 4 5 6 7 2. 필요한 정보를 찾기 위하여 지 문의 일부를 빠르게 읽었다. 1 2 3 4 5 6 7 3. 내용을 이해하기 위하여, 반복 하여 읽고, 다른 말로 풀어보 거나 번역을 하여 보았다. 1 2 3 4 5 6 7 4. 이해가 되지 않는 단어나 문장 들을 표시하며 읽었다. 1 2 3 4 5 6 7 Book Centre교보문고 KYOBO Test-taking Strategies of L2 Adolescent Learners: Three Multiple-choice Items and L2 Proficiency 87 5. 예측했던 바와 다른 내용이 나 오면 새로운 정보에 따라 나의 이해력을 조정하 다. 1 2 3 4 5 6 7 6. 지문의 핵심어를 표시하며 읽 었다. 1 2 3 4 5 6 7 7. 주제를 나타내는 문장을 찾아 보았다. 1 2 3 4 5 6 7 8. 모르는 단어의 뜻을 문맥을 통 하여 파악하 다. 1 2 3 4 5 6 7 9. 내용을 명료하게 이해하도록 질문을 다시 읽어 보았다. 1 2 3 4 5 6 7 10. 지문을 읽기 전에 선택지를 먼 저 읽어보았다. 1 2 3 4 5 6 7 11. (답이 확실치 않을 경우) 정답 인 것 같은 선택지에 미리 표 시하여 보았다. 1 2 3 4 5 6 7 12. 알고 있는 배경지식을 활용하 여 지문 내용을 추측하여 보았 다. 1 2 3 4 5 6 7 13. 오답인 선택지를 하나씩 제거 하며 답을 확인하여 보았다. 1 2 3 4 5 6 7 14. 지문에 있는 단어를 포함한 선 택지를 답으로 선택하 다. 1 2 3 4 5 6 7 15. 선택지의 문장을 빈칸에 넣고 지문을 다시 읽어 보았다. 1 2 3 4 5 6 7 ▣ Item 2 (“reordering the paragraphs”) 2. What is the best order of (A), (B), and (C) after the sentence in the box? Imagine a pearl diver pushes off his canoe from the shore, paddles out into the sea, dives deep into the water, picks an oyster off the bottom, paddles to shore, and opens the shell. (A) You’re right. Pearls are rare; a diver must open many oysters before finding one. Only a very foolish diver would waste time making a separate trip for each oyster. It is the same with producing ideas. (B) Foolish people think of a single solution to a problem and proceed as if that solution had to be creative. But creative ideas, like pearls, occur infrequently. So sensible Book Centre교보문고 KYOBO 88 Sung Hye Kim · Yuah V. Chon people produce many ideas before expecting to find a creative one. (C) Finding nothing but an oyster inside, he sets off in his canoe again. “Wait a minute,” you’re probably thinking. “He’s wasting a lot of time. The right way to do it is not to paddle back to shore with one oyster but to dive again and again, fill the canoe with oysters, and return to shore.” *paddle: 노를 젓다 ① (B)-(A)-(C) ② (B)-(C)-(A) ③ (C)-(A)-(B) ④ (C)-(B)-(A) ※ 2 번 문항 풀이에 적용한 읽기 전략의 사용 정도를 표시하여 주세요. 읽기 전략 전혀 아니다 대체로 아니다 약간 아니다 보통 이다 약간 그렇다 대체로 그렇다 매우 그렇다 1~14. (Same as the above questions for No. 1) 1 2 3 4 5 6 7 15. 자신이 선택한 순서대로 지 문을 다시 읽어 보았다. 1 2 3 4 5 6 7 ▣ Item 3 (“finding information not true of the passage”) Read the passage and answer the question. Do you think hearing a “no, thank you” at the end of your interview process is the end? It’s not. If you really want to work for this company, a “no” is just the beginning. Amy had an interview with the company she’d always wanted to work for, but was crushed when she heard she was out of the running. She decided, however, to use the connections she’d made during this process as potential networking opportunities. She sent a follow-up note to the hiring manager thanking and emphasizing she’d love to be considered for any opportunities that might be a good fit. She continued to send notes to this hiring manager to check in and stay “top-of-mind.” Amy had also made a connection with one of the people who had interviewed her; they had attended the same university. She sent the person an e-mail saying she truly enjoyed connecting with someone from her university and would love to get together for lunch someday. Amy ended up taking the person to lunch a few months later. They were able to make a successful connection and stayed in contact. It was a year later that her lunch-date contact saw an opening that matched her Book Centre교보문고 KYOBO Test-taking Strategies of L2 Adolescent Learners: Three Multiple-choice Items and L2 Proficiency 89 qualifications and forwarded the information to her. Amy asked the hiring manager she’d been keeping in touch with to put in a good word. Amy was interviewed and was finally in her dream job. Many times the first round of interviewing for the job of your dreams is just that, a first round. You don’t need to take a “no” as the final answer. 3. Which one is NOT true about Amy? ① She continued to stay in contact with the hiring manager. ② She had attended the same university as one of the interviewers. ③ She was informed about an opening by her lunch-date contact. ④ She was hired by the company without any interview process. ※ 3 번 문항 풀이에 적용한 읽기 전략의 사용 정도를 표시하여 주세요. 읽기 전략 전혀 아니다 대체로 아니다 약간 아니다 보통 이다 약간 그렇다 대체로 그렇다 매우 그렇다 1~14. (Same as the above questions for No. 1) 1 2 3 4 5 6 7 15. 각 선택지에 해당하는 내용 을 지문에서 찾아 확인하여 보았다. 1 2 3 4 5 6 7 Applicable levels: Secondary Sung Hye Kim Division of College Scholastic Ability Test Korea Institute for Curriculum and Evaluation Jeongdong Bldg., 15-5 Jeong-dong, Jung-gu Seoul 100-784, Korea Phone: 02-3704-3817 Email: shkim@kice.re.kr Yuah V. Chon Department of English Education College of Education, Hanyang University 17 Haengdang-dong, Seongdong-gu Book Centre교보문고 KYOBO 90 Sung Hye Kim · Yuah V. Chon Seoul 133-791, Korea Phone: 02-2220-1144 Email: vylee52@hanyang.ac.kr Received in December 2013 Reviewed in January 2014 Revised version received in February 2014 Book Centre교보문고 KYOBO Abstract 1. INTRODUCTION 2. BACKGROUND 3. METHODS 4. RESULTS AND DISCUSSION 5. CONCLUSION REFERENCES << /ASCII85EncodePages false /AllowTransparency false /AutoPositionEPSFiles true /AutoRotatePages /None /Binding /Left /CalGrayProfile (Dot Gain 20%) /CalRGBProfile (sRGB IEC61966-2.1) /CalCMYKProfile (U.S. Web Coated \050SWOP\051 v2) /sRGBProfile (sRGB IEC61966-2.1) /CannotEmbedFontPolicy /OK /CompatibilityLevel 1.4 /CompressObjects /Tags /CompressPages true /ConvertImagesToIndexed true /PassThroughJPEGImages true /CreateJobTicket false /DefaultRenderingIntent /Default /DetectBlends true /DetectCurves 0.0000 /ColorConversionStrategy /LeaveColorUnchanged /DoThumbnails false /EmbedAllFonts true /EmbedOpenType false /ParseICCProfilesInComments true /EmbedJobOptions true /DSCReportingLevel 0 /EmitDSCWarnings false /EndPage -1 /ImageMemory 1048576 /LockDistillerParams false /MaxSubsetPct 100 /Optimize true /OPM 1 /ParseDSCComments true /ParseDSCCommentsForDocInfo true /PreserveCopyPage true /PreserveDICMYKValues true /PreserveEPSInfo true /PreserveFlatness true /PreserveHalftoneInfo false /PreserveOPIComments true /PreserveOverprintSettings true /StartPage 1 /SubsetFonts true /TransferFunctionInfo /Apply /UCRandBGInfo /Preserve /UsePrologue false /ColorSettingsFile () /AlwaysEmbed [ true ] /NeverEmbed [ true ] /AntiAliasColorImages false /CropColorImages true /ColorImageMinResolution 300 /ColorImageMinResolutionPolicy /OK /DownsampleColorImages true /ColorImageDownsampleType /Bicubic /ColorImageResolution 300 /ColorImageDepth -1 /ColorImageMinDownsampleDepth 1 /ColorImageDownsampleThreshold 1.50000 /EncodeColorImages true /ColorImageFilter /DCTEncode /AutoFilterColorImages true /ColorImageAutoFilterStrategy /JPEG /ColorACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /ColorImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000ColorACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000ColorImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 300 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /GrayImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000GrayACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000GrayImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict << /K -1 >> /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile (None) /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False /CreateJDFFile false /Description << /ARA /BGR /CHS /CHT /CZE /DAN /DEU /ESP /ETI /FRA /GRE /HEB /HRV (Za stvaranje Adobe PDF dokumenata najpogodnijih za visokokvalitetni ispis prije tiskanja koristite ove postavke. Stvoreni PDF dokumenti mogu se otvoriti Acrobat i Adobe Reader 5.0 i kasnijim verzijama.) /HUN /ITA /JPN /LTH /LVI /NLD (Gebruik deze instellingen om Adobe PDF-documenten te maken die zijn geoptimaliseerd voor prepress-afdrukken van hoge kwaliteit. De gemaakte PDF-documenten kunnen worden geopend met Acrobat en Adobe Reader 5.0 en hoger.) /NOR /POL /PTB /RUM /RUS /SKY /SLV /SUO /SVE /TUR /UKR /ENU (Use these settings to create Adobe PDF documents best suited for high-quality prepress printing. Created PDF documents can be opened with Acrobat and Adobe Reader 5.0 and later.) /KOR >> /Namespace [ (Adobe) (Common) (1.0) ] /OtherNamespaces [ << /AsReaderSpreads false /CropImagesToFrames true /ErrorControl /WarnAndContinue /FlattenerIgnoreSpreadOverrides false /IncludeGuidesGrids false /IncludeNonPrinting false /IncludeSlug false /Namespace [ (Adobe) (InDesign) (4.0) ] /OmitPlacedBitmaps false /OmitPlacedEPS false /OmitPlacedPDF false /SimulateOverprint /Legacy >> << /AddBleedMarks false /AddColorBars false /AddCropMarks false /AddPageInfo false /AddRegMarks false /ConvertColors /ConvertToCMYK /DestinationProfileName () /DestinationProfileSelector /DocumentCMYK /Downsample16BitImages true /FlattenerPreset << /PresetSelector /MediumResolution >> /FormElements false /GenerateStructure false /IncludeBookmarks false /IncludeHyperlinks false /IncludeInteractive false /IncludeLayers false /IncludeProfiles false /MultimediaHandling /UseObjectSettings /Namespace [ (Adobe) (CreativeSuite) (2.0) ] /PDFXOutputIntentProfileSelector /DocumentCMYK /PreserveEditing true /UntaggedCMYKHandling /LeaveUntagged /UntaggedRGBHandling /UseDocumentProfile /UseDocumentBleed false >> ] >> setdistillerparams << /HWResolution [2400 2400] /PageSize [612.000 792.000] >> setpagedevice