Abstract

Objective – To identify how millennial generation students proceed through the information search process and select resources on the web; to determine whether students evaluate the quality of web resources and how they use general information websites.

Design – Longitudinal study.

Setting – University in the United States.

Subjects – 80 undergraduate students of the millennial generation enrolled in a business course.

Methods – The students were required to complete a research report with a bibliography in five weeks. They also had to turn in interim assignments during that period (including an abstract, an outline, and rough draft). Their search behaviour was monitored using a modified Yahoo search engine that allowed subjects to search, and then to fill out surveys integrated directly below their search results. The students were asked to indicate the relevance of the resources they found on the open web, to identify the criteria they used to evaluate relevance, and to specify the stage they were at in the search process. They could choose from five stages defined by the author, based on Wilson (1999): initiation, exploration, differentiation, extracting, and verifying. Data were collected using anonymous user IDs and included URLs for sources selected along with subject answers until completion of all assignments. The students provided 758 distinct web page evaluations.

Main Results – Students did not progress in orderly fashion through the search process, but rather proceeded erratically. A substantial number reported being in fewer than four of the five search stages. Only a small percentage ever declared being in the final stage of verifying previously gathered information, and during preparation of the final report a majority still declared being in the extracting stage. In fact, participants selected documents (extracting stage) throughout the process. In addition, students were not much concerned with the quality, validity, or authority of their sources, reporting that the main criteria they used to evaluate a web resource were its understandability, the amount of information in the source, its accuracy, and its recency. During the last stage of the assignment the main criteria were understandability and the amount of information. Finally, students used general information websites like Wikipedia throughout the process, but especially while preparing the final report.

Conclusion – The search behaviour of millennial students does not conform to existing search models. The models are appropriate but the execution of these models by students is problematic. Students gathered documents, including general websites like Wikipedia, through all stages of the assignment, including the preparation of the final report. They are likely to procrastinate and do some backfilling. Furthermore they show little concern for the validity of sources: very few verified their sources and quality of the information gathered was not a priority for them. Those findings point to a problem of perception rather than a lack of information search skills: millennial students know how to search and filter, but they do not believe that there is an objective standard to evaluate information and they have a non-critical view of information. More research about the causes of such perception should help us identify effective strategies to help students improve their searches.

Commentary

This study builds on existing information search process models which were developed before the internet became a popular information source. There is relatively little research assessing how such models may have changed since (Knight & Spink, 2008). The study also adds to growing research about the information seeking habits of the millennial generation, which has shown that millennials have superficial search habits, a fragmented view of information, and a conception of information as product rather than process. A third research area this study explores is that of relevance criteria used by searchers to select sources that meet their information need. The study brings these aspects together into an investigation of students’ information search behaviour over time and concludes that in all three areas the behaviour and perceptions of millennials differ from those of previous generations. While not groundbreaking, it provides some insight into ways millennials themselves experience and conceptualize searching.

The study’s greatest contribution springs from its methodology: it used a self-reporting online tool to monitor students completing a real assignment in their own space and at their own pace, away from the researchers’ presence. Yet subjects were not in a completely natural setting: they used a modified search engine that included data collection instruments. The author used Yahoo, which may not be students’ usual search engine. Although subjects were not required to use it, the vast majority stayed with the default because it was easier to enter the data. Furthermore the data collection method is susceptible to all the limitations of self-reporting. Reporting was a complex process that involved numerous criteria to choose from as well as specialized terminology, two issues that may have negatively affected student reporting. The author does not mention if the validity of the instrument was tested by external observation. It is regrettable that the instrument is not included in the report in its entirety, as it has no precedents (CRiSTAL Checklist, n. d.).

Other research has confirmed the study’s results about students’ tendency to procrastinate and engage in backfilling (Head & Eisenberg, 2009). Particularly striking is the students’ definition of the search process primarily in terms of extraction of information, which points to a confused and limited conception of searching. However, the study’s conclusion that pre-internet search models are appropriate in the digital age is not warranted. Although the author uses these models for his research instrument, he does not demonstrate how they are relevant for the study. More research is needed on the impact of unmediated web searching on search behaviors, including other models like berrypicking (Bates, 1989; Knight & Spink, 2008).

For the author, the key factor accounting for the students’ search behaviour is millennials’ relativistic perception of information, but he gives no evidence for this and does not explore the reasons for the students’ apparent neglect of evaluation. Research comparing perceived and actual search behaviours could yield more useful results. For example, Project Information Literacy found that students did not think of resource evaluation as a distinct step in the search process but in effect did evaluate sources (Head & Eisenberg, 2010, p.18). It has also found that students’ search behaviours are not just determined by a subjective conception of information but by uncertainty and stress about the entire research process, which leads students to rely on predictable and convenient but limited routines (Head & Eisenberg, 2009, 2010). Consequently, this study did not lead to innovative conclusions but like other similar studies could have practical implications for librarians who seek to help students construct more effective searches, and to help faculty design more successful assignments.

References

Bates, M. J. (1989). The design of browsing and berrypicking techniques for the online search interface. Online Information Review, 13(5) 407-424. doi: 10.1108/eb024320

CRiSTAL Checklist on Appraising a User Study. In Netting the Evidence Wiki. Retrieved 11 Feb. 2013 from http://nettingtheevidence.pbwiki.com/f/use.doc

Head, A. J., & Eisenberg, M. B. (2009). Lessons learned: How college students seek information in the digital age. Project Information Literacy Progress Report. Retrieved 11 Feb. 2013 from http://projectinfolit.org/pdfs/PIL_Fall2009_finalv_YR1_12_2009v2.pdf

Head, A. J., & Eisenberg, M. B. (2010). Truth be told: How college students evaluate and use information in the digital age. Project Information Literacy Progress Report. Retrieved 11 Feb. 2013 from http://projectinfolit.org/pdfs/PIL_Fall2010_Survey_FullReport1.pdf

Knight, S. A., & Spink, A. H. (2008). Toward a web search information behavior model. In A. Spink & M. Zimmer (Eds.), Web search: Multidisciplinary perspectives. (pp. 209-234). Berlin: Springer.

Wilson, T.D. (1999). Models in information behaviour research. Journal of Documentation, 55(3) 249-270. doi: 10.1108/EUM0000000007145