Science and News: A Study of Students’ Judgments of Online Scientific News Information

Tara Tobin Cataldo
Associate University Librarian
University of Florida
Marston Science Library
ttobin@ufl.edu

Kailey Langer
Research Assistant
University of Florida
Marston Science Library
kaileylanger@ufl.edu

Amy G. Buhler
University Librarian
University of Florida
Marston Science Library
abuhler@ufl.edu

Samuel R. Putnam
Assistant University Librarian
University of Florida
Marston Science Library
srputnam@ufl.edu

Rachael Elrod
Assistant University Librarian
University of Florida
Education Library
relrod@ufl.edu

Ixchel M. Faniel, PhD
Senior Research Scientist
OCLC Research
fanieli@oclc.org

Lynn Silipigni Connaway, PhD
Director of Library Trends and User Research
OCLC Research
connawal@oclc.org

Christopher Cyr, PhD
Associate Research Scientist
OCLC Research
cyrc@oclc.org

Brittany Brannon
Research Assistant
OCLC Research
brannonb@oclc.org

Joyce Kasman Valenza, PhD
Assistant Teaching Professor
Rutgers University
joyce.valenza@rutgers.edu

Erin M. Hood
Research Support Specialist
OCLC Research
hoode@oclc.org

Randy A. Graff, PhD
Director of Educational Technologies
University of Florida
rgraff@ufl.edu

Abstract

Introduction

What happens when a student researcher evaluates science news resources on a Google results page? Current focus on the integrity of news, as well as concerns about the credibility of scientific information, point to a critical need to understand how students evaluate their online search results (Bucchi 2017; Horrigan 2017; Lazer et al. 2018). Researching Students’ Information Choices: Determining Identity and Judging Credibility in Digital Spaces (RSIC) is a four-year Institute of Museum and Library Services (IMLS) funded research study examining the behaviors of students from late primary school through graduate school as they select resources for a science-related school project. The study’s primary research objective is to examine the different processes involved in determining the credibility of online resources, particularly the role of the container (resource type). This innovative methodology engaged students in task-based simulations to capture real-time resource selection behavior and enabled comparison between study participants within and across educational stages. This paper focuses on 116 students’ judgments of three science news resources. These judgments focused on whether students found the resources helpful, citable, and credible, and if they identified the resource container as news.

Literature Review

It is crucial to examine the perceived integrity of news resources. The declining public perception of science has become a hot topic as Americans’ mixed views towards science have ignited debate among scientists, media, and the public (Funk 2017). Effective science communication is important for ensuring accurate dissemination of scientific discoveries to the public through the media (Allan 2009; Gupta et al. 2013; Hansen 2016). Siegfried (2010) warned that statistical issues and early reporting of research can misinform the public and negatively affect the public’s understanding of science. Zhang (2018) described online science news as shifting away from journalistic institutions that traditionally report and criticize, and toward academic institutions that are often less critical. These issues correlate with public skepticism, since only 28% of individuals who get science news from general sources (described as those sources that cover a range of topics each day) trust those sources to get the facts right (Funk et al. 2017). In an era of misinformation, disinformation, and fake news, a multidisciplinary effort is required to empower individuals to evaluate resources and enact structural changes to systems which disseminate fake news (Lazer et al. 2018). Given the research above, and the need to equip students with evaluation strategies, it is important to first identify how information seekers use cues and content to make assessments about news resources.

This study is partially built on the credibility evaluation research of Andrew Flanagin and Miriam Metzger (Flanagin & Metzger 2000, 2007, 2008, 2010; Metzger et al. 2003, 2010, 2015; Metzger & Flanagin 2013, 2015). Since the late 1990’s, Flanagin and Metzger studied both children’s and adults’ perceptions of online information. They successfully identified heuristics, including reputation, endorsement, consistency, expectancy violation, and persuasive intent, and cues used in the credibility assessment of online information. The container the information is packaged in (e.g., book, journal, news) may represent an additional cue used in conjunction with these previously identified heuristics.

The potential role the container plays in the evaluation of online information is a newly emerging research topic, though it was noted as far back as 2007 that the behavior of today’s “format agnostic” students should be investigated (Williams & Rowlands 2007). In a preliminary study using a survey and screen captures of various online resources, Buhler and Cataldo (2016) demonstrated that university students did have difficulty in identifying the container. Leeder (2016) also determined that students are often inaccurate in determining online genre and had difficulty distinguishing between scholarly sources and non-scholarly online sources. The project researchers believe container collapse, a nuanced obstacle in the evaluation of online resources, is contributing to the “format agnostic” phenomenon. Container collapse is the flattening of information sources from the print containers that once provided context and cues to help identify a document’s origins. In digital format, the information is decanted from its original container and must be carefully examined for publishing indicators to determine the journey it took to reach the individual user. Cues such as domain and source recognition do not help students identify the container and therefore they are missing some steps in the process that information took to reach them. A deeper dive into this phenomenon will enable researchers to understand what growing up with online information means to the use of print containers in the digital world.

Methods

The data analyzed for this paper are part of the RSIC study that comprised two phases (Buhler et. al. 2015). In the first phase a prescreen survey was administered to students in a county in North Central Florida. The county is home to a large research university, a large community college, and a midsize K-12 school system. Survey results were used to recruit students for the second phase, in-person simulation sessions. The simulation participants were divided by educational stage into six cohorts: elementary school, middle school, high school, community college, university undergraduate, and university graduate.

Teams were formed and tasked with creating an age-appropriate research prompt for a science-related inquiry project, the Burmese Python in the Florida Everglades (Table 1). The teams were comprised of the project researchers and members of the project’s Advisory Panel. The Advisory Panel included two university librarians, one school librarian, one community college librarian, and three science instructors (one each from the university, college, and high school).

These teams also identified resources that should appear in the simulations. The higher-education teams ended up with enough overlap among their chosen resources that one simulation could serve all the higher-education students. The resources chosen for the K-12 simulations were disparate enough to require separate simulations for elementary school, middle school, and high school. Several factors went into decisions on the order the resources would appear in the simulation. One was the order they would potentially appear in a live online search and another factor was making sure the same container types were not clustered on one page.

Cohort	Research Prompt
Elementary School	You have an assignment to write a science report that investigates the Burmese Python in the Everglades and describes the ways that this animal is affecting the Everglades Habitat.
Middle School	You are assigned a report on the following: Citing specific evidence, in what ways has the invasion of the Burmese Python is impacting the health of the Florida Everglades’ ecosystem?
High School	You are asked to create a public service message based on solid evidence, addressing the following: How are pythons impacting the biodiversity of the Everglades ecosystem?
Community College	You are beginning a literature search for your General Biology (BSC 2005) final paper. You’ve decided to focus on the impact of the Burmese python (Python molurus bivittatus) to the biodiversity of the Florida Everglades.
University Undergraduate	You are beginning a literature search for your Wildlife Issues final paper. You’ve decided to focus on the impact of the Burmese python (Python molurus bivittatus) to the biodiversity of the Florida Everglades.
University Graduate	You are beginning a literature search for your thesis on the impact of the Burmese python (Python molurus bivittatus) to the biodiversity of the Florida Everglades.

Each student was given their age-appropriate research prompt, and asked to conduct a search for information. The searches were conducted in a simulated Google environment created with the Articulate Storyline software. A task-based methodology and a think-aloud protocol were employed to observe real-time cognition in action. Participants in each cohort were presented with a controlled set of search results and asked to determine the helpfulness, citability, credibility, and container of the resources. The first task was the Helpful Task, where students selected a prescribed number of resources they found helpful for the inquiry. Then they were asked if each helpful resource was citable and chose yes or no for their response in the Cite Task. They also determined the credibility of their helpful resources via a 5-point Likert scale ranging from 1 (not credible) to 5 (highly credible) in the Credible Task. Credibility is known to be a more nuanced judgment, requiring a more diverse scale of measurement, such as a Likert. Finally, the participants were presented with a predetermined set of the original resources (which they may or may not have selected as helpful) and asked to label them with one of eight possible containers in the Container Task. A brief demonstration of the simulation can be viewed at http://ufdc.ufl.edu/IR00010570/00001.

Data Collection and Analysis

Data for this paper come from the high school, community college, undergraduate, and graduate cohorts (n=116). While a variety of demographic data was collected on the participants, this analysis focuses on the educational stages. These four groups had three of the same news resources in their simulations, allowing for comparison. The analysis used a subset of the quantitative data that the Articulate Storyline software captured during the simulation sessions. This subset included students’ judgments from the Helpful Task, Cite Task, and Credible Task. Two of the three news resources were among the preselected resources in the Container Task and were analyzed to determine if the students identified them as news. The three science news resources were from three different organizations. The first resource was a news story from the U.S. Geological Survey (USGS) monthly newsletter Sound Waves, describing a USGS co-authored study (Figure 1). The second resource was a transcribed interview from National Public Radio (NPR) in which a university professor discusses the Burmese Python in the Everglades (Figure 2). The third resource was a short documentary video accompanying a New York Times (NYT) article (Figure 3). For comparison purposes, data on each resource’s citability, credibility, and container were obtained from members of the project’s Advisory Panel.

The number of students in each cohort who selected the three resources as helpful was identified. For the students who selected these resources as helpful, the number who also selected them as citable was identified. Chi-square tests were performed to assess the associations between cohort and helpfulness selections, and between cohort and citability decisions. The mean credibility ratings within each cohort were calculated. Additionally, the mean credibility ratings across all resources selected as helpful for each cohort were calculated. A one-way ANOVA test was performed to assess the association between cohort and credibility ratings. The percentages of students who correctly labelled the NYT and Sound Waves resources as news were calculated. Due to consideration of participants’ time and cognitive load during the simulation, only select resources were included in the container task build, and the NPR resource was not selected. A portion of the Advisory Panel data was incorporated as an additional point of comparison.

Results

Description of the Sample

Of the 116 students, 26 were in the high school cohort and 30 each in the community college, undergraduate and graduate cohorts. Fifty-three percent were female, 44% male, 2% genderqueer/transgender, and 1% preferred not to answer. Fifty percent of the students were White/Caucasian, 20% Asian, 12% Latino/Hispanic, 9% Black/African American, 6% Mixed Race, and 1% Native American. The remaining 2% preferred not to answer. Except for two high-school students, all had access to libraries, librarians, and library websites at school. Forty-one percent of the participants had asked a librarian for help with a research project and 43% of them are current or potential first-generation college students. Twenty percent were five years old or younger when they first accessed the Internet, 53% were between the ages of 6-10, 22% between 11-15, and 3% between 16-20, and the remaining provided no response. Thirty-five percent of the participants indicated they used social media for school work.

Helpful Task

For all four cohorts, the Sound Waves resource was selected as helpful more often than the NPR or NYT resources (Table 2). The NYT resource was selected the least. This trend was consistent in all cohorts except graduate, where selections of NYT equaled those of NPR. In general, community college and undergraduate students selected the three news resources as helpful more often than high-school and graduate students, and high-school students were the least likely to select the three news resources as helpful. For the Sound Waves resource, there was no evidence of a significant association between cohort and helpfulness selections (χ² (3, N= 116) = 2.81, p= .422). Statistically significant associations were found between cohort and helpfulness selections for the NYT (χ² (3, N= 116) = 11.81, p<.01) and NPR (χ² (3, N= 116) = 13.45, p< .01) resources.

	High School (n=26)	Community College (n=30)	Undergraduate (n=30)	Graduate (n=30)	Total (n=116)
Sound Waves	73%	83%	90%	80%	82%
NPR	38%	80%	76%	56%	64%
NYT	23%	66%	56%	56%	52%

Cite Task

All four cohorts perceived the Sound Waves resource as the most citable resource relative to its helpfulness and the NPR resource the least, except for the undergraduates who found the NYT resource the least citable, but only by a 3% difference. High-school students were most likely to consider a helpful news resource citable (100% for Sound Waves, 83% for NYT, and 80% for NPR) (Figures 4-6). Graduate students, on the other hand, were least likely to consider a news resource citable after selecting it as helpful (58% for Sound Waves, 53% for NYT, and 41% for NPR). For Sound Waves, there was a statistically significant association between cohort and citability decisions (χ² (3, N= 95) = 11.32, p= .01). Findings did not support the presence of a significant association between cohort and citability decisions for the NYT (χ² (3, N= 60) = 2.41, p= .492) and NPR (χ² (3, N= 74) = 6.04, p= .110) resources. Of the seven Advisory Panel members, four thought Sound Waves was a citable resource. Three found the NPR and NYT resources citable.

Credible Task

Students rated the credibility of their helpful resources on a scale of 1 (not credible) to 5 (highly credible). The average credibility rating was calculated for each of the three news resources by cohort (Figure 7). Each cohort rated the Sound Waves resource as the most credible, or at least equal to the next most credible in the case of graduate students, with an average of 3.9 or higher. The NPR resource received the next highest credibility ratings from community college (3.9), undergraduate (4.0), and graduate (3.9) students. However, high school students rated the credibility of NYT (4.3) almost as highly as Sound Waves (4.5) and gave NPR (3.7) the lowest credibility rating of the three resources. The three higher-education cohorts rated the NYT resource below average at 3.6 or lower. Results of a one-way ANOVA test did not support a significant association between cohort and credibility ratings for Sound Waves (F(3,91)=1.93, p= .130), NYT (F(3,56)=.98, p=.410), or NPR (F(3,70)=.38, p=.771). The average credibility of the Advisory Panel members was a 4.0 for all three resources.

Container Task

The research team selected resources from the original search results to be a part of the Container Tasks. Sound Waves and NYT were the only two news resources in this task for all four cohorts. When presented with eight options (blog, book, magazine, news, conference proceeding, pre-print, website, and journal), students identified the NYT resource as a news resource more often than the Sound Waves resource. Figure 8 shows the percentage of students who labeled these two resources as news. Over 75% of high school, undergraduate, and graduate students and over 50% of community college students identified NYT as news. In contrast, 33% of graduate students labeled Sound Waves as news, and this was the highest percentage of all four cohorts. The generic label of website was selected most frequently as the container for the Sound Waves resource (39% high school, 40% college, 57% undergraduates, 50% graduates). For the students who did not label NYT as news, magazine was the next popular choice. All seven Advisory Panel members agreed that the NYT and NPR resources were news, but only five labeled the Sound Waves resource news. One of the remaining two members labeled Sound Waves as a website and the other as a magazine.

The combined cohorts were parsed into two groups, students who labeled the resource news and students who did not. Then the helpful, citable, and credibility analyses were repeated. Approximately three quarters of students labeled NYT as news, but those that did were less likely to choose NYT as helpful (49% versus 59%) and slightly less likely to deem it citable (66% versus 69%) compared to those that did not label NYT news. But the credibility average between those two groups is almost the same - 3.7 for the news choosers and 3.6 for the not news group. A very similar pattern is seen with Sound Waves. Those who labeled it news were less likely to find it helpful (75% versus 84%) or citable (71% versus 81%) compared to those that did not label Sound Waves news. However, the credibility is slightly higher for the news choosers - 4.4 as compared to 4.2.

Discussion

News items are important resources in the dissemination of scientific studies for public consumption. The three news resources that the students evaluated were decidedly different in their presentation and were hosted on three different domains - .org, .com, and .gov. The students’ judgments need to be taken into account with regard to where and how these news resources appeared and the content they contained.

Helpful, Citable, and Credible Judgments by Resource

The resource from the newsletter Sound Waves was of moderate length and contained several quotes. It had a few small pictures, but the text was the dominant feature. It was found by all four cohorts to be the most helpful (82%) and citable (79%) of the three news resources. The Sound Waves resource was also considered the most credible of the three news resources with an overall mean credibility rating of 4.3 out of 5. One potential influence on these positive judgments could be that its URL contained a .gov domain. As one high school student stated, “Dot gov, I always trust dot gov.” This suggests that government resources benefit from the commonly taught heuristic that websites from .gov domains are citable (Treise et al. 2003; Klawitter & Hargittai 2018), colloquially referred to as the “.gov bump.” In its first paragraph, the article links to a study published in the Proceedings of the National Academy of Sciences. Many students might look favorably on a resource that led them to original research. One graduate student stated, “Sound Waves. Oh, it's a monthly newsletter. Okay. And it links to a study that was published in PNAS, so this one's good. I want this one.” The students’ overall credibility rating of 4.3 was also higher than the Advisory Panel whose average was a 4.0. In general, the Advisory Panel found the resource credible but identified issues such as missing author credentials and the newsletter format, which implicates the role of the container.

The NPR resource fell in the middle for all four cohorts in terms of helpfulness (64%) but went to the bottom in terms of citability (62%). It also fell in the middle for credibility rating with an overall mean of 3.9 out of 5. The resource was a transcript of an interview from the “Talk of the Nation” radio show, and the interviewee was a university professor in the field of wildlife ecology. An audio Play symbol was a prominent feature in the top left corner and there were no images in the article. While an interview is a primary source and could be considered valuable, perhaps students were deterred by the podcast genre. One community college student stated, “I feel like NPR is great, and I love NPR, but I don't know. I feel like I'd be-- I would want to put in something like a scholarly article rather than a link to a podcast.” The Advisory Panel recognized that this was a primary source with an expert in the field, but noted that it was for public consumption, contained no references, and featured callers sharing anecdotes.

The NYT can be considered a widely recognized source by the general public. However, the NYT resource was found helpful by only 52% of the students, and of these students, 67% considered it citable. On average students considered the NYT resource the least credible of the three resources with an overall mean credibility rating of 3.8 out of 5. A few factors may have played a role in these results. The resource led with a video, and many students steered away from videos for a school assignment. As one graduate student explained, “that was a video. I just, in general, don't like citing videos at all...They're usually published more for entertainment value. And then you have to sit through them instead of skimming, and I just don't bother with it.” Its title, The Snake That’s Eating Florida, may be too sensational or as one undergraduate student called it, “click baity.” The writing style was not appealing to some students. Another graduate student stated, “The New York Times...I don't really like how it starts...about Eve and the Garden of Eden and the Burmese python. It's probably still reputable, but I really wouldn't use it as a reference.” The Advisory Panel were in agreement that the resource itself is reputable and more accessible, but, like the Sound Waves resource, it is less desirable for use in a school project.

In sum, of the three news resources, two were from well-known news sources and one was from a lesser known government source. In addition to the .gov bump, Sound Waves was comparatively professional in its writing style and was the only resource of the three that linked to a scholarly journal article. NPR was likely more recognizable than Sound Waves, but the transcript of a radio broadcast may not have been as appealing. The NYT was likely the most recognizable source of the three, but the embedded video and casual writing style may have dissuaded students from using it.

Comparison Between Educational Stages

As may be expected, the greatest difference in judgments were observed between the high-school students and those in higher education. Overall, high-school students were more inclined than the other educational stages to say a helpful resource was also citable, with their lowest citability percentage at 80% for NPR. One-hundred percent of high schoolers who found the Sound Waves resource helpful judged it citable. Fewer high-school students selected the NYT resource as more helpful than those in any other cohort. However, all but one high schooler who selected the NYT resource as helpful considered it citable, a much higher citability percentage than observed in the other cohorts. Chi-square tests revealed a significant association between cohort and the choice of the NYT resource as helpful but did not support a significant association between cohort and citability decisions. Analysis for NPR showed the same pattern of significance as for NYT. Citability dropped more dramatically among the higher education groups with the community college students’ lowest at 58% (NPR), undergraduates at 71% (NYT) and the graduate students at 41% (NPR). This trend across the three resources suggests that high-school participants utilized a different judgment process than higher-education students or perhaps their criteria for a citable source differs. This judgment process needs further exploration. It could be influenced by prior assignments and expectations (Head 2013), or, perhaps, their criteria for a citable source differs for other reasons. It is beyond the scope of this study, but the judgment process is being explored in a future analysis. With respect to the acceptability of the NYT as a source, it is worth noting that both the high-school teacher and school librarian on the Advisory Panel also deemed it citable, suggesting this is an acceptable source in secondary school. They pointed out the resource’s evidence of fact checking and easy-to-understand language as credentials. High-school students also considered the NYT resource more credible than the other cohorts and deviated from the trend observed wherein credibility ratings are highest for the Sound Waves resource, then the NPR resource, and lowest for the NYT resource. This is an interesting break in the trend, though the ANOVA test did not find a significant association between cohort and the credibility ratings for any of these resources.

Based on previous studies in which search engine page ranking is shown to influence the selection and credibility decisions around the results, there was some expectation that Google page ranking would play a part in the resources students selected as helpful (Westerwick 2013; Haas & Unkel 2017; Unkel & Haas 2017). This may be the case with the high-school students, but not with the higher-education students. The Sound Waves resource was on the third page of results for the higher-education students (out of 4 total pages of 10 resources each) and the second page for high schoolers. It received the highest scores across the helpful, citable, and credible tasks within all of the cohorts, outpacing the NPR resource (page one for higher education and page two for high school) and the NYT resource (page one for higher education and three for high school). This suggests that the higher-education students were not heavily relying on this heuristic. The NYT resource was also on the third page of search results in the high-school simulation, and this may partially explain the relatively low number of helpful selections observed in the high-school cohort.

The current findings show student judgments of science news resources differ across education stages. From high school to graduate school, the gap between helpful selections and citable decisions widened. These observed trends may reflect the development of information evaluation techniques as students move through educational stages. The most notable differences between high school and the other cohorts, specifically graduate, related to judgments of the NYT resource.

The Role of the Container

In the Container Task, Sound Waves, the most helpful, citable, and credible resource, was not commonly identified as news. Only 24% of the 116 students labeled it news. In contrast, the more recognizable resource, NYT, was the least helpful, citable, and credible. The NYT resource was identified as news by a majority of the students (77%). Comments such as “Not all mainstream media always writes true papers” from a graduate student, prompted further exploration of the students’ container judgments. When the students were parsed into the two groups of those who identified the resources as news versus those that did not, there is a distinctive difference. Those who labeled the Sound Waves and NYT resources as news were less likely to pick the resource as helpful or citable. It did not have a strong association with the credibility, though. This suggests that the news label could have a negative connotation to the helpfulness and citability of the resource, but not the judgment of the source’s credibility. In the simulation, students were not required to make the container choice until the last task, but many were still using the heuristic in the other tasks. This should be explored further with all associated variables considered.

The majority of students labeled the Sound Waves resource as a “website.” This is not wrong, of course. All the resources in this study could technically be described as websites. Only five of the seven members of the Advisory Panel labeled this resource with the news container. A “newsletter” might not be considered “news,” or users may not have attended to other cues beyond the .gov domain. Sound Waves is a government newsletter (so it is news by function, not by the type of organization), whereas the NYT resource is news both by function and organization. This ambiguity would likely not be present if students were handling the physical form of the newsletter, but this becomes more challenging online.

Conclusion

This paper explored 116 students’ judgments of three online science news resources. Based on their determinations of helpfulness, citability, credibility, and container, we can make several observations. While students may find news resources helpful when starting research for science-related school projects, they often do not consider them worthy of including in their reference lists, especially in higher education. High-school students are more likely to find helpful news sources worthy of citing. Data suggests students rely on the .gov in a URL as a strong positive indicator of the quality, spotlighting how the domain cue (.com, .org, .gov, etc.) is still a popular heuristic in the evaluation process. However, not all students identified what we would consider a commonly recognized news resource as news.

This window into how students judge resources found through the open web can inform current and future information literacy instruction practices. Standards such as the Association for College and Research Libraries’ (ACRL) Framework (2015), especially the frames “Authority Is Constructed and Contextual” and “Information Creation as a Process,” are spaces where this knowledge can play an important part. Building students’ understanding that “authoritative content may be packaged formally or informally and may include sources of all media types” and that “the information creation process could result in a range of information formats and modes of delivery, so experts look beyond format when selecting resources to use” can foster deeper engagement with resources. The differences observed across the educational level call for the need to incorporate new research discoveries even sooner in a student’s literacy instruction. The knowledge of these evolving behaviors can inform practices that utilize the American Association of School Librarians Standards Framework for Learners (2018), particularly the learner standards "IV.A.3 Making critical choices about information" and "VI.A.3 Evaluating information for accuracy, validity, social and cultural context, and appropriateness for need." Given the pervasiveness of open-web use and as a means of imparting life-long skills, teaching the application of these standards should not be isolated to the use of curated information systems (i.e., databases), but extended to search engines as well.

Future analysis of this study’s quantitative results in conjunction with the qualitative and demographic data, will make necessary connections between the various judgments and the thought processes behind them. Future research into the evaluation of news resources should take the role of the container into the conversation along with variance in the access methods (search engine results pages vs. social media/mobile apps). As the wealth of online information grows, fewer resources will have print counterparts and fewer will be from widely recognizable sources, so it is important that students be taught to evaluate resources with these challenges in mind.

Acknowledgements

This project was made possible in part by the Institute of Museum and Library Services grant number LG-81-15-0155.

References

Allan, S. 2009. Making science newsworthy: exploring the conventions of science journalism. In: Hollman, R. et al., editors. Investigating Science Communication in the Information Age: Implications for Public Engagement and Popular Media. Oxford (UK): Oxford University Press. p. 149–156.

American Association of School Librarians. 2018. AASL standards framework for learners. [Internet]. Available from: https://standards.aasl.org/wp-content/uploads/2018/08/180206-AASL-framework-for-learners-2.pdf.

Association of College and Research Libraries. 2015. Framework for information literacy for higher education. [Internet]. Available from http://www.ala.org/acrl/standards/ilframework.

Bucchi, M. 2017. Credibility, expertise and the challenges of science communication 2.0. Public Understanding of Science 26(8):890-893. DOI: 10.1177/0963662517733368.

Buhler, A. & Cataldo, T. 2016. Identifying e-resources: an exploratory study of university students. Library Resources & Technical Services 60(1):23–37. DOI: 10.5860/lrts.60n1.23.

Buhler, A., Cataldo, T.T., Faniel, I.M., Connaway, L.S., Valenza, J.K., Graff, R., & Elrod, R. 2015. Researching students’ information choices: determining identity and judging credibility in digital spaces. [Internet]. IMLS Grant Project LG-81-15-0155. Available from http://ufdc.ufl.edu/AA00037208/00001/citation.

Flanagin, A.J. & Metzger, M.J. 2000. Perceptions of internet information credibility. Journalism & Mass Communication Quarterly 77(3):515–540. DOI: 10.1177/107769900007700304.

Flanagin, A.J. & Metzger, M.J. 2007. The role of site features, user attributes, and information verification behaviors on the perceived credibility of web-based information. New Media & Society 9(2):319–342. DOI: 10.1177%2F1461444807075015.

Flanagin, A.J. & Metzger, M.J. 2008. Digital media and youth: unparalleled opportunity and unprecedented responsibility. In: Metzger, M.J., Flanagin, A.J., editors. Digital Media, Youth, and Credibility. Cambridge (MA): The MIT Press. p. 5–27.

Flanagin, A.J. & Metzger, M.J. 2010. Kids and Credibility: An Empirical Examination of Youth, Digital Media Use, and Information Credibility. Cambridge (MA): The MIT Press.

Funk, C. 2017. Mixed messages about public trust in science. Issues in Science & Technology 34(1):86-88. Available from https://issues.org/real-numbers-mixed-messages-about-public-trust-in-science/.

Funk, C., Gottfried, J. & Mitchell, A. 2017. Science news and information today. [Internet]. Available from http://www.journalism.org/2017/09/20/science-news-and-information-today/.

Gupta, N., Hamilton, K. & Chamot, J. 2013. Conveying cutting-edge discoveries to nonscientists: effective communication with media. JOM: The Journal of the Minerals, Metals & Materials Society 65(7):835–839. DOI: 10.1007/s11837-013-0617-0.

Haas, A. & Unkel, J. 2017. Ranking versus reputation: perception and effects of search result credibility. Behaviour & Information Technology 36(12):1285–98. DOI: 10.1080/0144929X.2017.1381166.

Hansen, A. 2016. The changing uses of accuracy in science communication. Public Understanding of Science 25(7):760–774. DOI: 10.1177/0963662516636303.

Head, A.J. 2013. Learning the ropes: how freshmen conduct course research once they enter college. [Internet]. Project Information Literacy Passage Studies Research Report. Available from: https://www.projectinfolit.org/uploads/2/7/5/4/27541717/pil_2013_freshmenstudy_fullreportv2.pdf.

Horrigan, J.B. 2017. How people approach facts and information. [Internet]. Available from http://www.pewinternet.org/2017/09/11/how-people-approach-facts-and-information/.

Klawitter, E. & Hargittai, E. 2018. Shortcuts to well being? evaluating the credibility of online health information through multiple complementary heuristics. Journal of Broadcasting & Electronic Media. 62(2): 251–268. DOI: 10.1080/08838151.2018.1451863.

Lazer, D., Baum, M., Benkler, Y., Berinsky, A., Greenhill, K., Menczer, F., Metzger, M., Nyhan, B., Pennycook, G., Rothschild, D., et al. 2018. The science of fake news. Science 359(6380):1094–1096. DOI: 10.1126/science.aao2998.

Leeder, C. 2016. Student misidentification of online genres. Library & Information Science Research 38(2):125–132. DOI: 10.1016/j.lisr.2016.04.003.

Metzger, M.J. & Flanagin, A.J. 2013. Credibility and trust of information in online environments: the use of cognitive heuristics. Journal of Pragmatics 59, Part B:210–220. DOI: 10.1016/j.pragma.2013.07.012.

Metzger, M.J. & Flanagin, A.J. 2015. Psychological approaches to credibility assessment online. In: Sundar S.S., editor. The Handbook of the Psychology of Communication Technology. Oxford (UK): John Wiley & Sons. p. 445–466. DOI: 10.1002/9781118426456.ch20.

Metzger, M.J., Flanagin, A.J. & Medders, R.B. 2010. Social and heuristic approaches to credibility evaluation online. Journal of Communication 60(3):413–439. DOI: 10.1111/j.1460-2466.2010.01488.x.

Metzger, M.J., Flanagin, A.J. & Nekmat, E. 2015. Comparative optimism in online credibility evaluation among parents and children. Journal of Broadcasting & Electronic Media 59(3):509–529. DOI: 10.1080/08838151.2015.1054995.

Metzger, M.J., Flanagin, A.J. & Zwarun, L. 2003. College student web use, perceptions of information credibility, and verification behavior. Computers & Education 41(3):271–290. DOI: 10.1016/S0360-1315(03)00049-6.

Siegfried, T. 2010. Flaws of statistics afflict science news reporting. Science News 177(7):2. DOI: 10.1002/scin.5591770702.

Treise, D., Walsh-Childers, K., Weigold, M.F. & Friedman, M. 2003. Cultivating the science internet audience: impact of brand and domain on source credibility for science information. Science Communication 24(3): 309–332. DOI: 10.1177/1075547002250298.

Unkel, J. & Haas, A. 2017. The effects of credibility cues on the selection of search engine results. Journal of the Association for Information Science and Technology 68(8):1850–62. DOI: 10.1002/asi.23820.

Westerwick, A. 2013. Effects of sponsorship, web site design, and Google ranking on the credibility of online information. Journal of Computer-Mediated Communication 18(2): 80–97. DOI: 10.1111/jcc4.12006.

Williams, P. & Rowlands, I. 2007. Information Behaviour of the Researcher of the Future: The Literature on Young People and Their Information Behaviour. London (UK): British Library/JISC.

Zhang, Y. 2018. Retailing science: genre hybridization in online science news stories. Text & Talk 38(2):243–265. DOI: 10.1515/text-2017-0040.