key: cord-0803351-sahm6xr9 authors: Oberbauer, A M; Lai, E; Kinsey, N A; Famula, T R title: Enhancing student scientific literacy through participation in citizen science focused on companion animal behavior date: 2021-08-03 journal: Transl Anim Sci DOI: 10.1093/tas/txab131 sha: 6e2d268b9baeb5f1d3c6ac0c199ae9e70ac8a1f3 doc_id: 803351 cord_uid: sahm6xr9 The Covid-19 pandemic served as the impetus to implement activities designed to engage students in the remote instructional environment while simultaneously developing scientific literacy skills. In a high enrollment general education animal science course, numerous activities were designed to improve scientific literacy. These included specifically developed videos covering strategies for reading published science literature, the utilization of topically relevant scientific articles that captured student interest, and engaging students in a citizen science exercise on whether dogs align themselves to the Earth’s magnetic field during excretion behavior. Employing pre- and post- self-perception surveys coupled with tasking students to apply their scientific literacy skills in an assessment scenario demonstrated that students’ self-perception of their scientific literacy improved 30% (p < 0.05) with approximately 80% of students accurately applying their literacy skills. The citizen science study on excretory behavior was modeled on previously published findings thereby providing an opportunity to validate the published work which had indicated that dogs align their bodies in a North-South axis during excretion. The present study did not demonstrate preferential alignment to any geomagnetic orientation which emphasized to the students the need for scientific replication. Inclusion of simple activities that were relevant to students’ daily lives, and providing interpretive context for those activities, resulted in improved self-perceived scientific literacy. Scientific literacy encompasses not only basic knowledge of scientific concepts and how scientists do science, but also what can be done with this knowledge, critical thinking, and making informed decisions (Maienschein, 1998) . Developing scientific literacy in the undergraduate student population is a learning objective in many college courses and in fact, is often a component of the general education expectations of a university education (National Academies of Sciences and Medicine, 2016) . Although there is much agreement for the need of scientific literacy in the educational system and in the general public (Laugksch, 2000) , philosophically there are different interpretations of the concept (Sjöström and Eilks, 2018) . Furthermore, there are a multitude of ways suggested to achieve such skills (Hurd, 1998; Kampourakis, 2019; Fisher et al., 2009; Hand et al., 1999) often focusing on either the science component or the literacy component. Regardless of the approach, it has been speculated that efforts to achieve scientific literacy in the classroom that lack relevance to the student or connectivity to daily life, will fall short of the desired goals (Feinstein, 2011) . The Covid-19 pandemic necessitated alternative approaches to delivery of classroom instruction, including any content related to scientific literacy, and student engagement. In a general education course that fulfills topical breadth in science including competencies in scientific literacy, we used a multi-pronged approach to encourage student engagement. The course focuses on the biology of companion animals and with over half of US households owning a companion animal (Applebaum et al., 2020) , the overarching topic of the course is highly relevant to the students. Despite the remote instructional modality, we considered student engagement indispensable to achieve student learning goals. A recent study (Hart et al., 2013) provided the ideal vehicle to engage students through a topic that has direct connectivity to their daily lives while developing skills in scientific literacy. In that study, the A c c e p t e d M a n u s c r i p t authors reported that dogs preferentially orient themselves along the earth's North-South magnetic axis when urinating and defecating. That publication invited spirited conversation on the methods used and the findings reported. By asking the students to also collect excretion data on their dogs, we engaged students in a citizen science activity. As part of these exercises, the citizen science data collected was analyzed, thereby illustrating the value of replication. We posited that regardless of the student's prior level of scientific literacy and preparation, engaging students in the analysis of citizen science data would improve their scientific literacy. Other exercises to develop scientific literacy were also incorporated into the class as adjuncts to the citizen science. This paper presents the efficacy of those techniques determined through student self-reflection and an end of course objective assessment. The student cohort evaluated were the 330 undergraduate students who completed a general education science class entitled "Companion Animal Biology" during the 2021 winter quarter at the University of California, Davis. The course is a lower division general education science course that meets a portion of the campus general education requirements in science topical breadth, competencies in quantitative literacy, scientific literacy, and writing experience. A 5-and 7-point Likert scale survey (Likert, 1932) to assess students' self-perceived scientific literacy was administered via an internet survey mechanism at the beginning (pre) and once again at the conclusion (post) of the course (Additional files 1 and 2). Participation in the survey was voluntary and the project, reviewed by the University of California, Davis Institutional Review Board, was determined to be exempt. Students did not have access to their pre -responses when they completed the post-survey therefore students A c c e p t e d M a n u s c r i p t served as their own controls in this study (Kitchenham and Pfleeger, 2002) . For each survey, students were asked to characterize their familiarity with various forms of scientific materials and experimental approaches, define their comfort level at interpreting scientific publications, and to suggest explanations for why a research study may not be valid. An additional question at the end of course asked how they viewed the effectiveness of the class activities on influencing their scientific literacy. The survey data were examined for differences in preand post-survey responses to the questions (Boone and Boone, 2012) . Questions were considered individually and grouped when appropriate. As examples of grouping, there were questions related to "general science knowledge" on the concept of scientific literacy; questions focused on understanding of hypotheses and "research execution"; questions aimed at self-assessment of scientific "research interpretation"; and questions dealing with "research communication" comfort level. Data were analyzed with simple descriptive statistics, and responses are presented as means and standard deviations (McClurg et al., 2015) and with paired t-tests (α = 0.05) when appropriate with significance defined as p < 0.05. Using a sample size calculator with the most conservative loading values, 344 surveys would reflect a student body population of 20,000 in similar courses; the enrollment of 330 students, if they all participated, would be expected to be sufficient to yield significant effects. A blended learning approach was taken to develop scientific literacy skills. Students Other exercises were weekly readings of publications related to class topics followed by virtual classroom discussions led by teaching assistants. The discussions targeted comprehension of the scientific content of each publication. Papers for evaluation were chosen for having logical presentation and information easily accessible to a broad audience of diverse backgrounds. Most importantly, papers were selected for having content that would have high student interest while being complementary to lecture material. For example, when covering companion animal nutrition in lecture, many students express an interest in alternative or home prepared diets. A paper presenting research on the nutritional adequacy of home prepared diets based upon available recipes (Wilson et al., 2019) was assigned. Students read the papers in advance of discussion and during the online discussion, the content was reviewed and expanded. Students collaborated in small breakout groups with the teaching assistants providing feedback to the students as they completed worksheets for each scientific paper. The worksheets required the students to determine the actual research question(s) being addressed, define what motivated the research, understand how the authors collected and analyzed their data, and reflect upon the implications of the paper's findings. Each weekly worksheet allowed the students to utilize the principles detailed in the videos. Questions on the worksheets were posed in a sequence that reflected the order in which the authors presented information enabling the students to easily grasp the progression of thought; having the students read and reflect on scientific papers each week developed their comfort level in accessing information in the scientific literature. A c c e p t e d M a n u s c r i p t The citizen science portion was inspired by a published article that described dogs' sensitivity to magnetic fields and how that influences their excretory behavior (Hart et al., 2013) . Over a period of 3 weeks (February 11 to March 1, 2021), students were asked to collect data on the excretory behavior of dogs with respect to alignment to the earth's axis. Students used the free National Oceanic and Atmospheric Administration CrowdMag app compass to record magnetic direction and declination of the dog's spine while excreting with the magnetic direction lining up with the dog's head. Students uploaded that information to an online Google form along with sex of the dog, type of excretion (urination or defecation), if the dog was in a familiar location, the geographical locale (zip code or city/country), a unique dog identifier, and time of day. Data points were included in analyses only if there were a minimum of three records per dog per type of excretion to account for individual dog variability. The raw data (Fig 1) was then discussed in several lectures and all students were asked to consider what kinds of questions could be answered using the data collected by their classmates. Research questions posed by the students were addressed using methodology analogous to that in the original paper (Hart et al., 2013) but using open sourced analytic tools and included considerations for repeated observations. The full description of the methodology employed can be found in Additional File 3. At the conclusion of the course, in addition to the self-reflection surveys, the students' scientific competence was evaluated more objectively by asking students to apply their scientific literacy skills in an assessment. All students in the course were given a research scenario during the final discussion section (Additional file 4) and asked to define whether the scenario was an example of scientific hypothesis, experimental, observational or conclusion whether the information in the scenario could be used to answer a particular research question and if so, what question(s) could be explored scientifically. Questions were modeled after those used in the science knowledge quiz by the Pew Research Center A c c e p t e d M a n u s c r i p t (Kennedy and Hefferon, 2019) . Students had the option of requesting a hint to assist in answering the questions if they were not confident in their knowledge. Of the 330 students that completed the course, 82.7% were in majors that were based in biological sciences (e.g., animal biology, animal science, genetics, physiology; n = 273 students) and 17.3% were students in majors that were non-biological science (e.g., theater, human development, communication, sociology, English; n = 57 students). For the students who elected to complete the survey, 84.6% (n=231) were in biological science majors and 15.4% (n=42) were non-biological science majors. At the start of the course, students, on average, tended to be more uncertain of their knowledge of the scientific inquiry process, types of scientific literature, and citizen science (Table 1) . At the conclusion of the course, the students viewed their knowledge in these topics as having significantly improved and were more certain of their understanding (p< 0.005). Student perception of their understanding of research execution and hypothesis testing also changed over the course of the class (p < 0.006). With respect to research interpretation, students were on average tending toward slightly comfortable at the beginning of the course and had moved significantly toward more moderately comfortable at the end of the quarter (p < 0.007). With regard to scientific communication, students were more comfortable with written communication when compared to verbal communication; that differential perspective persisted at the end of the course, however students comfort in both forms of communication grew by ~ 20% (p < 0.05). Importantly, students viewed their overall skill and comfort with scientific research to have grown 30% (p < 0.05) by the end of A c c e p t e d M a n u s c r i p t the course. Students considered the class exercises as being very effective at increasing their comfort level with scientific literature. Interestingly, the question of "Do you agree with and believe everything you read if it is published in a scientific journal?" remained unchanged over the course with students viewing themselves as midway between neither agree or disagree to somewhat agree. Student perspectives on what might influence the accuracy of the conclusions reached by the author were also unchanged from the beginning of the course to the conclusion (Table 2) and targeted poor experimental design as the top reasons (experiment did not address the question, research approach incomplete and too few research subjects) which accounted for approximately two-thirds of the responses. The research scenario was presented during the final discussion sections of the class. A number of 64 students failed to attend and therefore the number of respondents was less than the enrollment in the class. The research scenario questions used to objectively assess scientific literacy indicated that of the 266 respondents, 209 correctly identified a scientific hypothesis ( Fig. 2A) and 234 correctly answered that the data collected for the dog excretions could be used to answer a research question whereas 25 were unsure, and 7 answered that the data collected could not be used to answer a research question (Fig. 2B) . Of the 234 respondents that answered correctly, 179 opted to list research questions using the data without a hint, whereas the other 55 opted for the hint that gave an example of a research question before listing their own research questions. The remaining students who answered "unsure" (n = 25) or "no" (n = 7) for the initial question (that is, can the data collected be used to answer a research question) were routed directly to the hint, after which they listed their research questions. When averaging the self-perception scores related to research execution, students who incorrectly answered the research scenario questions had selfperception ratings equivalent to those of the students who answered correctly (Table 3) . In A c c e p t e d M a n u s c r i p t contrast, when looking at the difference between needing a hint and the self-perception scores, those needing a hint scored their self-perception lower than students who did not need a hint. In their listed research questions, students proposed validating the findings of the work of Hart et al. (Hart et al., 2013) by determining if there was a tendency for dogs to align their body in a North to South magnetic field axis during excretion. Students also suggested to assess if the dog's sex influenced orientation of the different excretions, whether geographical locale or the dog's familiarity with the location influenced how the dog aligns its body, if the dog's age influenced its body alignment, and whether a given dog exhibited a preference to align in a particular magnetic axis (e.g., within dog variation). The data used to answer the students' research questions were comprised of at least three urination and three defecation events recorded for each of 101 different dogs (50 males and 51 females). The mean and mode of the study cohort was 5.95 and 6 years, respectively, with a range of 13 weeks to 14.5 years of age. Using the von Mises density analyses, the data demonstrated that there was no preference for body orientation for urination, defecation either alone or combined. Furthermore, when sex, geographical location, familiarity with the location of excretion, or age was included in the model, none of these effects reached statistical significance as influencers of excretion orientation as seen in the broad 95% limits of the estimates that overlapped extensively, nearly encompassing the full range of angles possible (Table 4 ). That is, there was no unimodal interest for dogs to use a common direction of excretion. Importantly, variability in orientation was similar between and within dogs indicating that individual dogs did not exhibit a preference for particular body alignment evidenced in the small repeatability values having broad 95% limits. The previously published work (Hart et al., 2013) reported that the significant alignment of the dogs' bodies to the North-South axis was seen when the data were adjusted for declination. When the A c c e p t e d M a n u s c r i p t present data were adjusted for declination, there was still no preferred axial orientation detected. Improving undergraduate comprehension of complex scientific topics is the goal of most science based undergraduate college courses along with a need to integrate science more generally into the curriculum (Roberts and Bybee, 2014) . Furthermore, developing lifelong scientific literacy skills in all students, beyond those destined for careers in science, is paramount as is the development of tools for educators (Liu, 2009 ). In fact, Liu (Liu, 2009) contends that a "continuum between formal and informal science education" should exist in a re-envisioning of science literacy instruction. To achieve such objectives, there is a need to creatively bring relevant and interesting approaches into the classroom, as exemplified in a recent report (Rosenthal, 2020) . The pandemic and remote instruction offered an opportunity to revise a general education science course to incorporate more student engagement activities which in turn led to enhanced scientific literacy self-perception by students at the conclusion of the course. Interestingly, when the self-perception responses were compared to the research scenario responses, for students who incorrectly answered the scenario questions, they were as confident in their self-perception related to research execution as students who were correct in their answers. Individuals overestimating their competence in tasks is not unusual and has been defined as the Dunning-Kruger effect (Kruger and Dunning, 1999) with M a n u s c r i p t numerous studies confirming this phenomenon (Burson et al., 2006; Dunning, 2011) although there have been suggestions that the misperception of one's own capabilities may not be real (Nuhfer et al., 2016; Nuhfer et al., 2017; Gignac and Zajenkowski, 2020) . Nevertheless, the total student cohort increased their degree of confidence in their scientific literacy in terms of knowledge about the scientific method and thinking critically. A call to action for science literacy education argued that for scientific literacy education to be useful, it needs to be connected to "the real uses of science in daily life" (Feinstein, 2011) . The course efforts described here included the incorporation of citizen science, that is students contributing to a scientific project by collecting and reporting data for a topic highly relevant to their interests, and to develop a sense of public engagement in the use and application of science. Outside of a classroom, citizen science has been proposed as a way to promote public engagement creating a more well-informed populace, although there have been reports tempering the enthusiasm of that proposition (Riesch et al., 2013; Martin, 2017) particularly because those volunteering to cooperate in citizen science often have a predilection for science. In a classroom setting, where there is opportunity to contextualize the student's citizen project and build skills for understanding and interpretation, controversy surrounding incorporation of citizen science into curricula has focused on the short comings of creating permanent civic engagement by students and limited resources for equitable and fruitful involvement of all students (Gray et al., 2012; Mueller et al., 2012) . The use of citizen science in the context of the present efforts was designed to mitigate some of those hurdles by using a real-world vehicle to build skills in scientific literacy in combination with other activities. A c c e p t e d M a n u s c r i p t In addition to engaging students in citizen science, the analysis of the data collected on dog excretory behavior provided an opportunity to demonstrate the value of study replication and study design. For example, in the present instance the data did not support the findings of the study (Hart et al., 2013) which had formed the impetus for the citizen science activity. Although in that study the authors reported a slight influence of the earth's magnetic field on body alignment during excretions, that was only detected under conditions of stable, calm magnetic fields. The present study, which did not detect any preference for body alignment, utilized a greater number of dogs, collected the data within a brief time window making the data very comparable, and had more balanced representation of individual dogs when compared to the previous study. In particular, with respect to the Hart et al., study (2013) , a single male dog contributed nearly two-thirds of the urination observations. The students in our class rightly questioned how that unbalanced sampling may impact results. A recent publication (Yosef et al., 2020) built upon the study of Hart et al. (Hart et al., 2013) by evaluating dogs observed in Israeli dog parks over two years and although both defecation and urination were recorded, only defecations were reported. They too reported that during defecation, dogs aligned their bodies along a North-South axis and the behavior could be disrupted in the vicinity of buried magnets. The results of the dog excretory behavior collected by the present study, however, do not support a substantive influence of magnetic field on defecation or urination either alone or when combined regardless of sex and location. Yosef et al. (Yosef et al., 2020) used the magnet experimental manipulation of dog excretory behavior also as a high-school class project. Similar to what we proposed here, the authors suggested that dog excretory behavior could be useful as a citizen science application for student learning. However the concerns of Gray et al., (Gray et al., 2012) were not explicitly addressed. That is, the students participating in the different parts of the study may A c c e p t e d M a n u s c r i p t not have been democratized nor were there descriptions of the tools and resources available to students to contextualize the findings, though the students were in a "scientific thought and processes" course led by the lead author (Yosef et al., 2020) . Engaging in directed group discussion of the data was essential. Although students initially expressed skepticism that dogs would align themselves to a particular magnetic axis during excretory behavior, when shown the raw data many students quickly concluded that dogs did in fact exhibit a preference. It is probable that those students did not take into consideration the aggregate number of observations that failed to show alignment preference, but rather focused their attention on clustered subsets of the data that appeared to reinforce the initial paper's findings. Students expressed surprise when the statistical analyses, requiring robust analytical approaches given the uniqueness of circular data, did not support a preference and the discussion demonstrated to the students the need for objective analyses prior to reaching conclusions. Asking the students to participate in this citizen science activity was simple to implement because most people relate to their pets and excretory behavior is something that is repeated daily and thus students could easily record their dog relieving itself. Dog behavior is of interest to students and therefore the act of observing and recording data related to their dog, met many of the criteria for effectively using citizen science as a way to build scientific literacy (Feinstein, 2011; Gray et al., 2012; Riesch et al., 2013) . Specifically, it was a relevant activity from which students could then derive scientific questions. We did not ask students to volunteer to analyze the data but rather all students were asked to participate in posing general questions that could be addressed thereby making engagement more universal (e.g., democratizing to all) regardless of their comfort level with the scientific process. We feel this approach empowered the students to both be engaged in the course as well as having a vested A c c e p t e d M a n u s c r i p t interest in the citizen science aspect which we speculate factored into their improved selfperception at the end of the course. Implementing adjunct learning activities as suggested by Gray et al. (Gray et al., 2012) A c c e p t e d M a n u s c r i p t M a n u s c r i p t M a n u s c r i p t Table 2 . Student responses to the question of "What might indicate that the conclusions drawn in a paper are incorrect?" recorded at the beginning of the course (pre) and then again at the conclusion of the course (post). M a n u s c r i p t A c c e p t e d M a n u s c r i p t Table 4 . Circular and axial angle estimates for each excretion type presented for males and females by absolute orientation and orientation adjusted for declination. Values are presented as the estimate followed by the upper and lower 95% limit in brackets. M a n u s c r i p t Students (n = 266) responded to the questions A) "Dogs align themselves within X degrees of Earth's magnetic north pole. One possible explanation for this is that dogs detect magnetic forces and are drawn to the magnetic pole. This explanation is a scientific___________." and B) "Over the past couple weeks, we've collected the following data for each time a dog relieves itself: name, type of excretion (urination/defecation), sex, age, location familiarity, date of observation, ZIP code or city of observation, magnetic direction, declination, and screenshots of compass app. Can we use this data to answer a research question?" A c c e p t e d M a n u s c r i p t A c c e p t e d M a n u s c r i p t Figure 2 Examining US pet ownership using the General Social Survey Analyzing likert data Skilled or unskilled, but still unaware of it: how perceptions of difficulty drive miscalibration in relative comparisons The Dunning-Kruger effect: On being ignorant of one's own ignorance Salvaging science literacy The Dunning-Kruger effect is (mostly) a statistical artefact: Valid approaches to testing the hypothesis with individual differences data Lessons learned from citizen science in the classroom. a response to" the future of citizen science Dogs are sensitive to small variations of the Earth's magnetic field What Americans Know about Science: Science Knowledge Levels Remain Strongly Tied to Education; Republicans and Democrats Are about Equally Knowledgeable Principles of survey research part 2: designing a survey Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments Scientific literacy: a conceptual overview A technique for the measurement of attitudes. Archives of psychology Beyond science literacy: Science and the public Scientific literacy Citizen science as a means for increasing public engagement in science: Presumption or possibility? The Future of Citizen Science Science literacy: Concepts, contexts, and consequences Random Number Simulations Reveal How Random Noise Affects the Measurements and Graphical Portrayals of Self-Assessed Competency How Random Noise and a Graphical Convention Subverted Behavioral Scientists' Explanations of Self-Assessment Data: Numeracy Underlies Better Alternatives Combining citizen science and public engagement: the Open AirLaboratories Programme Scientific literacy, science literacy, and science education Media literacy, scientific literacy, and science videos on the internet Reconsidering different visions of scientific literacy and science education based on the concept of Bildung, Cognition, metacognition, and culture in STEM education Evaluation of the nutritional adequacy of recipes for home-prepared maintenance diets for cats Directional preferences of dogs' changes in the presence of a bar magnet: Educational experiments in Israel