key: cord-0749442-hkby34bp authors: Berr, Alexandra L.; Ridge, Karen M.; Hu, Jennifer Yuan-Shih title: Pivoting to a Remote-Learning Summer Student Program during the COVID-19 Pandemic date: 2021-10-19 journal: ATS scholar DOI: 10.34197/ats-scholar.2021-0047ps sha: f395a51abe4a8640113939065346b709f90799be doc_id: 749442 cord_uid: hkby34bp Research experience garnered through summer student programs (SSPs) is critical for high school and college student retention in science, technology, engineering, and math (STEM) disciplines. However, the global coronavirus disease (COVID-19) pandemic prevented in-person SSPs in 2020, eliminating these essential experiences that students need to advance their STEM training. In response, we created a remote-learning model that can be broadly adapted for other SSPs. We aimed to uphold our traditional SSP’s academic rigor by cultivating critical thinking skills, providing mentorship, and equipping students with tools to serve as public health ambassadors in their communities. We designed the remote SSP around an anchor topic to integrate didactic lectures with research-based independent projects. Program success was evaluated quantitatively and qualitatively via content assessments and written feedback. By comparing preassessments to postassessments, we show that students gained general scientific literacy and improved critical thinking skills. Based on qualitative measures, students were satisfied with their mentorship, reported that they would use what they learned through the SSP in the future, indicated that they had the tools to understand and communicate public health information, and, overall, rated the quality of the SSP positively. As the pandemic continues to necessitate remote learning, traditional in-person experiences will need to be adapted to best support students. We have developed a modular and adaptable SSP that upholds the same standards as the traditional SSP by continuing to provide essential experiences necessary to advance students’ training in STEM. in-person experiences will need to be adapted to best support students. We have developed a modular and adaptable SSP that upholds the same standards as the traditional SSP by continuing to provide essential experiences necessary to advance students' training in STEM. Keywords: remote learning; undergraduate medical education; community outreach; experiential learning; STEM education There is a projected shortage of science, technology, engineering, and math (STEM) professionals in the United States (1) . Concomitantly, students' pursuit of STEM-focused careers is declining, in part because of misperceptions of the STEM field and the lack of resources to participate in STEM programs (2) (3) (4) . Summer student programs (SSPs) are critical for improving high school and college students' retention in STEM (5) (6) (7) (8) . Research experience gained through SSPs is crucial for students' retention in science majors and can serve as a steppingstone to careers in STEM, particularly for students who identify as underrepresented minorities (URM, defined here as Black or African American, Hispanic or Latino, American Indians or Alaska Natives, Native Hawaiians, and other Pacific Islanders) and/or women (9, 10) . Additionally, participation in research has been recognized as one of the fundamental approaches for maintaining student interest in STEM disciplines (11) . The Division of Pulmonary and Critical Care Medicine at Northwestern University has been hosting an SSP for more than two decades. The program is dedicated to providing enriching research training to high school and university students interested in the biological sciences, with priority given to women and URM students. The backbone of in-person SSPs was a dedicated research experience: students were paired with a mentor and participated in hands-on, wet-bench research for a period of 8-12 weeks. During the in-person SSP, we emphasized three goals: to cultivate critical thinking skills, to provide mentorship, and to train students to be public health ambassadors in their communities. Unfortunately, the coronavirus disease (COVID- 19) pandemic has halted in-person research activities, including traditional SSPs. To remedy this limitation, many institutions, including ours, have been forced to find alternative ways to support students interested in pursuing STEM (12) . Because of the continued urgency to provide summer research experiences for high school and college students, we created an initiative with which to carry on the SSP remotely while still retaining the three key tenets of the in-person SSP. We outline the differences between in-person and remote-learning SSP in Table 1 . In this manuscript, we report how we transitioned the SSP from an in-person to a remote-learning program and our evaluation of the effectiveness of the online SSP. We describe our highly adaptable learning model as a resource for other academic institutions' SSPs. We show that, amid a global pandemic, our program could continue supporting STEM education without compromising the quality of the learning and research. We received more than 60 applications from high school and college students across six states. The program enrolled 21 students (14 high school juniors and seniors and 7 undergraduate students). Forty-five percent of the accepted students admitted to our SSP identified as URM, and 68% identified as female. To continue offering hypothesis-driven research and learning experiences to the students, we divided the curriculum into Block A (9:00 A.M. to 12:30 P.M.) and Block B (1:30 P.M. to 4:00 P.M.) learning blocks, which were anchored around the topic of COVID-19. We chose COVID-19 because of its relevance to both our student population and our division at large, and also because it served as a strong common platform for integrating different areas of knowledge (basic biology, immunology, epidemiology, and research techniques) and skills (critical thinking, problem-solving, teamwork, and presentations) for cultivating students' interest in STEM and for training students to become public health ambassadors. Block A consisted of synchronous lectures and guided discussions. Faculty, postdoctoral and clinical fellows, graduate students, and research technicians led these sessions on the topics listed in Table 2 . Leveraging support from trainees across our department was a central tenet of the SSP; this approach both helped to distribute labor equitably across laboratories and provided an opportunity for trainees to gain valuable mentorship experience. In addition, tutorial videos were assigned to complement lecture material each week. This form of asynchronous learning helped reinforce concepts in the lectures and also helped reduce "Zoom fatigue" brought on by the morning lectures. Gaps in student knowledge, after attending the synchronous lectures and viewing the tutorial videos, were addressed using problembased learning strategies via discussions led by either a clinical fellow or medical student. We designed Block A of the SSP to prime students with the basic scientific information needed to understand the pathogenesis of COVID-19 and to perform research projects in Block B (Table 2 ). This structure, generally speaking, moved from broad to specific. We began with the basics of viral infections and the host response before moving on to the pathophysiology of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection, patient-specific treatment options, and COVID-19 epidemiological trends. New knowledge was built gradually from previously weeks, adding more complexity and specialty as we moved along the program. The more intensive Block A lectures were concentrated in What laboratory method is used to diagnose a coronavirus infection in most hospitals? 1. Which of the following is NOT a feature of the SARS-CoV-2 virus? 2. Which of the following is animal models are used to study COVID-19? 3. Which of the following describes an experiment conducted in a test tube or cell culture dish? Which cell surface receptor mediates entry of the SARS-CoV-2 virus? 6. Which of the following is a type of immune cell? 7. You want to compare relative levels of protein X in Cell A versus Cell B. Which of the following methods would be best suited for this experiment? Rank the following from smallest to largest: alveoli, coronavirus, alveolar epithelial type II cell (ATII), a molecule of carbon dioxide (CO2) 9. Which of the following type of drugs would you expect to be most effective against COVID-19? Which of the following treatment has NOT been considered in COVID-19 clinical trial? Which of the following is true of an effective vaccine? 12. Which of the following is most likely to result in transmission of COVID-19? Which of the following is NOT a risk factor associated with severe COVID-19? Which of the following is true? 15. Oxygen must travel through the lungs before diffusing into the bloodstream. Which of the following most closely describes the proper order of air flow? Weeks 1 and 2 of the program as students built a foundational knowledge of the material and were pared down in Weeks 3 and 4 as students shifted their focus to their Block B independent group projects. The goal of Block B was to help students develop critical thinking, teamwork, and presentation skills. Groups of 4-5 students worked together on original group research proposals with mentors from different laboratories during the asynchronous Block B. Group mentors, who were mainly faculty and postdoctoral fellows, had full autonomy over students' research proposals and progress. Their role was to provide guidance and feedback to help students achieve the weekly research goals set by the SSP (Table 2 ). These goals included identifying a research question, using PubMed to gather relevant background information, identifying appropriate experimental and analysis methods, and preparing a final presentation. Students worked in groups without their mentors but were required to check in daily and to have more in-depth meetings as their mentors saw fit. We collected weekly feedback to ensure the students were on track. To quantitively assess critical thinking skills gained through the program, students were subjected to a pre-and postassessment. Questions were designed to test general knowledge about COVID-19 and to measure how well students could integrate knowledge to answer more complex questions (Appendix 1 in the data supplement). Students displayed mastery of biologic concepts related to the COVID-19 pathogenesis (Figure 1 , Q4, Q7, Q12) and COVID-19-specific information ( Figure 1 , Q1-3, Q6, Q11, Q13). Students also displayed improvement on all questions that tested their critical thinking skills (Figure 1 , Q5, Q8-10, Q15). For example, most students correctly chose the best way to experimentally test for proteins (Figure 1, Q8) . Overall, the percentage of correct responses was significantly increased for all of the questions during postassessments. Of note, one student wrote in their exit survey, "Although I didn't get a chance to continue with hands-on laboratory work, I feel like I learned so much more about critical thinking and how a scientist thinks when designing their experiments, which is just as important and arguably more than just learning experimental techniques themselves." Together, these results indicate that students retained content and improved their critical thinking skills over the course of the SSP. Throughout the SSP, mentorship took several forms. First, small groups of students were directly matched to a division member who served as their primary research mentor for Block B projects. The mentors' role was to guide students through a process of experiential learning (13) . Through this approach, students learn through a cycle of hands-on experience and reflection. Specifically, this approach takes place in four phases: concrete experience, reflective observation, abstract conceptualization, and active experimentation. Students first garnered "concrete experience" through didactic lectures in Block A. This information was reinforced with "reflective observation" through discussions, weekly feedback solicitation, and informal conversations with mentors and peers. In Block B, students experienced the "abstract conceptualization" phase of experiential learning by applying what they were taught in Block A and to address their assigned scientific topics. This allowed students to directly test their understanding of underlying principles, processes and procedures obtained in Block A. Finally, students took part in "active experimentation"; they synthesized information and skills they developed from both content blocks to present to one another in discussion sessions and to ultimately create a project proposal that they presented to the entire department. Working through each of these four tenets of experiential learning created a feedback loop, whereby students took part in problem-solving exercises, received feedback from their mentors and peers, and then continued attempting to solve problems. Because of the compact nature of the material, students received rapid feedback and could immediately identify areas for improvement. This approach was validated by 100% of participants who either strongly agreed or agreed with the statement, "my mentors are approachable, and I feel comfortable asking questions/discussing matters with them" (Figure 2A) . Moreover, 73% of participants strongly agreed with the statement, "I have continued to receive constructive and timely feedback from my mentors on my work" (Figure 2B ). Additionally, students received mentorship through a series of lectures related to career planning. For example, we hosted a weekly "Lunch and Learn" series, in which we enlisted trainees to participate in an informal question-and-answer session covering topics such as college, graduate school, and medical school admissions. Our program also offered professional development opportunities and learning resources by partnering with The Graduate School at Northwestern University in hosting workshops on writing personal essays, developing independent research projects, and communicating research findings ( Table 2 ). These opportunities, together with Block B research experience, helped students clarify their career goals, with one student reporting, "I definitely want to dive deeper into [computational projects] in the future." Interestingly, students also reported experiencing peerto-peer mentorship. Overall, 80% of participants either agreed or strongly agreed with the statement, "I prefer smaller group discussions to big group discussions" (Figure 2C ). One student wrote, "I think working in small groups for the project was very effective for building collaboration skills and allowed us to learn from each other while supporting each other." This feedback indicates that students typically preferred small-group, learner-centered approaches. When asked to rate the quality of the professional network built through the program, 53% of participants rated the quality as "good" or "excellent" ( Figure 2D ). These reports suggest that students received valuable mentorship from a diverse group of people. Building an effective professional network is an important goal of the SSP. However, because of the nature of the remotelearning SSP, the opportunity for in-person interactions between peers and mentors was eliminated. Although students were given many opportunities to network with their peers, mentors, and speakers remotely, some of them did not feel that the networks that they built were as effective as the ones they built in person. This was especially true for professional networking events that we hosted in collaboration with The Graduate School on career development. This was partially because of the lack of structured networking outlets and dedicated networking time after the events. We also observed that students were hesitant to ask questions or speak up in networking events with large audiences. These factors likely contributed to the lower rating of the professional network quality. Misconceptions surrounding COVID-19 are common (14) . To train students to disseminate science-based knowledge of COVID-19, we offered guided discussion sessions as part of the reflective observation stage of experiential learning. First, a guided journal club discussion was designed for students to critically analyze breakthroughs on COVID-19. Students were assigned to a scientific article each week ( Table 2 ). Groups of 3-4 students each presented one figure and posed a discussion question for the rest of the group. Students also participated in a guided news discussion. In small groups, they prepared comparisons of different news stories that covered the same COVID-19-related breakthrough. Students related these media reports to their original scientific data and evaluated whether the media's portrayal accurately recounted the scientific findings. These guided discussion formats helped train students in using science-based knowledge to ensure the spread of accurate information. When asked if the SSP changed students' perceptions of COVID-19, 89.5% answered that it had. One student replied, "I would say this program definitely gave me an amazing education, not just about COVID-19, but about how to assess the news and trials featured in the media critically." Other students echoed the idea that students felt better equipped to analyze media coverage of COVID-19-related topics. Students indicated that they were going to take the pandemic more seriously following the completion of the program. For example, one student said, "[The SSP] taught me that COVID-19 is not a disease you should take lightly and that even if I am healthy, I can still infect and harm other people. While before, I used to push to go to school for this year, now I understand why it is so dangerous." Students also discussed using the knowledge they gained in the program to inform their peers, "I have friends who easily buy into a thing they hear, and I have to tell them that they shouldn't go running toward an idea according to evidence 1, 2, and 3." Another student wrote, "This program has helped me to explain to members of my family and community scientific information that is true to combat the massive amounts of false information." Taken together, these results suggest that students developed the essential skills required to evaluate COVID-19-related media reports and to serve as ambassadors of science-based knowledge within their communities. Students were asked, "How much do you think the materials you learned in this summer program will have practical applications in your life outside of this program?" In response, 68% of the students answered, "Very much" and 21% answered "To the greatest extent" ( Figure 2E ). Specifically, one student cited "The Prism graphing, searching PubMed, and just how to organize a slideshow for a presentation" as skills they would apply in the future. Students also noted collaboration, critical thinking, and experimental design as skills they built throughout the program. Overall, 100% of the students rated the quality of the program as either "Excellent" or "Good," suggesting that they considered their time in the SSP a positive experience ( Figure 2F ). Together, these results indicate that students were enriched by their SSP experience and plan to use what they learned in future endeavors. This manuscript presents an effective model for transitioning from an in-person to a remote-learning model SSP. Specifically, we met the same goals as our traditional SSP, including cultivating critical thinking skills, providing mentorship, and equipping students with tools to serve as public health ambassadors. We showed, through qualitative and quantitative metrics, that the remote-learning SSP improved scientific literacy and critical thinking skills, provided quality mentorship, and prepared students with the tools to understand and communicate public health information. Overall, students rated the quality of the program highly. Although it is likely that our students will return to in-person research this summer, this highly adaptable remotelearning model can be feasibly modified for dynamic hybrid learning in the future to complement the in-person learning model of SSP with any other themes and topics. For example, core lectures can be prerecorded and/or take place remotely to provide both mentors and students with more flexibility and accessibility. Online platforms, such as Zoom and Slack, can also provide an additional space for students to work collaboratively outside the laboratory setting. Incorporating elements of remote learning into our traditional SSP will improve accessibility for students who are not able to be in the laboratory 40 hours a week. Together, these modifications will provide a more robust learning experience for all students. As we continue to build on this model, we note several challenges that we would address in future iterations of this program. First, students reported experiencing "Zoom fatigue" in their qualitative assessments. Therefore, we recommend shortening lecture time from 1 hour to 50 minutes, scheduling break time between each lecture, and minimizing scheduling consecutive lectures. It is likely that, as we return to in-person and/or hybrid learning formats, screen time will decrease naturally. Second, the evaluations on the quality of the professional network formed during remote-learning SSP were not as high as other metrics ( Figure 2D ). This was likely due to the constraint of the remotelearning format and the lack of opportunity for in-person interaction. We therefore recommend building in structured, informal networking time for students to better establish and maintain networks with their peers and members of the division. Another challenge was the burden of lecture planning that fell on members of the division with little preparation time. To assuage some of this responsibility, we recommend taking advantage of outside resources to supplement the content that instructors create. This saves instructors time and can also reframe information, which can be helpful to students across different learning styles. To this end, we worked with the Northwestern Galter Health Sciences Library and The Graduate School, which sponsored seminars and workshops that they opened to our summer students. We also sourced videos that we used to supplement lecture information. This approach created a "flipped" classroom to accommodate those who learn better from Socratic discussions than from traditional lectures. This allowed students to move at their own pace and removed the burden of lesson planning from mentors. We have several recommendations for other institutes that plan to implement similar programs. First, universities should provide stipends to those participating in their SSPs, even if they are held remotely. For many students, receiving a stipend is critical for their ability to participate in these programs. Therefore, although the pandemic has decreased operating budgets at many higher education institutes, SSP budgets should be prioritized. Second, we recommend taking an active approach to cultivate community between participants. This includes building in time during which students can bond among themselves without members of the division present. Finally, we suggest gathering anonymous feedback weekly. This can help catch problems early; for example, a short survey can reveal if students are overloaded, understimulated, or experiencing screen-time burnout. In summary, we have created a remotelearning model for an SSP. This model leverages the knowledge base of many different division members, including research technicians, trainees, and faculty. Although the COVID-19 pandemic has drastically changed the landscape of higher education and laboratory-based research, there remain viable ways to maintain student engagement in SSPs. Although we do not yet know our model's long-term success in retaining students in STEM, our metrics indicate that the program was successful in the short term. The US science and engineering workforce: recent, current, and projected employment, wages, and unemployment STEM attrition: college students' paths into and out of STEM fields Institute of Education Sciences, US Department of Education Half of Americans think young people don't pursue STEM because it is too hard Incremental beliefs, STEM efficacy and STEM interest among first-year undergraduate students I actually contributed to their research': the influence of an abbreviated summer apprenticeship program in science and engineering for diverse high-school learners Maximizing future potential in physics and STEM: evaluating a summer program through a partnership between science outreach and education research The STEM pipeline: the role of summer research experience in minority students Retention of STEM majors using early undergraduate researchexperiences Equity for women and underrepresented minorities in STEM: graduate experiences and career plans in chemistry Diversifying science: underrepresented student experiences in structured research programs Division of Behavioral and Social Sciences and Education; Division on Earth and Life Studies; Board on Science Education; Board on Life Sciences; Committee on Strengthening Research Experiences for Undergraduate STEM Students. Undergraduate research experiences for STEM students: successes, challenges, and opportunities Transitioning undergraduate research from wet lab to the virtual in the wake of a pandemic Experiential learning: experience as the source of learning and development Knowledge and perceptions of COVID-19 among the general public in the United States and the United Kingdom: a cross-sectional online survey Susceptibility of ferrets, cats, dogs, and other domesticated animals to SARS-coronavirus 2 Complex immune dysregulation in COVID-19 patients with severe respiratory failure A randomized trial of hydroxychloroquine as postexposure prophylaxis for Covid-19 Modelling the COVID-19 epidemic and implementation of population-wide interventions in Italy COVID-19. Coronavirus: how is coronavirus diagnosed COVID-19. Coronavirus: epidemiology, pathophysiology. YouTube How to choose the right statistical test How to perform common statistics in prism COVID-19. Coronavirus: treatment, prognosis, precautions. YouTube Coronavirus outbreak: performing a nasal swab test on patients inside a rapidly deployable facility optimized for epidemics. Coronavirus Free Access Resource Center Coronavirus outbreak: donning and doffing personal protective equipment (PPE) for healthcare providers. Coronavirus Free Access Resource Center Coronavirus outbreak: how to perform a bronchoscopy The authors thank the faculty, fellows, graduate students, and staff in the Division of Pulmonary and Critical Care Medicine for their continuous support for the summer student research program and for making the remote-learning SSP during the COVID-19 pandemic possible. They also thank The Graduate School's Office of Diversity and Inclusion Summer Research Opportunity Program for providing the program with professional development workshops resources. Author disclosures are available with the text of this article at www.atsjournals.org.