key: cord-0290596-lxb88obb authors: Archer, Martin O.; DeWitt, Jennifer; Thorley, Charlotte; Keenan, Olivia title: Evaluating participants' experience of extended interaction with cutting-edge physics research through the PRiSE 'research in schools' programme date: 2021-04-09 journal: nan DOI: 10.5194/gc-4-147-2021 sha: f48720019b8b9525ea725c1919cc56de8c48f443 doc_id: 290596 cord_uid: lxb88obb Physics in schools is distinctly different from, and struggles to capture the excitement of, university research-level work. Initiatives where students engage in independent research linked to cutting-edge physics within their school over several months might help mitigate this, potentially facilitating the uptake of science in higher education. However, how such initiatives are best supported remains unclear and understudied. This paper evaluates a provision framework, `Physics Research in School Environments' (PRiSE), using survey data from participating 14-18 year-old students and their teachers to understand their experience of the programme. The results show that PRiSE appears to provide much more positive experiences than typical university outreach initiatives due to the nature of the opportunities afforded over several months, which schools would not be able to provide without external input. The intensive support offered is deemed necessary, with all elements appearing equally important. Based on additional feedback from independent researchers and engagement professionals, we also suggest the framework could be adopted at other institutions and applied to their own areas of scientific research, something which has already started to occur. the alignment of physics with cleverness and can make even high-attaining students' confidence in the subject precarious. Teachers and the school environment often (even unconsciously) reinforce stereotypes about physics and physicists that are patterned by biases. Curriculum practices in physics often teach oversimplifications at younger ages which are later completely reconceptualised without being presented as refinements to a model, making students perceive the simpler versions as "lies". Furthermore the general deferment of "interesting" physics in the curriculum produces a disconnect between "school physics" and "real physics", i.e. the cutting-edge research undertaken by professional physicists, making continued participation in physics education something of a "test of endurance". These concerns are further reflected in results from national surveys. While 20% of 16-18 year-old physics school students in the UK aspire towards a physics degree and 80% aspire towards STEM more broadly (Wellcome Trust, 2017) , only 9.7% and 59.3% actually go on to study either physics or STEM respectively (McWhinnie, 2012) . These constitute odds ratios for aspirations vs. destinations of 2.3 and 2.7, both of which are considerable. All of these issues raised cultivate and contribute to reproducing inequitable, and low overall, patterns of participation within physics (L. . suggest that for STEM subjects in general an intervention approach that sustains and supports science identity is most appropriate for students in late secondary/high school education in the context of their educational journey. However, in the case of physics specifically, L. comment that existing interventions based on simply enthusing, inspiring and informing students about physics will not significantly change uptake or diversity in post-compulsory physics. While they advocate for widespread changes in science education policy and practice, both at school and university levels, they note that if interventions are also used they need to fundamentally address the problematic processes and practices present within both physics teaching and physics as a field generally. The stark differences between school, university, and professional science practices have long been noted -while research is one of the main activities of professional scientists, it is quite removed from how science is taught in schools with some arguing science education is not "authentic" in this respect (e.g. . Indeed, report that school students are largely unaware of what research actually is, finding a disconnection between 'research as information gathering' and the 'research question', and in general have little opportunity to set their own research questions within their school environment. Independent research projects, which provide extended opportunities for students to lead and tackle open-ended scientific investigations (not simply literature reviews or essays, e.g. Conner, 2009; Corlu, 2014) , may be one way of exposing students to "real science". These align with established international pedagogical initiatives such as 'inquiry-based science' (e.g. Minner et al., 2010) , 'problem-based learning' (e.g. Gallagher et al., 1995) and 'authentic science' (e.g. . A survey of such projects across twelve countries (Australia, Ireland, Israel, Netherlands, New Zealand, Qatar, Singapore, Spain, Taiwan, Turkey, UK, and USA), however, found them to be rare globally and only sometimes supported by mentors from university/industry (Bennett et al., 2016 . This review found considerable variability in the nature of independent research projects such as their focus, delivery/provision models, external support, and funding/costs. It was noted that such programmes place demands on time and money beyond standard provisions for all stakeholders, on the skills required by teachers and other adults involved, and on the supporting infrastructure. For successful projects Dunlop et al. (2019) recommend that students should be given the freedom to devise a research question, have ownership over their own data 2 analysis and decision-making, and be given access to experts in their project work. Broadly there are two distinct formats of independent research projects: -Those associated with dedicated out-of-school events of only a few weeks' duration such as internships/apprenticeships (e.g. 'Nuffield Research Placements' in the UK, Cilauro and Paull, 2019 ; 'Raising Interest in Science and Engineering' in the USA, Stanford Office of STEM Outreach, 2020), summer schools/camps (e.g. the 'International Astronomical Youth Camp' run across Europe and parts of Africa, Dalgleish and Veitch-Michaelis, 2019) , or science competitions/fairs (particularly prevalent in the USA, e.g. Yasar and Baker, 2003) . -Those undertaken within school itself over the course of several months to a year, either in class or supplemented with time in after-school clubs (e.g. an after-school mechatronics project in Taiwan, Hong et al., 2013; class-based biology project in Singapore, Chin and Chia, 2010; or various 'CarboSchools' climate change projects between research institutes and schools across 7 European countries, Dijkstra and Goedhart, 2011) . We do not discuss the former here as they are necessarily limited in reach, catering only to heavily bought-in individuals (i.e. typically 1-3 students; Paull and Xu, 2017) from any given school. Most independent research projects based within schools are not linked to current cutting-edge and novel scientific research topics or questions. However, relatively recently so-called 'research in schools' projects have emerged, which do provide students with experiences of genuine contemporary STEM research within their own school environment over several months. While several citizen science projects have also run within schools and aim to help participants learn about current science and to experience the scientific research process, these are typically secondary aims since they primarily concern a single (or small number) of well-defined science questions which will be assisted through developed citizen science protocols (Bonney et al., 2009 (Bonney et al., , 2016 Shah and Martinez, 2016) . This contrasts with independent research projects, and thus also 'research in schools', where positively affecting the participants is the primary concern and the projects are necessarily open-ended. Nonetheless, the different approaches can have some overlap and indeed some projects denoted as citizen science, such as the 'curriculum-based' projects based in the USA described by Bonney et al. (2016) , might perhaps be better framed as 'research in schools'. 'Research in schools' programmes appear at present to be most prevalent in the UK and we are aware of three featuring projects in the physical sciences (outside of that at Queen Mary University of London, QMUL, which forms the subject of this paper). HiSPARC (High School Project on Astrophysics Research with Cosmics) is a scintillator-photomultiplier cosmic ray detector project originating in the Netherlands at Radboud University in 2004, which has subsequently been adopted by other Dutch (Amsterdam, Eindhoven, Groningen, Leiden, Nikhef, Twente, Utrecht) as well as UK (Bath, Bristol, Birmingham, Sussex) and Danish (Aarhus) universities (Colle et al., 2007; van Dam et al., 2020; HiSPARC, 2018) . Many of these universities operate a tiered membership scheme for schools: 'Gold' enables schools to buy their own detector (£5,500 in the UK); 'Silver' is a detector rental scheme (£300 p.a. plus an installation fee) with the contract specifying if they do not participate the detector will be collected with an additional fee; and 'Bronze' membership (£200 p.a.) gives schools access to HiSPARC data but not their own detector. Schools signing up for 'Silver' or 'Bronze' membership are contractually obliged to generate funding to upgrade to 'Gold'. While the 'Gold' membership fee covers the costs of the detector and installation, the other memberships are to ensure that schools make a commitment to working with the university (J. Velthuis and M. Pavlidou, personal communication, 2016; National HE STEM SW, 2012) . It is not possible to compare how these schools go about project work and how much support they are given by participating universities, which may vary by institution, as at the time of writing HiSPARC has not published any reviews of their processes or evaluation. IRIS (Institute for Research in Schools) is a UK charity formally launched in March 2016 (IRIS, 2020) , building on the previous CERN@School project conceived in 2007 (Whyntie et al., 2016; Parker et al., 2019) . While IRIS's projects cover all the sciences, current physics projects include the aforementioned CERN@School, Higgs Hunters (Barr et al., 2018) , LUCID Hatfield et al., 2019) , and Webb Cosmic Mining (in preparation for the James Webb Space Telescope). They have rapidly expanded across the UK since formation, having worked in some capacity with over 230 schools as of 2020. Publications have provided technical details of their projects and case studies of some students' successes within them, including a few examples of resulting peer-reviewed scientific work, however the exact provision/delivery model implemented and precisely how project work is supported is not fully explored in the available literature. IRIS aims to develop 'teacher scientists', teachers that identify as both science teachers and research-active scientists (Rushton and Reiss, 2019) , which suggests a teacher-driven model. While some researchers/academics have designed or consulted on some IRIS projects, they appear in general to have little involvement supporting students or teachers (O. Moore, personal communication, 2020) with IRIS itself being the main point of contact for schools. With a recent change of staff at IRIS in late 2019 has come a reformulation of how they classify their projects. 'Seed' projects are for new schools, are the most straightforward, and receive the most support from IRIS though it is not clear in what form that takes. 'Sprout' projects are more advanced seeing students carrying out more complex activity to assist scientists with their research questions, though how this collaboration operates is not specified. 'Grow' projects are where students have proposed their own research questions, either independently or using IRIS resources, with IRIS merely providing advice in producing posters, talks, or papers as well as opportunities to present. ORBYTS (Original Research By Young Twinkle Scientists), based at University College London, was piloted from January 2016 and is nominally based around the Twinkle mission, though has expanded into other research areas since ORBYTS, 2019) . A select group of students (with an imposed limit of 4-6) from each school undertakes fortnightly meetings with early career researchers (either PhD students or post-docs) throughout their project aiming to achieve, where possible, publishable scientific results (McKemmish et al., 2017; Chubb et al., 2018; Holdship et al., 2019) . Teachers, while present, are not typically actively involved in these sessions and students tend to do little independent work outside of the sessions (W. Dunn, personal communication, 2018) . The content of the projects change each year to align with the researchers' current focus, with them typically working with only one school per year each. PhD researchers are paid for their (preparation, travel, and session) time with funds from independent schools, who pay not only for their school but for enabling an additional school from a lower socio-economic background to take part. It is clear that there is currently a lack of published details and evaluation on provision within the emerging area of 'research in schools'. This paper therefore explores these aspects applied specifically to the 'research in schools' programme of QMUL's School of Physics & Astronomy. This was piloted between 2014-2016, as detailed in M. O. , and is now known as 'Physics Research in School Environments' (PRiSE, 2020) . Section 2 introduces PRiSE's framework, which is then evaluated in section 4 in terms of participating students' and teachers' experience. We also briefly discuss how the PRiSE approach has been received by the university sector in section 5. 'Physics Research in School Environments' (PRiSE) is a collection of physics-based 'research in schools' projects (see Ta- ble 1) brought together under a coherent provision framework. The programme aims to equip 14-18 year-old school students (particularly those from disadvantaged backgrounds) with the ability, confidence, and skills to increase/sustain their aspirations towards physics or more broadly STEM, ultimately enabling them to realise these at higher education and thus contributing to increased uptake and diversity of physics, and to some extent STEM (cf. L. . Through working with teachers, PRiSE also aims to develop their professional practice and build long-term university-school relationships that raise the profile of science and mitigate biases/stereotypes associated with physics within these schools, generally making them environments which nurture and enhance all students' science capital (cf. IOP, 2014). Our rationale for these particular outcomes is left to the supplementary material. This section summarises the PRiSE framework, discussing the ethos of the programme, the roles played by the schools and university, as well as outlining the various activity stages, interventions, and resources which it consists of. In-depth practical details aimed at practitioners looking to replicate the framework are given in the supplementary material. PRiSE takes the 'research in schools' approach to schools engagement, whereby students are given the opportunity to lead and tackle open-ended scientific investigations in areas of current research. Therefore, the PRiSE projects were developed to transform current scientific research methods, making them accessible and pertinent to school students so that they could experience, explore, and undertake scientific research themselves. Since the programme intends to influence school students and teachers a number of ethical considerations have been taken into account, following the BERA (2018) guidance for educational research, with regard to safeguarding and to ensure that no harm results. Firstly, to ensure equality of access to the programme we do not charge schools to be involved (cf. Harrison and Shallcross, 2010; Jardine-Wright, 2012 ) and try to provide them with all the physical resources they need for their project, thereby removing potential barriers to entry for less resourced schools. Our targeting takes into account several school-level metrics (type of school, students on free school meals, indices of multiple deprivation, gender balance etc.) to ensure diversity. We aim for the programme to be equitable with all schools being offered the same interventions/opportunities, taking into account and being flexible to their specific needs where necessary. We work with as many schools as we have capacity to do so each year and do not withold interventions from any students for the purpose of having control groups. The projects are optional and presented as an opportunity that students can take advantage of which will be supported by their teacher and the university, therefore students are not pressured into being involved. Students and schools can drop out at any point within the programme with no penalty, which does occur in a minority of schools. The involvement of teachers at all stages is of paramount importance in terms of delivery and safeguarding. They have helped shape the design of the programme, inform how we update it each year, and serve as our liaison to schools and the students involved. It is the teachers that decide who projects are offered to within their school, with us simply advising that the projects should be suitable for all A-Level (16-18 year-old) students as well as high-ability GCSE (14-16 year-old) students (further contextual information on the UK education system is given in Appendix A). These recommendations were made based on the basic background knowledge required to meaningfully engage with the research. Invariably teachers choose to involve older age groups, with 79 ± 1% of PRiSE students being aged 16-18 (and so far only one student below our recommended ages has been involved, being 13-14). Unfortunately, we do not have any specific information on exactly how teachers go about selecting students. However, the average number of students per school each year is around 12, which compared to the national average class size in A-Level physics of 16 (RAE et al., 2015) indicates teachers involve a significant majority (or in many cases the entirety) of their cohorts in PRiSE. We allow teachers to determine how best to integrate the projects within their school, though provide advice on this. We also aim, through our resources and communications, to equip teachers to manage the day-to-day aspects of the projects without overly burdening them -their role is chiefly one of encouraging their students to persist, providing what advice they can, and then communicating with the university. Teachers' involvement at all stages also presents opportunities to them for continuing professional development, helping them nurture and cultivate STEM aspirations among students throughout their school (i.e. not just PRiSE students). This is implemented informally and integrated within the programme in the form of both bespoke resources and ongoing dialogues between teachers and researchers. These are aimed to enhance teachers' knowledge about the underlying science and how they link to curriculum-based topics where appropriate, their skills and confidence surrounding current research topics and methods, and their pedagogy in mentoring independent project work. It was recognised that teachers in general likely will not have the skills or experience in research to manage projects without expert assistance (Shah and Martinez, 2016; Bennett et al., 2016 . Therefore, PRiSE was designed to be supported by active researchers equipped with the necessary expertise to draw upon in offering bespoke, tailored guidance to the students and teachers. Well-defined roles within the university have been established for each of the PRiSE projects to provide this support: -Outreach Officer: Manages the entire programme including university-school relationships, communications, intervention/event co-ordination, programme finances and evaluation. -Project Lead: Visible figurehead for the project to schools, typically an academic member of staff. -Researcher: Providing advice and guidance to students and teachers throughout the programme. Can be delegated to or shared with early career researchers. Since the primary focus of PRiSE is (unlike typical citizen science) on the participants rather than the research, researchers should not consider students contributing to novel research as their rationale for being involved. Our position is that it is rather unreasonable to expect investigations that are motivated by school students themselves (an established element of good practice in independent research projects, e.g. Dunlop et al., 2019) to be able to make meaningful contributions to the physics research as a matter of course. We note that in some exceptional cases PRiSE students' work has arrived at promising preliminary results, though these have required significant follow-up work by professional researchers to transform the results into publishable research (e.g. M. O. and thus should not be considered the archetype. Instead, researchers are enticed by the possibility of societal impact underpinned by their research. This is something which is increasingly called upon from funders (e.g. National Coordinating Centre for Public Engagement, 2020) and is notoriously difficult for areas of 'blue skies' research such as physics. Furthermore, significant contribution to a coordinated departmental outreach programme can be used as criteria for academic promotions (cf. Hillier et al., 2019) . Physics researchers though are largely unmotivated in delivering curriculum content as part of their engagement work, valuing instead aspects relating to their research and role as a researcher . PRiSE thus also aligns with this direction. Ultimately, researcher buy-in is vital to the delivery of protracted research-based engagement programmes such as PRiSE. It is clear from Table 1 that the topics and activities of current PRiSE projects vary considerably. This suggests that a wide range of fields and project ideas might be able to adopt the PRiSE framework. How projects have been developed has also varied (further explored in the supplementary material) though we have adopted a pragmatic approach in taking advantage of opportunities (grant funding, internships etc.) and adapting existing materials where possible, since creating a project from scratch is a significant undertaking far beyond what most academics (unfortunately) have capacity to do (cf. . PRiSE runs from the start of the UK academic year to just before the spring/Easter break, which teachers had informed us during the pilot stage is manageable and largely fits around exams / other activities for most (but not necessarily all) schools (M. O. . The structure has evolved naturally from the pilot to that shown in Figure 1 . Students work in research groups of typically five people and they are advised to try and work on the project on average for 1-2 hours a week. The bulk of this is done outside of regular physics lessons, though some schools integrate the projects within their timetabled 'science clubs' or required extra-curricular blocks, whereas other teachers arrange a regular slot for students to work on the projects or leave it up to the students to arrange (though this latter approach often proves unsuccessful). PRiSE projects are split up into three activity stages (see Figure 1 ). -Prescribed work: Given that independent research in STEM is probably unfamiliar to the students, rather than expecting them to be able to come up with their own avenues of investigation in an unfamiliar research topic straight away, we instead give them an initial prescribed stage of research. -Independent project: Groups are encouraged to set their own research questions and undertake different projects in the topic area when ready. This enables every group to explore something different so that students gain a sense of independence and ownership of their work. -Writing up: Near the end of the project students produce either a scientific poster or talk to be presented at an annual conference. Several different interventions form the structure and support behind the activity stages: -Assignment: The opportunity is advertised to schools via existing teacher networks and teachers apply to participate in the following academic year. Schools are assigned a project before the summer break. -Kick-off: Typically on-campus event featuring an introductory science talk, outline of how the project will work, and a hands-on workshop. -Visit: Researchers visit schools to mentor students (and their teachers) on their project work in a student-driven intervention. -Webinars: Drop-in online sessions similar to school visits but allowing students and teachers to gain further support. -Ad hoc: Further asynchronous support via email as required. -Comments: Students are offered the opportunity to receive comments on their draft slides or posters.. -Conference: Students present the results of their projects as oral or poster presentations at a special conference at the university, attended by researchers as well as the students' teachers, peers, and family. Stakeholders' roles within these interventions are given in Figure 1 , with photos depicting some of them displayed in Figure 2 . All stages of the programme and the processes involved are communicated to teachers via email to pass on to their students. To enable the students to take part in PRiSE, the students and teachers are also provided with numerous resources. While some projects require specific equipment, data and software, here we discuss more common types of resources across the different projects as shown in Figure 1 . -Project poster/flyer: Given to teachers to help advertise projects in their school. -Project guide: Each project has a student guide covering an introduction to the research field, background physics/theory, an explanation of the equipment/data, discussion of analysis techniques, details of the initial prescribed activity, suggested research questions / methods for independent research, and links to other sources of information. Teachers are provided with the same guide, but with extra guidance. -Project webpage: These showcase anonymised examples of good quality talks/posters that previous students have produced as well as providing any links or videos relevant to the project. -'How to' guide: General articles applicable to most research projects, such as producing scientific talks and posters. PRiSE's framework attempts to find a balance between the (necessarily competing) reach and significance of the interactions. For example, an academic staff member acting as both project lead and researcher within PRiSE can support 4 schools' participation (∼50 students), taking around 8 hours over the course of 6 months (cf. Figure 1 ). Using PhD students or postdocs in the researcher role(s) makes even more efficient use of time. In contrast, under the ORBYTS model each early career researcher can support only one school (4-6 students) with 10 hours of their time. As noted in the introduction, mentorship from active researchers throughout does not appear prevalent in other 'research in schools' programmes at present. Programmes of repeat-interventions with schools will necessarily have a smaller reach than various one-off events. However, PRiSE's efficiency in researcher time is reflected in its reach as shown in To determine the perceived value of PRiSE's approach with its key stakeholders, namely participating students and teachers as well as those across the wider university sector, we have maintained regular collection of evaluative data (cf. Rogers, 2014, and references therein) via various surveys which we detail here. This data underpins our understanding of PRiSE and has been collected securely to protect all participants, in compliance with GDPR and in line with the BERA (2018) guidelines for educational research. We gathered feedback from participating students and teachers via paper questionnaires handed out at our student conferences each year. The only exception to this was in 2020, where online forms were used due to the COVID-19 pandemic causing that year's conference to be postponed. The questionnaire method was chosen so as to gather data from as wide a range of students and teachers as possible, respecting the limited time/resources of all involved (both on the school and university sides). For ethics considerations all feedback was anonymous, with students and teachers only indicating their school (pseudonyms are used here to protect anonymity) and which project they were involved with. Students were not asked to provide details of any protected characteristics (such as gender or race) or sensitive information (such as socio-economic background). Both students and teachers were informed via an ethics statement on the form that the information was being collected for the purpose of evaluating and improving the programme and that they could leave any question they felt uncomfortable answering blank (this functionality was also implemented on the online form for consistency). The open and closed questions concerning participants' experience of the programme, which varied slightly year-to-year, are given in Appendix B. While we attempted to collect responses from all participants in attendance, invariably only a fraction did so yielding results from 153 students and 45 teachers across 37 schools. A breakdown of the number of respondents and their schools per year is given in Table 3 , where the number of participants and schools in attendance at our conferences are also indicated. We do not have reliable information on how many students, teachers, and schools would have successfully completed the programme in 2020 due to the COVID-19 disruption. Students and teachers did not always answer all of the questions asked, hence we indicate the number of responses for each question considered throughout. There is no indication Teachers 1/1 (100%) 6/6 (100%) 6/11 (55%) 9/16 (56%) 6/16 (38%) 17/? Schools 1/1 (100%) 6/6 (100%) 11/ 11 (100%) 13/15 (87%) 11/15 (73%) 19/? Table 3 . Response rates to questionnaires at PRiSE student conferences. that the respondents differed in any substantive way from the wider cohorts participating in the programme. While ideally one would also gather feedback from schools which dropped out during the year, a similar formal feedback process has not been viable bar in a few cases where only the teachers responded. Feedback from the university sector came from a session at the 2019 Interact symposium that presented the challenges to STEM outreach practice highlighted by recent educational research, the need for deeper programmes of engagement with young people, and then summarised the PRiSE approach as one possible example (M. O. Archer, 2019) . Throughout this workshop an anonymous interactive online survey was used for interactivity and to collect data presented in this paper. The survey included both closed and open questions as listed in Appendix C. Attendees were fairly evenly split between UK university researchers and engagement professionals (gauged in-person by attendees raising their hands when asked), with 19 people participating in the survey and only 7 not doing so. Participants were allocated a unique number by the online survey itself, which did not distinguish between researchers and engagement professionals. Both qualitative and quantitative approaches were utilised in data analysis, as the open and closed ended questions present in the questionnaires produced different types of data. For all quantitative data, standard (i.e. 68%) confidence intervals are presented throughout. For proportions/probabilities these are determined through the Clopper and Pearson (1934) method, a conservative estimate based on the exact expression for the binomial distribution, and therefore represent the expected variance due to counting statistics only. Several statistical hypothesis tests are used with effect sizes and two-tailed p-values being quoted, with the required significance level being α = 0.05. In general we opt to use nonparametric tests as these are more conservative and suffer from fewer assumptions (e.g. normality, interval-scaling) than their parametric equivalents such as t-tests (Hollander and Wolfe, 1999; Gibbons and Chakraborti, 2011) . The Wilcoxon signed-rank test is used to compare single samples to a hypothetical value, testing whether differences in the data are symmetric about zero in rank. When comparing unpaired samples a Wilcoxon rank-sum test is used, which tests whether one sample is stochastically greater than the other (often interpreted as a difference in medians). Finally, for proportions we use a binomial test, an exact test based on the binomial distribution of whether a sample proportion is different from a hypothesized value (Howell, 2007) . For ease of reference, further details about the quantitative analyses are incorporated into the relevant sections of the findings. Qualitative data were analysed using thematic analysis (Braun and Clarke, 2006) . Instead of using a priori codes, the themes were allowed to emerge naturally from the data using a grounded theory approach (Robson, 2011; Silverman, 2010) as follows: 1. Familiarisation: Responses are read and initial thoughts noted. 2. Induction: Initial codes are generated based on review of the data. 3. Thematic Review: Codes are used to generate themes and identify associated data. 4. Application: Codes are reviewed through application to the full data set. 5. Analysis: Thematic overview of the data is confirmed, with examples chosen from the data to highlight the themes. In this section we use the feedback from participating students and teachers to evaluate the provision offered within the PRiSE framework, specifically assessing their experience and the level of support offered. Firstly from 2016 onwards we asked both students (n = 150) and teachers (n = 42) "Have you been happy with the research project overall?" giving options on a 5-point Likert scale, which we coded to the values 1-5. This scale and the results are displayed in Figure 3 , revealing that 91 ± 3% of students and 95 ± 5% of teachers rated their experience as positive (scores of 4-5) with only three students giving a negative reaction (scores of 2). Teachers tended to rank this question somewhat higher (their mean score was 4.50 ± 0.09, where uncertainties refer to the standard error in the mean) than students (mean of 4.17 ± 0.05), with p = 0.002 in a Wilcoxon rank-sum test. The PHwP project scored slightly higher (average of 4.59 ± 0.11, p = 8 × 10 −4 ) than the overall results with students, whereas ATLAS scored slightly lower with both students (3.92 ± 0.09, p = 0.012) and teachers (3.80 ± 0.20, p = 0.017) than their respective means. No obvious trends were present by school. While suggestive of extremely positive experiences with PRiSE, one also needs to compare these distributions against the typical responses of students and teachers for schools STEM engagement programmes. We use the results of Vennix et al. (2017) as such a benchmark, which surveyed 729 high-school students and 35 teachers about 12 different STEM outreach activities in the USA and Netherlands. This comparison reveals that PRiSE seems to be perceived considerably more positively than usual by both students (benchmark average 3.66 ± 0.01, p = 1 × 10 −15 in a one-sample Wilcoxon signed-rank test) and teachers (benchmark average 3.84 ± 0.08, p = 1 × 10 −7 ). Secondly, students (n = 135) were asked for adjectives describing their experience of the projects overall. They were free to use any words they wanted and were not given a pre-selected list. Teachers (n = 38) were similarly asked to indicate observations of their students' experience also. Since 2016 this has resulted in 88 unique adjectives, with both students and teachers typically writing 2-3 words each. We present the results as the word cloud in Figure 4 , where students and teachers have been given equal prominence by normalising their counts by their respective totals. We have indicated by colour from which group(s) the words originated, generally showing a lot of agreement between students' thoughts and teachers' observations. The most cited adjectives were (in descending order) interesting, challenging, exciting, inspiring and fun, similar to those from the pilot (M. O. , with the top two adjectives being significantly greater than the subsequent ones. While in the pilot stage only positive adjectives were expressed, since then a few negative experiences have been conveyed such as time consuming, frustrating and stressful. These constitute a small minority of experiences though (6 ± 1%) and in most cases the same students also listed positive adjectives apart from only four individuals. Following on from these quantitative analyses, we qualitatively explore the potential reasons behind the results. The most common themes that emerged from students' (n = 110) responses to open questions about their experience were that they feel they learnt a lot (62 responses) "We have learnt so many new things relating to the magnetosphere and waves and we have developed new skills." "It was very nice to work with friends and work together to produce something." (Student 106, Boston Bay College, "It was very fun to do our own research and I appreciated that help was always available even thought it is very independent. It also shows how challenging research can actually be but also how rewarding it is once you start This is exactly the experience I wanted them to have, and they were able to discover some genuinely novel processes that had not been observed before -the hallmark of great scientific research!" (Teacher 44, Sunnydale High School, MUSICS 2020) Therefore, both quantitative and qualitative data suggest students and teachers had much more positive and rewarding experiences participating in PRiSE projects than is typical for schools engagement programmes from universities in general. We originally asked students whether they felt they had received adequate support, finding overall positive results on a 5-point Likert scale (M. O. . However, students' qualitative responses explaining their answers often revealed a conflation of the support provided by Queen Mary with that offered by their teacher. Therefore, from 2019 onwards we explicitly separated these two aspects. Students (n = 68) were asked "Do you feel that support from your teacher was provided/available during the project?" which yielded the following results: Strongly Agree (30), Agree (34), Neither Agree or Disagree (3), and Disagree (1). The average response is 4.37 ± 0.08, which is considerably greater than the benchmark on teacher support reported by Vennix et al. (2017) of 3.60±0.03 (p = 4×10 −10 ). Students' comments explaining their ratings (n = 56) revealed that teachers provided them with advice, encouragement, and enthusiam (49 responses) "My teacher has been very supportive and has helped us when we didn't understand something as well as encouraging us to taking a more innovative approach." (Student 124, Quirm College for Young Ladies, MUSICS 2020) "If we had a question, teachers were probably not useful. But if we did not know what to do or we were stuck, here teachers were really useful and that was what we needed." (Student 145, Sunnydale High School, MUSICS 2020) as well as arranging regular sessions for students to meet and visits or calls from the university when required (7 responses which is something we don't expect of teachers (cf. Shah and Martinez, 2016; Bennett et al., 2016 , hence why support from the university is also offered. Teachers' (n = 18) responses on a yes/no scale (chosen due to expected small number statistics) of whether they felt able to support their students were also highly positive with only 2 negative responses, a significant majority (p = 0.001 in a two-tailed binomial test). Bear in mind, however, that these responses were in light of the support provided from the university, something which a few teachers referenced in explaining their answers "My own experience with research was handy but I felt that without this the students would still have been sup- "Second year that I ran it I feel more confident" (Teacher 21, Hogwarts, SCREAM 2018) The final theme raised was that for successful participation teachers believed the students needed the external motivation coming from the university rather than having project delivery being solely teacher-driven "Dr Archer was a great external lead to have. If I had been pushing them myself they would have taken it less seriously" (Teacher 17, Sunnydale High School, MUSICS 2018) Therefore, the comments from both students and teachers indicate that teachers alone would likely not have been able to successfully support these research projects in their schools without both the resources and external motivation/mentoring provided as part of the PRiSE framework. We now consider the specific elements of support shown in Figure 1 . From 2019 onwards we investigated participants' thoughts on each of these various aspects offered. Students (n = 68) and teachers (n = 23) were asked to rate the usefulness of these as either 'unimportant', 'helpful', 'essential', or 'unsure'. This was chosen over a 5-point Likert scale due to an expected low number of responses, particularly from teachers. Any unsure or blank responses are neglected yielding 326 (out of a potential 408) student and 156 (out of 161) teacher responses. We divide these responses into negatives ('unimportant') and positives ('helpful' or 'essential'), though we acknowledge some may consider the 'helpful' response as neutral and thus our analysis takes both interpretations into account. The results are displayed in Figure 5 for the individual elements as well as overall results obtained from totalling all responses. Both students and teachers overall rated the elements positivelycoding the responses to values of 1 (negative) to 3 (essential) the overall means were 2.62 ± 0.04 for teachers and 2.23 ± 0.03 for students. The majority of teachers tended to give 'essential' ratings to most aspects and while these majorities are not statistically significant in a two-tailed binomial test, the average value for each element was greater than 2 to high confidence (p < 0.002 in one-sample Wilcoxon signed-rank tests). Students, on the other hand, mostly rated each element as 'helpful' as well as stating slightly more negative responses than teachers, though again all elements' mean scores (apart from the kick-off workshop at 2.15 ± 0.08) were significantly greater than 2 (p < 0.023). While there are some variations in scoring amongst the different support elements, such as students and teachers respectively rating researcher visits and communications (the latter of which includes webinars and ad hoc emails) as the most essential, these differences to each group's overall results are slight and not statistically significant. One interpretation of this might be that most respondents answered unreflectively, ticking the same boxes for each item. However, no students and only 3 teachers gave the same answer in every category. This therefore suggests that all of the elements of support provided as part of PRiSE are almost equally important and necessary. This has been further elaborated on in teacher feedback: Based on the highly positive results from participants, we think there is potential for the PRiSE framework to spread beyond QMUL and be applied to other institutions' own areas of physics (and perhaps even STEM more generally) research. We therefore wanted to assess how it is perceived by those from the university sector with interests and/or expertise in schools engagement. Feedback from our partner organisations seemed promising, for example with the South East Physics Network The respondents, while heavily bought into schools engagement, tended to only undertake one-off activities (as detailed in Appendix C). After presenting the PRiSE framework to them, when asked on a 5-point Likert scale whether they (n = 19) would now consider deeper approaches to outreach / engagement with schools, the results were: Strongly Agree (5), Agree Figure 5 . Usefulness of support provided to teachers (T, n = 23) and students (S, n = 68). Results are divided (black lines and associated error bars) into negative (red) and positive responses, with the latter subdivided (grey lines and error bars) into 'essential' (blue) and 'helpful' (yellow) elements. Error bars denote standard (1σ) Clopper and Pearson (1934) intervals. (9), Neither Agree or Disagree (5), and no responses in the two negative options. Coding these to a 1-5 scale yields a mean of In an open-ended question, participants were also asked to identify the main thing they had taken away from the session. The final theme (5 responses) concerned potential impacts from deeper programmes of schools engagement, beyond the scope of this paper. These results suggest researchers and engagement professionals also see the value in PRiSE's approach and may be receptive to adopting the framework, though evidence of action following these immediate attitudes is really needed. We also acknowledge that this was a rather small survey from a group already highly bought-in to schools engagement, thus results would likely be less positive from a wider and more representative sample of all researchers. These are avenues which could be explored further in the future to gain a better perspective on whether the PRiSE framework could realistically be rolled out further. 'Research in schools' programmes, where school students and their teachers get to experience and interact with cutting-edge science through independent projects over several months, may have some role to play in addressing current issues around participation and equity in science (physics in particular) at university. However, how best to go about delivering such projects for schools remains unclear. This paper evaluates the provision framework of the 'Physics Research in School Environments' (PRiSE) programme. These 6-month-long projects mentored by expert researchers include a suite of activity stages, interventions and resources to to enable a wide range of students, teachers, and schools to be involved. Feedback from participating students and teachers upon completion has been significantly more positive than benchmark results on schools engagement programmes with STEM in general. This is because students and teachers have found the research projects of great interest and have relished the challenge of working differently to their regular school experience. This suggests that 'research in schools' projects are of greater value to schools than more common forms of outreach activities. Since these experiences can uniquely be provided by active researchers compared to other possible STEM engagement providers, we strongly urge universities consider 'research in schools' approaches to engagement. Participants find the numerous elements of supporting interventions and resources provided in PRiSE's framework, uncommon in general with other schemes, as equally valued and necessary for their participation. We therefore recommend that providers of independent research projects offer schools detailed project guides, mentorship from experts (in-person, live-online, and asynchronously), online resources (such as 'how to' articles, examples of previous students' work, and videos), and finally the opportunity to present their work to peers, teachers and family. We note that even with all the support provided there is some attrition within the programme, which is to be expected for any protracted engagement programme. While this has not been explored here, it is assessed in a companion paper (M. O. Archer, 2020) . Currently though we have little data on the experience of students and teachers that have dropped out of the programme, which is something that could be explored in the future. We have also gathered data on the impact of the PRiSE initiative, which is assessed elsewhere (M. O. ). The PRiSE model attempts to make efficient use of researchers' time, enabling more schools to be supported per institution than other current formats. We have presented data from independent researchers and public engagement professionals showing they seem receptive to the PRiSE framework. Indeed, it is already slowly beginning to spread to other institutions. This potential expansion might allow an assessment of how generally applicable the framework is outside of its current London location and what other affordances might be required in these contexts. While PRiSE has so far only concerned areas of physics research, 'research in schools' in general already span all the sciences (e.g. Bennett et al., 2016 IRIS, 2020) . We therefore see A-Level 17-18 Table A1 . Summary of the stages of secondary education in the English system. no reason why PRiSE's approach could not also be broadened to other STEM areas, particularly areas of research based in data and/or analysis. We therefore encourage researchers, and the public engagement professionals who facilitate their activity, to consider adopting this way of working and hope this paper can inform this practice. In such cases, it is recommended that PRiSE projects be embedded as core schools engagement activity within research groups. We would be happy to support groups in developing, delivering, and evaluating pilot PRiSE projects around their own research, thereby making use of the learning that has arisen from the programe over the last 6 years. To those unfamiliar with the UK/English education system system, we provide some further notes here. The curriculum is broken down into Key Stages of duration 2-4 years, with those for secondary schools displayed in Table A1 . The final two Key Stages culminate in GCSE and A-Level examinations respectively, with the latter being optional as education post-16 is not compulsory. Our recommendations to teachers about which year groups should be involved with PRiSE are also highlighted in the table. Here we list the questions posed in questionnaires that are considered within this paper, giving details of what phrasing was used, how participants could respond, and which years the question was posed. Follow-on questions are indicated by indentation and a down-right arrow ( ). Students' responded to the following: For context on these participants, Figure C1 shows the distribution of the types of activity they undertake where they could select from: -Stall/stand: drop-in activities for schools at STEM or careers fairs -Talk: a typically one/two lesson slot featuring a predominantly one-way interaction -Workshop: a typically one/two lesson slot with mostly two-way interaction and often hands-on activities for students -Masterclass/taster: half-day or day-long activities which may be comprised of talks and/or workshops -Summer school: several-day to week-long activities often involving some project work as well as talks and/or workshops -Extended programme: multiple interventions with the same group of students over a protracted period of time Unsurprisingly, one-off activities such as talks and workshops scored the highest whereas more the protracted engagements, summer schools and extended programmes, were significantly (p < 0.0019) less common. The attendees were also asked what they hoped to achieve (i.e. the aims or intended impacts) through their school engagements via an open question. Performing a thematic analysis of the qualitative results, it was possible to categorise the majority of answers into the following: -Change school students' aspirations (9 people), with the word "inspire" often used -Enhance students' awareness or understanding of STEM (6 people), often in the context of primary research -Tackle societal biases in STEM (4 people), most often gender Note that some responses covered more than one of these aims. Other stated motivations outside of these themes included "access to a student population for [research] studies", to "build relationships", and to deliver "meaningful content". The three themes are in general agreement with those determined by in a larger survey of UK physicists. Data availability. Data supporting the findings of this study that is not already contained within the article or derived from listed public domain resources are available on request from the corresponding author. This data is not publicly available due to ethical restrictions based on the nature of this work. A Error bars denote the standard (1σ) Clopper and Pearson (1934) interval. Author contributions. MOA conceived the programme and its evaluation, performed the analysis, and wrote the paper. JDW and CT contributed towards the analysis, validation, and writing. OK contributed to the writing. Competing interests. The authors declare that they have no conflict of interest. There is no "magic bullet" to increasing physics (or more broadly STEM) uptake and diversity at higher education -multiple different approaches are needed with each addressing different stages of young people's educational journey as well as their key influencers and wider learning ecology in relevant ways (e.g. . Furthermore, research has shown that young people's aspirations are incredibly difficult to influence (L. Archer et al., 2013 Archer et al., , 2014 with standard one-off interventions, or even short-series, showing no real changes, highlighting the need for more extended and in-depth programmes for significant lasting impact (see the review of M. O. Archer et al., 2021, and references therein) . Given this complexity, we contructed the aims of the PRiSE programme through a theory of change (TOC; Sullivan and Stewart, 2006) . This approach is recommended by several organisations and has been applied to other STEM outreach/engagement programmes (e.g. . A TOC is designed to rationalise the intended outcomes and impacts of an initiative by outlining causal links. The process of creating a TOC works backwards, starting at the intended ultimate impact and mapping the intermediate outcomes (both short-, medium-, and long-term) that are thought to be required to enable that goal. The resulting outcomes pathway (which may require iterating several times) should be accompanied by the rationale for why specific connections exist between different outcomes in the theory narrative along with any underlying assumptions and potential barriers. We note that by no means are all the causal links presented in a TOC guaranteed to occur, however, by considering them during development programmes are more likely to realise them. Figure S1 displays the TOC for PRiSE, which covers participating students (blue) as well as their parents/carers (yellow) and their teachers and school environment (both red). The intended impact of PRiSE is to contribute towards the increased uptake and diversity of physics at higher education. By serving students near the end of their school-based educational journey, somewhat necessitated by the content and style of open-ended 'research in schools' projects, the programme acts to support students' existing identity with science in general and enhance, or at least maintain, physics aspirations to help transform these into degree subject destinations -a known issue at this stage. Therefore, an assumption feeding into this TOC is that PRiSE students will have an existing positive association with science beforehand and we make no claim that the PRiSE approach would be effective for students who are generally uninterested or unengaged with STEM from the outset. Students' interest or enjoyment in the subject as well as its perceived usefulness in a career are key factors affecting degree choices (DeWitt et al., 2019), with students (particularly girls) often thinking physics is less useful or relevant . Additionally, the stereotypes and school-based practices associated with physics make many, even highly-able and interested students, at this age conclude it is 'not for me' (L. Archer et al., 2020a) . PRiSE attempts to be a factor in addressing all of these factors in some way. By interacting first-hand with "real physics" through the projects and working with active researchers, students (especially those from under-represented groups) should feel included and have their interest in physics enhanced or at least sustained . By experiencing success at 'being' a scientist and meeting similar students from other schools, it is hoped their confidence will be boosted leading to a feeling that physics is indeed for 'people like me' . Furthermore, through working in new ways students should develop numerous transferable skills which might help them recognise the usefulness of the subject (Soh et al., 2010) . Teachers are much stronger influences on students' aspirations than university staff/students could ever be (L. Archer et al., 2013 Archer et al., , 2020b . Experience from physics outreach officers (e.g. through discussions via the South East Physics Network's Outreach and Public Engagement and Ogden Trust's Outreach Officer programmes) has shown that most teachers are more interested in activities for their students from universities rather than continuing professional development opportunities, which they may seek elsewhere. Therefore, opportunities for teachers' development are integrated within the programme rather than being a separate offering to schools. While the number of students working on PRiSE may be relatively small, by influencing teachers through our sustained programme, the aim is that the impacts of PRiSE can be felt much more widely. Indeed, our hope is to affect the environments within the diverse range of schools we work with on the programme so that they are places that are able to support and nurture the science capital (L. Archer et al., 2013 Archer et al., , 2020b ) of all their students, thereby also contributing to our goal of increased uptake and diversity of physics Figure S1 : The Theory of Change for PRiSE concerning school students (blue), their parents/carers (yellow), and their teachers and school environment (both red). Shading indicates the timeframe of outcomes going from lighter (short-and medium-term) to darker (long-term and impact) with time running vertically from top to bottom. Arrows denote theorised causal links between outcomes, accompanied by references where possible, with dotted arrows indicating links that might feed back into successive years of the programme. (2014) recommends to help achieve such an environment that schools should raise the overall profile of science in school, endeavour to build long-term relationships between pupils and role models, and ensure all teachers are aware of the influence they can have on children's future careers. We aim to further all these points by PRiSE providing more collaborative working opportunities between teachers and students, exciting success stories that teachers and students can share across their schools, and the gateway to building longer-term relationships between schools and the university. It is likely the teachers that sign up for such a protracted programme and/or benefit the most from it are relatively engaged generally. How to enhance the practice of disengaged teachers is a challenge beyond the scope of PRiSE. Finally, another major influence on young people's aspirations is family, particularly parents or carers (e.g. Clemence et al., 2013) . Parental engagement is notoriously difficult within school-based programmes in general (see the review of M. O. Archer et al., 2021) , so we simply aim to include parents/carers to celebrate in students' project work at the end of the programme. These parents/carers are likely interested in their children's education. It is hoped that by witnessing their child's successes and development through physics, they will be more positive, and thus supportive, towards physics aspirations going forward, reinforcing the impacts of the programme. This TOC for PRiSE was used in motivating the provision within the framework. Evaluating the impact of PRiSE is beyond the scope of this paper / supplementary material and is explored in a companion paper (M. O. . We stress that this TOC for PRiSE does not exist in isolation and other TOCs which focus on different stages of and aspects to a young person's learning ecology are required to improve the overall issue of uptake and diversity in physics. For example, Davenport et al. (2020) present a potential STEM outreach TOC designed towards primary and lower secondary school students, with outcomes focused more on STEM careers. Queen Mary University of London's (QMUL) physics research areas concern astronomy (space science, planetary physics, and cosmology), particle physics (the Standard Model and beyond via particle colliders and neutrino observatories), condensed matter physics (e.g. material structure, organic semiconductors, and applications thereof), and theoretical physics (e.g. string theory, and scattering amplitudes). Of these, it was decided to initially base PRiSE around the space and planetary sciences as well as particle physics. These exciting topics are thought to inspire awe in the public due to the "big" questions they address and the senses of scale and wonder beyond our everyday experience (cf. Madsen and West, 2003; IPPOG, 2020) . However, exactly how this science is conducted is not often well understood outside of academia, particularly at school-level due to the lack of research methods within current science teaching (e.g. . Currently four projects have been developed for the PRiSE programme at QMUL, which we briefly summarise here indicating key project personnel referring to the roles mentioned in the main article (note the outreach officer role for the entire programme was performed by the first author). • Scintillator Cosmic Ray Experiments into Atmospheric Muons (SCREAM, 2014 (SCREAM, -2020 was adapted by the outreach officer from a dissertation project designed for undergraduates using a scintillator -photomultiplier tube muon detector (Coan and Ye, 2016; TeachSpin, 2016) fundamentally similar to those found in current neutrino experiments such as SNO+, where cosmic ray muons serve as an important background source that can be used for calibration (Alves et al., 2015) . Students calibrate their borrowed detectors and collect counts of comic ray muons and muon decays. Initially they use this data to perform a measurement of muons' mean lifetime (using both software that comes with the detector and programmes we have created especially) before progressing to a wide variety of potential topics on these cosmic rays such as their angular distributions or dependence on atmospheric/solar conditions. Since detector usage and particle physics are part of most A-Level physics syllabuses, it complements their studies. At present QMUL only has four of these detectors as they are expensive (around £5,000), which limits the number of schools we can work with each year. As this project has been especially popular with teachers when signing up, we have made it open only to schools that have successfully undertaken a different project with us previously, also limiting how long they can borrow a detector to a maximum of 4 years. • Magnetospheric Undulations Sonified Incorporating Citizen Scientists (MUSICS, 2015 (MUSICS, -2020 was created especially for PRiSE by the project lead. Geostationary satellite data of the "sounds of space", ultra-low frequency fluid plasma waves in Earth's magnetosphere, have been made audible. Students are given this data on preloaded USB flash drives and explore it through the act of listening (we also provide them with earphones). Audacity audio software (https://www.audacityteam.org/) is used to analyse any events identified, which can then be logged in a specially created spreadsheet which performs some routine calculations. We stress that students do not have to focus on the space plasma physics aspects, which will largely be completely unfamiliar, but rather just the waves topics that they cover in class both at GCSE and A-Level. While students are given guidance on how to listen to and analyse the waves, we do not prescribe to them exactly what to listen out for as we are instead interested in what they pick out themselves. This approach has already led to novel and unexpected scientific results on the resonances present in Earth's magnetosphere during the recovery phase of geomagnetic storms (M. O. . Based on these results, an optional 'solar storms campaign' was created for 2019/20, providing more concrete prescribed instructions to build up a dataset of similar events followed by suggestions of unanswered questions about these resonances that students could investigate. While a few schools followed this route initially, they all eventually decided to go their own way with it. Key personnel: Dr Martin Archer (project lead and researcher, 2015-2020) • Planet Hunting with Python (PHwP, 2016-2020) was initially developed by a post-viva PhD student, Dr Gavin Coleman, through a one-day-per-week buyout over three months (funded by a grant obtained by the outreach officer) and has subsequently been modified by the project lead each year. The project aims to address the UK coding agenda (the UK government's desire for more young people to develop computer programming skills, e.g. Department for Education and Gibb, 2018) by applying Python computer programming to data from NASA's Kepler (Jenkins et al., 2010) and later TESS (Ricker et al., 2015) missions, whereby students write programmes to detect exoplanet transits. Transit photometry, where an exoplanet blocks some of the star's light, can be fairly easily understood by school students in terms of geometry and the equations from A-Level physics. The students try to independently implement each step laid out in their guide (period detection, phase folding, model fitting, and parameter estimation) applied to specially selected star systems chosen for their relative simplicity. Example code is given to teachers. While extension activities are suggested, so far very few students have progressed beyond the prescribed activities within a single year, though some students have returned for a second year making further progress. • ATLAS Open Data (2017-2020) was adapted by the outreach officer for PRiSE from a public resource produced by CERN aimed at undergraduates (ATLAS Experiment, 2017 ). An undergraduate summer student, under the instruction of the outreach officer, produced a guide so that school students could build up to the documentation provided online by CERN. At kick-off workshops students play a loaded dice game, developed by the outreach officer and freely-available online as a resource (PRiSE, 2020), which serves as an analogy for why particle physicists need to use statistical methods and big data in discovering new particles such as the Higgs boson (ATLAS Collaboration, 2012) . This leads into the main activity, using CERN's interactive histograms to see how performing cuts on the data increase/decrease the significance of the desired signal, i.e. the Higgs, compared to the backgrounds. While the CERN guides provide extensions by using their statistical software (ROOT) for more detailed analysis, this has been beyond almost all PRiSE students thus far, with most groups simply investigating the underlying physics behind their chosen cuts to justify them. Of the current projects at QMUL, only MUSICS at present has the scope to lead to novel publishable scientific research, which it already has done. The Kepler dataset has largely been mined of the clearest exoplanets, often now requiring advanced machine learning techniques for new discoveries (e.g. Shallue and Vanderburg, 2018) which are currently also being implemented on TESS. The other two projects have limitations based on the equipment (Coan and Ye, 2016; TeachSpin, 2016) and amount of data used (ATLAS Open Data's first release contained only a fifth of the data used in the Higgs boson's discovery, ATLAS Collaboration, 2012, however, more data was released in 2020). While this is not perhaps ideal, it is due to the realities of pressures on academic staff time limiting their ability to develop outreach projects from scratch (e.g. . M. O. recommended that the development and delivery of 'research in schools' projects should be distributed within each research group sharing the load out amongst academics, post-docs, and PhD students. This would allow more schools to participate without overburdening individual researchers. However, this research group buy-in has proven difficult to achieve at Queen Mary due to an overall poor culture towards public engagement and outreach in their physics department, a fairly common barrier to public engagement in general (Burchell et al., 2017) . Responsibilities have thus largely been falling to only a few people per PRiSE project which has limited the number of schools which could be involved each year. Nonetheless, there have been some positive steps in the last year with project leads, along with the outreach officer, being able to convince a few early career researchers to help with delivery, which may indicate the department slowly moving towards a more embedded approach. Institutions with a more positive culture of public engagement and outreach likely could support even more schools with research projects than has been possible at QMUL. This section is designed to provide sufficient additional detail to enable practitioners to fully understand how the PRiSE framework has been implemented, with the aim of informing their schools engagement practice. • Prescribed work: This stage involves following a set of instructions to undertake an experiment/activity designed to cover most aspects of an investigation and to build their confidence in the project topic. Students are still required to problem solve throughout these stages and we purposely do not provide them with all the answers to prompt this, though teachers are given guidance in their resources to support student efforts. The stage is designed to enable students and teachers to initially be able to access and interact with the research, enhance teachers' knowledge, and hopefully make underrepresented groups feel included in physics (cf. Figure S1 ). • Independent project: This continues in a similar way to the prescribed work except now independently motivated. In visits and webinars the question has been raised by students whether there is a risk that they investigate the same thing as another group at a different school, though given the broad scope of most of the PRiSE projects so far this has rarely occurred. Potential research questions are suggested in the guides provided, with further advice for teachers on these being given in their versions, and students' ideas are discussed during visits and/or webinars. During this stage we aim that students' interest in physics is increased or sustained through pursuing their own research questions and that they develop physics-related transferable skills (cf. Figure S1 ). We encourage teachers to work with their students throughout this stage, e.g. holding regular meetings, which may help enhance their perceptions of students' ability (cf. Figure S1 ). • Writing up: Near the end of the project students produce either a scientific poster or talk to be presented at our annual conference. Guidance on how to approach these is provided online as well as during visits and webinars. We have found that with the researcher support provided that all student groups that persist with project work to this stage are able to produce a poster or talk, thereby experiencing success at 'being' a scientist (cf. Figure S1 ). • Assignment: Using existing teacher networks, such as the Institute of Physics' Stimulating Physics Network (Hartley, 2011) and the Ogden Trust School Partnerships (Ogden Trust, 2020) in the UK, not only allows us access to schools from lower socio-economic areas given the networks' focus but also act somewhat like a word-of-mouth recommendation. We have found these networks to be more successful at attracting new schools to the programme than our existing schools events mailing lists. Teachers fill out an online form providing school and contact details, project preferences, and the estimated number of students who will be involved. Previously participating schools have to reapply each year. Once applications are in we assess the capacity of the programme (taking into account data about the schools) and inform teachers before the summer break whether their school has been allocated a project or not. Most schools are assigned only one project, which both helps with logistics and makes it easier for teachers to manage, and where possible we take into account their stated preferences though this is not always possible given the researchers' workloads. • Kick-off: Due to constraints on time for academic members of staff leading projects, the kickoffs are typically an evening event on (university) campus. In some cases where schools could not attend the event we have repeated it at their school. Projects led by non-academic staff are usually hosted in-school, sometimes within a normal lesson or at lunchtime/after-school depending on the teacher. The events start with a 20-30 minute introductory talk by the project lead concerning the underlying physics and research topic, leading up to an overview of what the project is about, which is presented to students as an opportunity that they can take advantage of if they wish. The outreach officer then discusses the differences between learning styles in the research project compared to their regular classroom experience, how the project will work, the support available, and how to go about obtaining this. The event ends with a hands-on workshop for at least 20 minutes usually run by the outreach officer and facilitated by researchers (though not always the project lead). This workshop, which teachers as well as students are actively encouraged to participate in, typically forms either the early part of or a lead into the initial stage of the project work. Experts are on hand to assist with any questions or initial troubles, with the aim of getting the students and teachers to a place where they can continue this work without too much extra help for the next month or so. Students and teachers are given all the project's resources so they can begin/continue their project work from this point on at their school. The outreach officer will also have an informal chat with (particularly new) teachers concerning how to go about undertaking and supervising the project, answering any questions or concerns they may have with either the science, activities, or project management. • Visit: These school visits by researchers are often administered through a rolling Doodle (http: //www.doodle.com) poll where teachers can sign up to a session given the researcher's availability. Schools taking advantage of this stage typically receive only one visit, though if further demand is communicated we try to accommodate one (or occasionally two) additional visits. The visits typically last around an hour and occur around the stage where groups have finished the prescribed activities and are thinking about or are in the early stages of undertaking their independent research. They are very much student-driven meetings, where the researcher asks the groups of students to show what they have done, probes their understanding of this, puts their work into the context of current research, provides answers to any questions the students have, and gives advice on what the direction and next steps with their specific project ideas might entail while bearing in mind what methods/results may be achievable within the timeframe of the project. Only active researchers have the necessary expertise to draw upon in offering such bespoke, tailored guidance to students and teachers working on projects in their research area. With one project (ATLAS) and for a few schools on other projects it has not been possible to have in-person visits for logistical reasons, however, similar interactions were done via specifically arranged Skype calls to the schools in these cases. Teachers are encouraged to participate in these meetings and additionally further informal chats (similar to those taking place during the kick-off) between the researcher and teacher occur to help with their project supervision and continuing professional development. These researcher interactions with students and teachers are aimed at not only supporting project work within the schools, but further enhancing teachers' knowledge and students' sense of inclusion in physics (cf. Figure S1 ). • Webinars: One project (MUSICS) has experimented with additional support to schools through monthly drop-in webinars between November-February, providing further opportunities for students and teachers to ask questions of the researcher and get advice on how to progress with their project work in a similar manner to the visits. This was first trialled in 2018/19 through a Google Hangout simultaneously streamed on YouTube, however, this option was later discontinued so a solution using a Skype group call also broadcast to YouTube (via the NDI® feature and using Open Broadcast Software, https://obsproject.com/) was implemented in 2019/20. The YouTube streams are unlisted so that only project students with a link can access them, making the webinars a safe space for them to discuss the project. While almost all students and teachers preferred to simply join the YouTube stream and contribute via its live chat facility, the rationale behind incorporating the Hangout/Skype option was so that participants could directly talk to the researcher and/or show their work. In 2018/19 the webinars were organised in a somewhat ad hoc manner and due to technical limitations the only way of communicating the links to join was via an email immediately before the webinar. With the move to Skype we were able to create a stable hyperlink to join the group as well as being able to embed the YouTube events in advance on a password-protected webpage, both of which allow for easier access to the webinars. In terms of organisation, at the beginning of the 2019/20 academic year we sent out an online form asking teachers to identify when might be the best times for webinars. While the response rate for this was low (only four), we used this to set a regular monthly schedule (in this case the first Monday of each month at 4-5pm) which was communicated to teachers far in advance. All these changes considerably increased the uptake of webinars: 10 out of 14 schools participated in at least one webinar in 2019/20 compared to only 2 out of 14 the previous year. The rationale behind webinars is that they further support the projects and their aims in a way that makes efficient use of researchers' time. • Ad hoc: We explain at the kick-off meetings that when students get stuck at any point (which they invariably will do due to the nature of research) they should try to first tackle this themself, before discussing in their groups, and then raising with their teacher. In general, teachers act as the primary contact to students offering encouragement and any support or advice they can. If students' questions go beyond what their teacher can answer and is not covered by the teacher guides we provide, the teachers should get in touch through the outreach officer. This is done not only for logistical and safeguarding reasons, but also provides further opportunities for university-teacher dialogues that can contribute to their professional development. Some teachers, however, instruct their students to email directly. Not all schools require this option of further support and we have not yet been overloaded with additional questions. Only in a few cases has the outreach officer not been able to directly answer the question, subsequently passing it on to a relevant researcher to answer, though in general this would depend on the background and experience of the outreach officer. • Comments: These are currently given by the outreach officer, though in general this would depend on their background/experience and could instead be done by the relevant researcher role(s). Teachers (or students directly) email their work to the outreach officer and receive annotated versions back the week before the final deadline, allowing the students at least a few days to implement any changes. • Conference: The evening is primarily based around oral and poster presentations by the students. Food and drink are provided during the poster session and we also put on various physics demonstrations. At the end of the evening all student groups are congratulated and given a thank you letter, with a select number of groups highlighted by researcher judges also receiving prizes in the form of various science gadgets (some prize winners have also had the opportunity to present their research at a national student conference hosted by the Royal Society). As of 2019 we limited the number of talk slots available to four in total, both for time and so each topic can be covered. Schools are only able to solicit one talk (where desired) by providing a title and abstract in advance (early March), with the decisions of who will present being made that same week. There are currently no limits on the number of posters a school can enter into the conference. The event further enables students to experience success at 'being' a scientist as well as getting to know other people, outside of their school from a variety of backgrounds, with interests in physics. It is hoped that this might lead to increased confidence, seeing themselves as equals in physics, and ultimately that physics is something for 'people like me' (cf. Figure S1 ). The conference provides parents/carers in attendance the opportunity to have a positive experience with physics and witness their child's interest and ability in the subject, which could in the long-term result in their supporting and encouraging their child's physics aspirations (cf. Figure S1 ). Finally, from teachers' perspectives the conference lends an avenue for them to support and encourage their students as well as share successes across their school (cf. Figure S1 ). The outreach officer typically sends updates and reminders about these possible interventions to all teachers involved fortnightly throughout the programme. This frequency attempts to tackle teachers' generally low levels of response to a single email (Sousa-Silva et al., 2018, also reported teachers' generally poor communication through ORBYTS). In addition to email communications, we also set up a password-protected teacher area on our website in 2019 which always contains the latest information on the programme. It appears from website analytics that teachers have used this to some extent (there were 76 unique page views amongst the 38 teachers involved that year), though we are unsure which teachers these were and how often they visited the page throughout the programme. For any on campus events, schools travel to Queen Mary using London's extensive public transport network and we are unable to offer schools compensation for travel expenses due to limited funding. Outside of London or other well-connected cities this travel may be a greater barrier to participation than in our case so may need further consideration. • Project poster/flyer: At the start of the academic year we send posters/flyers to teachers to help attract attention to the project within their school. Some teachers had been making their own such adverts previously, which motivated us to create these. • Project guide: Our guides are presented in the style of an academic paper. Printed copies of these are given out at the kick-off event and electronic version can also be found on the project's page on our website. These serve as an introduction, providing enough information for students to start working on their project and be something they can refer back to throughout. However, the guide is not intended to be exhaustive (that would be impossible given the open-ended nature of the projects) and students are made aware that we expect them to read additional materials as they progress. Throughout the guides there are exercises and discussion questions for students to consider, designed to help them think more critically about their project work. Teachers' guides include answers to the exercises, hints and tips about different methods, common pitfalls that students make etc. and these are distributed to teachers at the kick-off, via email, and also stored on the password-protected teacher webpage. These project guides have been updated every single year based on feedback and professional experience, which is straightforward to do given the chosen style compared to say a more illustrated glossy guide that would require a professional designer each time. • Project webpage: Including previous work on these had been suggested by students and teachers in feedback for a few years as something that would be helpful. Some projects have also produced videos which provide further information on the science / research area or demonstrate how to use tools provided for the project, which have been included on these pages. • 'How to' guide: These articles have been produced by the outreach officer. The most used of these are the guides on producing scientific talks and posters that we point students and teachers to ahead of the conference. While articles in this section designed for teachers have also been planned, which would highlight elements of good practice that have emerged from other teachers on how to successfully integrate and nurture project work within their schools outside of the support offered by researchers, we have not had sufficient time or detailed input from teachers to be able to co-create these yet. Examples of all these resources are given in section S4. The operational costs of PRiSE's interventions have largely been covered by QMUL's departmental physics outreach budget. Running the kick-off events and conferences is comparable in cost (∼£3,000 p.a. at PRiSE's current scale) to many summer school programmes aimed at high school students which universities offer and often absorb the cost of. However, as noted in the review of M. O. Archer et al. (2021) , summer schools have severely restricted places (∼10-30 students) and limitations on impacts upon students. The PRiSE model thus potentially offers much greater value for money. The programme management falls within the scope of the funded outreach officer post, a role which many university departments employ (e.g. Ogden Trust, 2020) . Whether to pay early career researchers or assign workload allocations to academics in delivering projects would realistically be down to the policy/strategy of the department or institution. In the case of Queen Mary, we have opted to only offer pay to PhD student researchers (sourced from the department's outreach budget) to try to increase uptake of engagement. Workload allocation in outreach / public engagement for academics is dedicated only to roles aimed at embedding engagement throughout the department, e.g. champions in research groups who help recruit their colleagues to contribute to PRiSE. Delivery of engagement is considered an expectation of the role of an academic, though significant contributions may be used as criteria for promotions. Grants have been obtained to assist with project development, as detailed in section S2. If institutional funds are not available for the delivery of a 'research in schools' programme, we would recommend either costing these into research grants as impact-related costs or employing the ORBYTS funding model of charging independent schools to allow both them and a few less-resourced schools to participate . Finally we give examples of the resources within the PRiSE framework. We display the posters (A3 size) provided to teachers to advertise projects to their students in Figure S2 . Leaflets (A5) size used similar designs and content. We also show a project webpage ( Figure S3 ) and teacher guide ( Figure S4 ), in both case for the MUSICS project. Student guides are identical but do not include the red text, which is for teachers only. Finally, a 'how to' style online guide for students is given in Figure S5 , specifically the one on creating research posters. The magnetosphere is the space environment formed due to the interplay of the solar wind with Earth's magnetic field. It is highly dynamic and rife with analogues to sound waves in space, fluid plasma waves which occupy the ultra-low frequency (ULF) range (< 1 Hz). Many questions concerning ULF waves still remain, such as how often and at what frequencies do various resonances of the magnetosphere occur. This research project allows you to study these waves by using perhaps the best pattern recognition system that we know of, the human auditory system. By listening to satellite data and using audio software you will explore the waves present in near-Earth space and undertake your own research project in groups, the findings of which could contribute to improving our understanding of our protective magnetosphere. Earth's magnetosphere is the space environment around the Earth formed by the interaction of the solar wind (plasma continually streaming away from the Sun at supersonic speeds) with the Earth's magnetic field. The solar wind compresses this magnetic field on the dayside, confining it to typically within 10 times the Earth's radius (R E ), whereas it sweeps back the magnetic field lines on the nightside to some unknown length, possibly up to 1000 R E . In turn the solar wind is itself slowed and deflected around the magnetic barrier (by a shock wave, the bow shock, since the flow is supersonic). Figure 1 illustrates some of the basic structure of the magnetosphere. The magnetosphere is far from static, for example the solar wind pressure and magnetic field continually change causing the size and shape of the magnetosphere to adjust accordingly. These dynamics of the magnetosphere manifest in many ways, including a number of different plasma waves. In the ultra-low frequency (ULF) range, defined as waves/oscillations of frequency <1 Hz, the plasma can be treated as a single fluid in much the same way as air or water. This means there are two fundamental types plasma waves: • Magnetosonic waves: the equivalent of sound waves in plasmas, however, unlike a gas in which sound is driven by thermal pressure only, plasmas also exhibit magnetic pressures too hence the name of these waves. These waves can have both longitudinal and transverse components and can transport energy across magnetic field lines. • Alfvén waves: a sister wave not possible in gases but similar to some seismic waves in solids. These are analagous to waves on a string since magnetic field lines in a plasma exhibit a form of tension. Their wave perturbations are perpendicular to the background magnetic field i.e. transverse, so they do not increase the magnetic field strength, and they transport energy along the direction of magnetic field lines. Inside the magnetosphere both these waves have approximately the same wave speed: the Alfvén where B is the magnetic field strength, ρ is the plasma mass density and µ 0 = 1.2566 × 10 −6 m kg s −2 A −2 is the permeability of free space/magnetic constant. Therefore the wave speed (and thus the frequency of for example any Figure S4 : Example PRiSE project teacher guide. resonances) depends on both the magnetic field and the amount of plasma present, both of which change with location and time throughout the magnetosphere in ways that we still don't fully understand yet. Exercise: At geostationary orbit magnetic field strengths of ∼90 nT and proton number densities of ∼10 cm −3 are typical. What is the wave speed under these conditions? First don't forget to convert the units into SI i.e. 90 nT = 90 × 10 −9 T and 10 cm −3 = 10 × 10 6 m −3 . Also don't forget to include the mass of the proton when calculating the mass density v A = B/ √ µ 0 ρ = 90 × 10 −9 / 1.2566 × 10 −6 × 10 × 10 6 × 1.67 × 10 −27 = 620 km s −1 Note we tend to use km rather than m for distance due to the generally large scales involved. It is worth mentioning that the majority of the dynamic solar wind -magnetosphere interaction is invisible, bar phenomena such as the aurora. Therefore much of our understanding of magnetospheric processes come from spacecraft/satellites in orbit around the Earth which can directly measure the particles and fields. There are still many aspects about these waves we do not know. For example, the variability in the frequency of different types of magnetospheric resonances (such as those shown in Figure 2 ) are not well understood. You will therefore be investigating various aspects of ULF waves in the magnetosphere through the use of spacecraft observations at geostationary orbit. Exercise: Field lines near the dayside magnetopause are typically about 16 R E long, where 1 R E = 6378.1 km is the radius of the Earth. Using your Alfvén speed from earlier and assuming a constant wave speed over the entire field line, estimate the fundamental frequency of standing Alfvén waves on these field lines (like standing waves on a stringed instrument) as illustrated in Figure 2 . The assumption of a constant wave speed is, however, not really valid in the magnetosphere since of course the magnetic field strength gets much larger at the poles meaning much larger wave speeds. A way around this is to use the time-of-flight technique whereby the frequency of the wave is found by integrating the amount of time it takes a wave to travel infinitesimally small segments of the field-line i.e. the period of the standing wave is This is beyond the scope of the students' projects though. Note that the variability in these fundamental frequencies of field-line resonances are thought to be some 40-80%, hence the still active research in this area. Exercise: Standing magnetosonic waves can also form, as shown in green in Figure 2 . Assuming a typical distance between boundaries of 6 R E and that the outer boundary is open (an anti-node like in some wind instruments) whereas the inner boundary is fixed (a node), estimate the fundamental frequency of these waves again assuming constant speed. Variability in frequency of these resonances are thought to be around 28-72% but is not clear how often they occur. The Geostationary Operational Environmental Satellites (GOES) are a series of spacecraft in geostationary orbit above North America. They are equipped with Space Environment Monitoring Subsystems (SEMS), which include a magnetometer for measuring changes to the magnetospheric magnetic field, useful for both research purposes and in monitoring/forecasting space weather -which concerns how phenomena from space can affect our everyday lives, such as disrupting our technology. An example of the GOES spacecraft which were available in 2008 is given in Table 1 , listing their location in longitude as well as how to calculate their local time (LT). Local time essentially measures position relative to the Sun (think about why we have time zones for instance). Therefore, a local time of 12h/noon means the spacecraft is directly between the Sun and the Earth; whereas a local time of 00h/midnight means the spacecraft is behind the Earth compared to the Sun. See Figure 3 for an illustration. In geostationary orbit this is a very easy quantity to calculate as the spacecraft orbit at the same rate as the Earth's rotation, so there is a direct link between Universal Time (the standard time used in science, a modern continuation of Greenwich Mean Time) and the spacecraft's Local Time. GOES magnetometer data can be used to research ULF waves in Earth's magnetosphere since the magnetic field moves with the plasma. In this project you will be undertaking such a study using the novel approach of actually listening to these waves. This is because, unlike many automated computer algorithms, the human auditory system is perhaps the best pattern recognition system that we know. In order to make the f real =0.5-244 mHz waves audible to human ears though, the data has had to be rescaled in time and thus also frequency where F s = 44, 100 Hz is the sampling frequency of the audio file and ∆t real = 2.048 s is the time resolution of the magnetometer data used. This rescaling converts an entire year of magnetic field measurements into an audio file less than 6 min long. Exercise: Calculate how long an entire day in is in the audio files. i.e. students should quote times as accurately as possible here. Clearly one decimal place is not nearly enough, though two may be sufficient depending on the circumstances e.g. long-lived waves. Note we must keep the local time within the range 0-24 h, just like with hours during the day. Since Local Time is a measure of spatial position relative to the Sun though, we don't have to worry about altering the date as one would when working out the date and time for a given time zone, compared to GMT (or equivalently UT). You can use the provided spreadsheet to automatically do these conversions from now on. The filename of your audio files are in the format: g10 2008 Ball 10nT diff.ogg • g10: which spacecraft the data is from • 2008: the year the data is from • B : magnetic field data in the following co-ordinates pol = poloidal component which points radially outwards from Earth tor = toroidal component which points eastwards com = compressional which points along the magnetic field all = a combination of all three with pol in the left channel, tor in the right and com shared between both. This is good for initially listening to the data. • 10nT: the data has been divided by this amplitude factor to give dimensionaless waveform units between -1 and 1 • diff: if present the data has been differenced in time to make spectograms clearer You will be using an audio editing package, Audacity, to listen to and analyse the magnetic field data provided. If this software is not installed on your computer, you can download a portable version at http://portableapps.com/apps/music video/audacity portable. Audacity allows you to look at audio either as a waveform or as a spectrogram (a visual representation of the spectrum of frequencies in the audio as they vary with time). For the latter, you should (at least initially) use the log(f) spectrogram view since this is closer to how we interpret sounds ourselves and also allows you to clearly see the full range of frequencies from low to high. You may need to change some of the Preferences (in the Edit menu of Audacity) to show the ULF waves more clearly e.g. Window Size=1024, Maximum Frequency=20000 Hz, Gain=0 dB, Range=60 dB. Note that the differenced waves (i.e. those with "diff" in the filename) will show the clearest spectrograms, whereas the original ones (i.e. without "diff") will show the clearest waveforms. It may therefore be beneficial in your analysis to import both of these tracks into Audacity and mute one of them, as displayed in Figure 4 . In your analysis you may wish to use a number of Audacity's tools and effects, for example: • Analyze > Contrast: This can be used to measure the root mean squared (RMS) of the selected audio, a measure of the overall volume/amplitude. • Analyze > Plot spectrum: Quantifies the amount of signal at each frequency for the selected audio, which can be used to find any clear peaks at specific frequencies. • Effects > Spectral edit multitool: By making a selection in frequency and time in spectrogram view, you can filter out unwanted signals. This may be useful if multiple signals at different frequency ranges are present. If this isn't available, you can do the same using low and high pass filters. • Effects > Noise Reduction: By providing a sample of noise or unwanted signals, these are reduced thereby making other signals more prominant. You should watch the 'How To' guides online and read the Audacity Manual (see section 5) for more details on all these tools/effects and others. Be careful not to overwrite your audio file with any changes you may make to it in the analysis process. You may wish to add labels/markers for any events/sounds you find. This can be done by pressing Ctrl+M to add one at the playback position, i.e. when you're listening to the audio, or Ctrl+B to add a marker to the selected audio. Note that you can add text to your markers as a description. You will be conducting independent research into magnetospheric ULF waves and oscillations. As a first step you should simply listen to some of the sonified magnetic field data to get accustomed to what it sounds like and how to use Audacity. So pick a year and spacecraft and listen to one of the files to start with. Any sound event you pick out to investigate should be relatively short, no more than a few seconds in the audio. Below are some suggested things to try, you should initially attempt at least two of the following: • Pick a distinct sound and characterise it. -How would you describe the sound? This is subjective, but it may help distinguish between the different types of waves that are present. -How loud is it? What is its RMS amplitude as measured by the Analyze > Contrast tool? Can you convert this from dB back into the physical units of magnetic field in nT? Amplitudes should be estimated using the non-diff files. They can either try and read off the amplitude of the waveform or measure the RMS within the Contrast Analysis tool. Students may need to convert from decibels to the (dimensionless) waveform units: Note the factor is 20 and not 10 because we are not converting to power, which is A 2 , here. The peak amplitude of a perfect sine wave is related to the RMS by a factor of √ 2. Don't forget also to multiply the dimensionless waveform units into physical units using the 10nT factor at the very end. This conversion is implemented in the spreadsheet too. -Look at the spectrogram or plot a spectrum (Analyze > Plot Spectrum) using the diff file. Does it have a clear peak at a single frequency or set of frequencies/harmonics? Or does the sound show enhancements over a wide range of frequencies? Remember that spectra or spectrograms should be done with the diff files. The reason for this is that, like many other physical systems, the background noise profiles approximately follows a 1/f spectrum meaning that there is more power at low frequencies than at high. Spectrograms with the normal files will therefore be red near the bottom and blue near the top irrespective of what sounds are present as shown in the top panels of the figure below. The diff file essentially flattens out this background by taking the time difference (or derivative in time) so features on top of the 1/f noise are more distinguishable, therefore serving itself better for spectra and spectrograms as demonstrated in the bottom panels below. If a number of well defined frequencies are present, it is likely a standing wave of some sort within the magnetosphere and you can attempt to estimate the fundamental frequency. Often the fundamental frequency itself will not be excited/detected though by calculating the spacing between the detected harmonics or matching the ratios of these frequencies you can usually get an idea as to which harmonics they are. -Does the frequency or amplitude change as part of the sound? Is there a recurring pattern to this? The spectrogram is key to this first part. The frequency can change because of the changes to the wave speed (magnetic field strength and/or density) or the field-line length. These two factors can change either because the spacecraft is sampling a different region of the magnetosphere or that these quantities have actually changed e.g. in response to a change in the solar wind. Similarly with the amplitude, this may be due to the wave amplitude actually increasing/decreasing due to driving/damping or because the spacecraft has moved into/out of the region where these waves are occurring. -Is the wave predominantly poloidal, toroidal or compressional? This can inform what type of plasma wave it actually is. If it is solely poloidal or toroidal this points to Alfvén waves as these are the transverse components of the magnetic field. A compressional wave must be magnetosonic in nature. • Try to identify at least three different types of wave events / sounds that are present? -How would you describe the sounds? -Look at the spectrogram or plot a spectrum. Can you relate how the waves sound to the different types of spectra? Broadband waves (waves with enhanced power across a wide continuous range of frequencies) should have more noisy or thud like sounds compared to waves of distinct frequencies which should have sounds a bit more like musical instruments. -Where do these waves occur in local time? e.g. are they around for example dawn (06:00), noon (12:00), dusk (18:00) or midnight (00:00)? This only really applies to sounds less than a day in duration, otherwise it's clear the wave is a global phenomenon and likely of solar wind origin or due to a geomagnetic storm. -How long until the next similar event occurs? Is this waiting time always the same or does it differ between different events? You could build up a histogram of waiting times to see whether the events occur regularly (peaked distribution at a specific waiting time), at random (exponential distribution whereby you could use a fit to get the characteristic waiting time) or some other distribution. • Is there an identifiable daily cycle in the ULF activity? -Where in local time do they occur? -How variable is this cycle from day to day? The frequencies and amplitudes should vary from day to day, the variability of these is a key research question at the moment. Often students find two effects which are not physical. Firstly there are some periods in the audio of complete silence. This is where data is missing and could be for a variety of reasons. It is therefore not a particularly instructive area to focus on and students should be careful not to factor in any periods of silence in their analysis. Secondly there are some signals from the satellite itself present in the data at the high end of the audible frequency range which follows a daily cycle as shown below. It is clear from the very well defined frequencies and perfect repetition that these waves are not physical and are thus not worth investigating futher. In your research groups you should now decide what it is you'd like to research in more detail using the sonified magnetic field data. This can build on some of the sounds you worked on in the initial activities. You should attempt to identify, analyse and catalogue ULF events which fit within your chosen topic. Here are few ideas or approaches you may wish to take in your research: • Case studies: You may wish to focus on just one event or a handful of similar ones. These types of studies are particularly important for rare events. You should then perform thorough analysis on it to fully characterise when and where it occurred and what the waves' properties were. Looking for the same event at the different spacecraft may help get a feel for size. You could also try to find out what geomagnetic or solar wind conditions were present during the wave. You may not be able to determine with certainty what caused the event, but that is fine -often even professional researchers don't have enough information to do so. Students may need additional data to look into their case study events, please get in touch if you need help finding this. • Statistical surveys: Ideal for wave events that occur many times within the data, statistical studies can help us understand how often and where similar events occur and what range of properties they have. You should decide what aspect(s) of the waves you'd like to investigate in this manner (e.g. frequency, RMS, local time, time between events) and attempt to build up a comprehensive picture of these aspects of the waves over the course of a year-long data file, or indeed across multiple years if you have time. • Surveying Specific Conditions: You may wish to take a converse approach, rather than going through the ULF data to find events, you could choose a specific set of solar wind or magnetospheric conditions or previously identified events found online and then investigate the magnetosphere's ULF activity/response for these. Students can look up published catalogues of events like Coronal Mass Ejections (CMEs), Corotating Interaction Regions (CIRs) etc. or specific solar wind conditions like southward solar wind magnetic field. There are also magnetospheric activity indices which denote geomagnetic storms which may be of interest. Get in touch if you need help finding this additional data. • When and where do specific types of wave events occur? • How variable are the frequencies of certain types of wave events? • When or how often do large amplitude ULF events occur? • What were the causes of certain wave events? • How effective are different solar wind structures or features at driving ULF waves? Because a lot of the underlying physics will be unfamiliar to students, they can focus purely on wave topics surrounding the data, as per the initial activites. They do not have to worry too much about explaining everything in the context of the magnetosphere. If they make sufficient progress in identifying and characterising wave events, then the magnetospheric context can be explored with help from the researchers. You do not have to follow one of these approaches or topics, though do discuss thoroughly in your group and also with your teacher before getting started with your research. If you're still unsure what to do, please tell your teacher to get in touch with us so we can visit and provide guidance and assistance. Be sure to collate all your results on ULF wave events into the provided spreadsheet template. Enter your data into the white boxes, these will automatically calculate the dates, times and local times for you to save effort. You will likely, however, need to add extra columns depending on your research topic so discuss what information it is you need. You may also wish to use Audacity's editing tools to save clips of specific types of events for cataloguing and/or presenting your findings. Remember, that this is a taste of real research so you will get stuck and the answers may not be known. This is why it is important to persevere, discuss in your groups and with your teacher how to overcome any problems. If at any point if you find yourself unable to continue or completely unsure about something, ask your teacher to get in touch with us so that we can help you. On our website we also have advice on how to integrate and support students with projects, based on other schools' successful experiences. Providing some structure for your students, and mostly just encouragement throughout, is key to their and your success with these sorts of programmes. This guide is merely an introductory overview to the project and is by no means exhaustive. This means you will also need to refer to other sources as you work on your project. Firstly we have a number of resources on the project's website (http://qmul.ac.uk/spa/musics) including video guides on how to use some features of Audacity specifically applied to the space sounds (Audacity also has a very comprehensive manual covering all its features), and how to make scientific posters or talks to present your work at our student conference. We also have a number of videos which go into some more detail about aspects of these waves and examples of students' previous work. However, there is plenty of information about Earth's magnetosphere, the waves present in it and how we detect them available online from a variety of sources. Below are just some sources which you may find helpful: MUSICS website (videos, how to guides): http://qmul.ac.uk/spa/musics 012334 ÿ 36 ÿ 7289 19ÿ ÿ 9 338 12 ÿ 012334012334 ÿ 1 99 12ÿ ÿ 012334 93ÿ 3 01 6 1ÿ 39 9 ÿ 012334 ÿ ! ÿ "#$%ÿ #ÿ &'( %)! ( *'ÿ +&! %, 739 9ÿ ÿ 3ÿ 36 ÿ 2ÿ 3ÿ ÿ 6 3 9ÿ 36 ÿ 13 1 -ÿ 91 6 1ÿ 9 12ÿ 136 19. ÿ 9 12ÿ 39 9ÿ 9 9ÿ 6 3 3ÿ 3 ÿ 9 12ÿ 131 94 8ÿ 1 4 8ÿ 3ÿ 24 ÿ -ÿ 9199 3. ÿ ÿ ÿ 136 1/ ÿ ÿ 9 12 ÿ 9 9ÿ 08ÿ 2 39 ÿ 94 8ÿ 2 4 ÿ 3 2 ÿ 1 9ÿ 1ÿ 13ÿ ÿ ÿ 2ÿ 9 3ÿ 1 ÿ 2ÿ 2. 16 3 4 8/ ÿ 39 ÿ 39 9ÿ 2 ÿ 9 ÿ ÿ 136 19ÿ 08ÿ 36 99 34 91 9 9ÿ ÿ 3ÿ 0ÿ 3 45 56 78 4. ÿ 928:ÿ 94 4 3;ÿ 28< ÿ ÿ 9ÿ ÿ 03 34 99 ÿ 3ÿ ÿ 4 4 ÿ 36 ÿ 83 ÿ / ÿ -29ÿ 12 14 ÿ 4 -3. ÿ 28ÿ 94 4 8ÿ 2ÿ 8 33ÿ 12ÿ = ÿ 2 < 9ÿ 33ÿ > 14 ÿ ? 03 -@ ÿ 3ÿ / ÿ A -ÿ 2ÿ 39 4 -ÿ 3ÿ 83ÿ 0 39 -ÿ 6 3 6 . B C6 3 ÿ 83ÿ ÿ 9 ÿ 3ÿ 8ÿ ÿ 9 -ÿ 83 ÿ 39 / ÿ 83ÿ ÿ 3ÿ 6 -ÿ 3 ÿ 2 ÿ 2 3 4 4 ÿ 3 ÿ 36 ÿ ÿ 9ÿ ÿ 23ÿ 83ÿ 9234 ÿ -3ÿ 03 ÿ 13 1 -ÿ 2 . D92ÿ 9ÿ 2ÿ 39 ÿ 3 E 9 -E 9 3 -ÿ 6 -ÿ 6 3ÿ 8ÿ 9 12 3F 1 : D923ÿ 9ÿ 8ÿ -ÿ 1ÿ ÿ 2 ÿ 4 4 ÿ 36 ÿ 9 -ÿ 4 4 ÿ 28ÿ 2 06 3 2: 012334 ÿ 36 ÿ 7289 19ÿ ÿ 9 338 Figure S5 : Example 'how to' guide. 01ÿ 3415ÿ 6 3ÿ 7819 6 359ÿ 5ÿ 591 ÿ 6 1ÿ 9ÿ 3519 6 ÿ 9ÿ 4399 1 ÿ ÿ 38ÿ 35 6 ÿ 53ÿ 6 1 591 ÿ 3 ÿ 38 1ÿ 536 ÿ 6 1ÿ 3 ÿ 6 386 ÿ 386 ÿ 6 ÿ 6 1ÿ 41 935ÿ 99 ÿ 43 6 1 ÿ 9ÿ 93 ÿ 9 9ÿ 8ÿ 16 6 1 ÿ 6 5ÿ 6 5ÿ 6 3ÿ 1ÿ 84ÿ 5ÿ 5 3 1ÿ 14 56 35ÿ 35ÿ 6 1ÿ 8 6 1 ÿ 5 3 6 35 !11ÿ 6 1ÿ 1 3ÿ 8 19ÿ 3 ÿ 8 6 1 ÿ 6 49ÿ 35ÿ 5ÿ ÿ 9 156 ÿ 439 6 1 " #$3ÿ 6 3ÿ 19 5ÿ 5ÿ 55 5ÿ 35 1 151ÿ 439 6 1 #%19 5 5ÿ 385 6 359ÿ 3 ÿ ÿ 439 6 1 ÿ #&ÿ 9ÿ 6 3ÿ 16 1ÿ ÿ 431 8 ÿ 191 ÿ 439 6 1 #'318 ÿ 439 6 1 9 (336 35 ( 85 ( )ÿ !33 9ÿ 5ÿ 59 6 6 86 19 319 4 1 ÿ ÿ 4 39416 89 48 ÿ 5 Learning that Physics is 'Not for Me': Pedagogic Work and the Cultivation of Habitus among Advanced Level Physics Students So you're looking to run a research in schools project? Practical tips from the evaluation of a pilot programme How to undertake a programme of research-based engagement with schools and evaluate it, Session at Interact 2019 Symposium Schools of all backgrounds can do physics research: On the accessibility and equity of the PRiSE approach to independent research projects Thanks for helping me find my enthusiasm for physics!" The lasting impacts 'research in schools' projects can have on students, teachers, and schools Hill School Year 12 Physics students: First results from sonification and exploratory citizen science of magnetospheric ULF waves: Long-lasting decreasing-frequency poloidal field line resonances following geomagnetic storms Citizen scientist community engagement with the HiggsHunters project at the Large Hadron Collider A Rapid Evidence Review of Practical Independent Research Projects in Science Practical independent research projects in science: a synthesis and evaluation of the evidence of impact on high school students Citizen science: a developing tool for expanding science knowledge and scientific literacy Can citizen science enhance public understanding of science? Using thematic analysis in psychology Towards a More Authentic Science Curriculum: The contribution of out-of-school learning Implementing Project Work in Biology through Problem-based Learning MARVEL analysis of the measured high-resolution rovibrational spectra of C 2 H 2 Nuffield research placements: interim report, Tech. rep., Nuffield Foundation The use of confidence or fiducial limits illustrated in the case of the binomial The HiSPARC project: science, technology and education Student Engagement in an Independent Research Project: The Influence of Cohort Culture Which preparatory curriculum for the International Baccalaureate Diploma Programme is best? The challenge for international schools with regard to mathematics and science Assessing the influence of one astronomy camp over 50 years A Theory of Change for Improving Children's Perceptions, Aspirations and Uptake of STEM Careers Evaluation of authentic science projects on climate change in secondary schools: a focus on gender differences Students becoming researchers First results from the LUCID-Timepix spacecraft payload onboard the TechDemoSat-1 satellite in low Earth orbit Implementing Problem-Based Learning in Science Classrooms Nonparametric statistical inference Towards sustainable public engagement (outreach), New Directions in the Teaching of Physical Sciences IRIS opens pupils' eyes to real space research Demystifying academics to enhance university-business collaborations in environmental science Practical Work in School Science Which Way Now?, chap. Is this really what scientists do? Seeking a more authentic science in and beyond the school laboratory Observations of CH 3 OH and CH 3 CHO in a Sample of Protostellar Outflow Sources Nonparametric statistical methods Vitalizing creative learning in science and technology through an extracurricular club: A perspective based on activity theory, Thinking Skills and Creativity Statistical methods for psychology IOP: Raising Aspirations in Physics: A review of research into barriers to STEM participation for students from disadvantaged backgrounds, Tech. rep., Institute of Physics Making outreach work MARVEL Analysis of the Measured High-resolution Rovibronic Spectra of 48 Ti 16 O Degree-Course Destinations of Accepted Applicants with Physics and Mathematics A-level or Scottish Higher Inquiry-based science instruction -what is it and does it matter? Results from a research synthesis years Girls in the Physics Classroom: A Review of the Research on the Participation of Girls in Physics, Tech. rep., Institute of Physics National Coordinating Centre for Public Engagement: What is public engagement? HiSPARC project on National HE STEM Programme South West Region website ORBYTS: Original Research By Young Twinkle Students Transforming Education with the Timepix detector -ten years of CERN@school Nuffield research placements study: composition report, Tech. rep., Nuffield Foundation School sixth forms with no entries for A-level physics Real World Research Ways of framing the difference between research and evaluation From science teacher to 'teacher scientist': exploring the experiences of research-active science teachers in the UK Current approaches in implementing citizen science in the classroom Doing Qualitative Research: A Practical Handbook Original Research By Young Twinkle Students (ORBYTS): when can students start performing original research? Raising Interest in Science and Engineering summer internship program Physicists and Outreach: Implications of schools physics outreach programmes from the perspective of the participating physicists, phdthesis, Institute of Education The HiSPARC experiment Perceptions of STEM-based outreach learning activities in secondary education Wellcome Science Education Tracker CERN@school: bringing CERN into the classroom Impact of involvement in a science fair on seventh grade students Just Google it?': pupil's perceptions and experience of research in the secondary classroom The calibration system for the photomultiplier array of the SNO+ experiment ASPIRES Young people's science and career aspirations age 10-14, Tech. rep., King's College London It didn't really change my opinion': exploring what works, what doesn't and why in a school science, technology, engineering and mathematics careers intervention Learning that Physics is 'Not for Me': Pedagogic Work and the Cultivation of Habitus among Advanced Level Physics Students ASPIRES 2: Young people's science and career aspirations, age 10-19, Tech. rep., UCL Institute of Education So you're looking to run a research in schools project? Practical tips from the evaluation of a pilot programme Thanks for helping me find my enthusiasm for physics!" The lasting impacts 'research in schools' projects can have on students, teachers, and schools Hill School Year 12 Physics students: First results from sonification and exploratory citizen science of magnetospheric ULF waves: Long-lasting decreasing-frequency poloidal field line resonances following geomagnetic storms Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC Practical independent research projects in science: a synthesis and evaluation of the evidence of impact on high school students Towards a More Authentic Science Curriculum: The contribution of out-ofschool learning A 'work in progress' ?: UK researchers and participation in public engagement Wellcome Trust Monitor Wave 2: Tracking public views on science A Theory of Change for Improving Children's Perceptions, Aspirations and Uptake of STEM Careers 15/16-Year-Old Students' Reasons for Choosing and Not Choosing Physics at a Level The impact of the stimulating physics pilot on student uptake of physics post-16 Practical Work in School Science Which Way Now?, chap. Is this really what scientists do? Seeking a more authentic science in and beyond the school laboratory IOP: Raising Aspirations in Physics: A review of research into barriers to STEM participation for students from disadvantaged backgrounds, Tech. rep., Institute of Physics Overview of the Kepler science processing pipeline Who has high science capital? An exploration of emerging patterns of science capital among students aged 17/18 in England Science capital or STEM capital? Exploring relationships between science capital and technology, engineering, and maths aspirations and attitudes among young people aged 17/18 Girls in the Physics Classroom: A Review of the Research on the Participation of Girls in Physics, Tech. rep., Institute of Physics Transiting Exoplanet Survey Satellite (TESS) Identifying Exoplanets with Deep Learning: A Five-planet Resonant Chain around Kepler-80 and an Eighth Planet around Kepler-90 The Relationship of 21st Century Skills on Students' Attitude and Perception towards Physics Original Research By Young Twinkle Students (ORBYTS): when can students start performing original research? Who Owns the Theory of Change? TeachSpin: Muon physics Physicists and Outreach: Implications of schools physics outreach programmes from the perspective of the participating physicists, phdthesis Capturing impact: an informal science education evaluation toolkit Just Google it?': pupil's perceptions and experience of research in the secondary classroom Acknowledgements. We thank Dominic Galliano for helpful discussions. MOA and CT are grateful for funding from the Ogden Trust. MOA holds a UKRI (STFC / EPSRC) Stephen Hawking Fellowship EP/T01735X/1. This programme has been supported by a QMUL Centre for Public Engagement Large Award, and STFC Public Engagement Small Award ST/N005457/1. ÿ ÿ 7 7 ÿ 49 7 ÿ 8 14ÿ 414 9ÿ 2 27 7 2ÿ 28 ÿ 4ÿ 1ÿ 7 7 ÿ 2 3ÿ 4ÿ 987 9ÿ 28ÿ 4 ÿ 47 9ÿ ÿ 47 ÿ 94 ÿ 38 ÿ 34ÿ 98 4ÿ 4 ÿ 2 14 ÿ 9 23 289 ÿ 07 924 ÿ 125 ÿ 2 4 ÿ 2ÿ 7 814 28 ÿ 29 4 5 9ÿ 7 44 ÿ 28 ÿ 49ÿ 1 4 4ÿ 285 4ÿ 2ÿ ÿ 4ÿ 2 ÿ 0 4 4 449ÿ 28ÿ 441ÿ 2ÿ 7 814ÿ ÿ 34ÿ 4 ÿ 2ÿ 97 7 ÿ 94 2ÿ ÿ 4ÿ 41ÿ 1ÿ ÿ 894ÿ 92 4 ÿ 2 9ÿ 2 ÿ 4 4 4 4 ÿ 9ÿ 95 ÿ 87 7 ÿ 37 2ÿ 4 ÿ 64 ÿ 7 7 25 ÿ ÿ 1ÿ ÿ 22ÿ 8ÿ 2ÿ 28 ÿ 29 4 ÿ 74 4ÿ 94ÿ 9ÿ 7 29 ÿ 9ÿ 2 9ÿ 7 7 ÿ 4ÿ 987 9ÿ 1ÿ 48 ÿ 934ÿ 98 4ÿ 28ÿ 7 44ÿ 428ÿ 94ÿ ÿ 4ÿ 4149ÿ 2 ÿ 28 29 4 ÿ 1ÿ 34 44ÿ 7 7 ÿ 4ÿ 47 9ÿ 1ÿ 987 9 ÿ 02 1ÿ 33 281ÿ 8 49ÿ 2 ÿ 2 29ÿ 9 4ÿ ÿ 34ÿ 1 9 :4ÿ 4 87 ÿ ÿ 28 ÿ 894ÿ 2 ÿ 27 28 ÿ ;27 28 9ÿ 44 ÿ 38 ÿ 4ÿ ÿ 22ÿ 27 28 9ÿ 24 4 ÿ 28 ÿ 29 4 ÿ ÿ 49 7 9 ÿ 2ÿ 7 223ÿ 14289 ÿ <4ÿ 2ÿ 2 ÿ 44 9149ÿ 2 ÿ ÿ ÿ 27 28 ÿ 2 ÿ 28 2 44 ÿ ÿ 4 ÿ 27 28 ÿ ÿ 9 19ÿ 28 4 1ÿ ÿ 287 4ÿ 2 ÿ 48 ÿ 27 28 9 ÿ 6 ÿ ÿ 27 28 944ÿ 2 ÿ 9ÿ 3 14 ÿ 28ÿ ÿ 894ÿ 4 4 ÿ 27 28 ÿ 2ÿ 1 ÿ 4 2ÿ 2ÿ 4 4 28ÿ ÿ 427 4ÿ 2ÿ 7 223 ÿ =4ÿ 2 ÿ 9ÿ ÿ 28ÿ 894ÿ 4ÿ 4 ÿ 27 28 ÿ ÿ 214 2 7 7 4 ÿ 34ÿ 98 4ÿ 28ÿ 894ÿ ÿ 7 4 ÿ 9 7 4ÿ 2 ÿ 1ÿ ÿ 28ÿ 94 ÿ 4ÿ 9 49ÿ 7 4ÿ 428 427 4ÿ 441ÿ 2ÿ 34ÿ 37 4ÿ 2ÿ 41ÿ 44 ÿ 9 221ÿ ÿ ÿ ÿ 1 9 4ÿ 2ÿ 28 ÿ 29 4 25 ÿ 2 4 ÿ 4ÿ 494 2 29 4 9ÿ 4ÿ 24 9 2ÿ 9 4 94 ÿ 92ÿ 494 ÿ ÿ 29 4 ÿ 9ÿ 3927 8 47 ÿ 7 ÿ 28ÿ 9287 1 4 4ÿ ÿ >?ÿ 9421ÿ 2ÿ @ÿ 8 4ÿ -1 -A0, #$ ÿ +( , )/ÿ 2 ÿ 28 ÿ 2 3ÿ ÿ 7 7 ÿ 34ÿ 427 4 B8 37 ÿ 28ÿ 28 ÿ 29 4 ÿ 1ÿ 9ÿ ÿ 2 9 ÿ = 9ÿ 9ÿ 7 92ÿ 4 4ÿ 28ÿ ÿ 92ÿ 2C ÿ 2 ÿ 28 ÿ 11 27 ÿ 4 7 9