Prior Art Research in the Capstone Design Experience: A Case Study of Redesigned Online and In-person Instruction

Graham Sherriff
Engineering Librarian
Howe Library
University of Vermont
Burlington, VT
graham.sherriff@uvm.edu

Dustin Rand
Senior Lecturer
College of Engineering and Mathematical Sciences
University of Vermont
Burlington, VT
dustin.rand@uvm.edu

Abstract

Keywords: Engineering, Instructional design, Prior art, Patents, Standards

Recommended citation:

Sherriff, G., & Rand, D. (2022). Prior art research in the capstone design experience: A case study of redesigned online and in-person instruction. Issues in Science and Technology Librarianship, 100. https://doi.org/10.29173/ist2598

Introduction

This paper presents an innovative approach to a prior art review assignment in the capstone course for mechanical, electrical, and biomedical engineering students at the University of Vermont (UVM). Capstone courses are a core element of undergraduate engineering education that provide a range of experiential learning experiences that extend and enhance the skills that students will use after entering the profession. As such, they meet the ABET accreditation requirement for “a culminating major engineering design experience that (1) incorporates appropriate engineering standards and multiple constraints, and (2) is based on the knowledge and skills acquired in earlier course work” (ABET, 2020, p. 6). UVM’s Senior Experience for Engineering Design (SEED) course for mechanical, electrical, and biomedical engineering students is a two-semester, six-credit course, in which project teams of typically four students work with a community- or campus-based client to design and develop a solution to an existing engineering problem.

The Fall semester of the SEED course includes a prior art review assignment (PAR), where project teams research the “prior art”—the state of a technology’s development, as manifested in literature, documentation, and artifacts—in relevant fields of technology. Prior art often includes patents and research literature, though it can also include standards, product/trade literature such as manufacturers' catalogs and manuals, and other types of publications or documents (Bourbon, 2006; Clarke, 2014; U.S. Patent and Trademark Office, n.d.). Prior art research centers on learning what has been done to solve all or part of a problem. “Prior art does not need to exist physically or be commercially available. It is enough that someone, somewhere, sometime previously has described or shown or made something that contains a use of technology that is very similar to your invention” (European Patent Office, n.d.-b, para. 2).

The PAR assignment has three learning outcomes, each one aligned with ABET Criterion 3 Student Outcome 7, which is “an ability to acquire and apply new knowledge as needed, using appropriate learning strategies” (ABET, 2020, p. 6):

1. To learn and demonstrate how prior art relates to and informs design decisions on your project.
2. To give you experience of searching engineering literature and documentation that will support your project.
3. To use existing literature to understand and describe the novelty of your problem statement.

The PAR assignment emphasizes that prior art is not found in one location. Instead, it is spread across multiple types of publications and mediums. “A prehistoric cave painting can be prior art. A piece of technology that is centuries old can be prior art… Anything can be prior art” (European Patent Office, n.d.-b, para. 3). SEED teams are tasked with searching a variety of sources that will support their project, focusing principally on research literature, patents, standards, and commercial products (Table 1).

Types of Information	Types of Publication	Resources
Research literature	Journal articles and conference papers	Indexes (e.g., Engineering Village, PubMed, Google Scholar)
Intellectual property	Patents	USPTO databases and Google Patents
Industry best practices	Standards	(1) Consultation with project client; (2) standards indexes (e.g., ANSI, FDA); (3) standards organization websites (e.g., ASTM, ASME)
Commercial development	Products	Manufacturers' websites and catalogs

However, review of project reports by course faculty showed that many teams were failing to research the prior art with the necessary breadth or complexity of technique. Bibliographies were often brief and contained “popular” sources rather than academic or technical ones. Entire publication types, such as patents or standards, were often overlooked. This suggested a hypothesis that the shortcomings in project reports reflected knowledge gaps and that students' overall levels of familiarity with the types of publication used in the PAR assignment were low.

There were multiple factors that might have contributed to these knowledge gaps. First, SEED faculty were aware that most SEED students were entering their senior year with negligible experience of engineering publications. A 2016 curricular mapping exercise by the UVM College of Engineering and Mathematical Sciences, in conjunction with the campus Writing Center, had shown that information literacy skills were being taught in only a small number of engineering courses and without a systematic or scaffolded approach. Anecdotal observation suggested that some SEED students had coursework or work experience using a single publication type such as journal articles or standards, but most did not.

Secondly, the time available for information literacy instruction in the SEED course is limited. Course instruction needs to address various aspects of project management and product development, with teams also needing to dedicate much of their time to client communications, product development, and testing. Class time for PAR instruction was effectively limited to a single 50-minute session in which core concepts and research skills could be covered only with limited depth.

Thirdly, reviews of PAR reports indicated that SEED students tended to do a minimum of research and rely on the most easily available sources. In other words, they were “satisficing,” that is, conducting a minimally satisfactory research process rather than an optimally extensive one (Prabha et al., 2007). This approach may be efficient and not uncommon in professional practice (Allard et al., 2009). It may even be considered appropriate to students' development of their identity and practices as engineer-designers (Mercer et al., 2019). However, it may be a disadvantage in terms of finding a full range of relevant sources and not overlooking key information. It may also be disadvantageous in the context of prior art research, where the researcher is seeking to understand as fully as possible the extent to which previous designs have solved the problem (or not). “A thorough and well-recorded search is essential - because how else do you prove an absence of prior art?” (European Patent Office, n.d.-a, para. 22).

Lastly, some of the dynamics of engineering design courses may be obstacles to effective prior art research. Students need to conduct their research at the outset of their project, at a time when they may have limited understanding of the design problem or their client's needs, or limited technical knowledge of the technologies they will be using. Students may prefer to invest their time in “doing” and “making” activities rather than information-gathering and reading (Clarke, 2014). Teams may be inclined to delegate a group task like prior art research to one member—typically, someone who already has some competence in that type of task, with the result that the members most in need of skills development opt out.

Some of these factors would be easier to resolve than others. What was clear to the course faculty was a need to confirm the gaps in students' knowledge and redesign the instruction for the PAR assignment to fill those gaps and equip students for effective research.

Literature Review

Exploration of prior art has many benefits for capstone students. Existing technologies are the basis of understanding what is currently possible, what is legally protected, and how problems are currently solved. Prior art reveals a range of possible design solutions, provides the engineering and physical principles involved in existing devices, and clarifies what is truly novel about a project (Clarke, 2014; Nazemetz et al., 2007). For example, Phillips and Zwicky (2017) observed that mechanical engineering students found patents useful for assessing the patentability of their designs, exploring the current state of the art, reviewing previous responses to technological problems, verifying the feasibility of their ideas, and inspiring creativity. According to Brown (2016), patent instruction helps students to deepen their understanding of a technology's design “journey” and what makes their own design inventive. Using the case study of barbed wire, they show how patent research in open databases reveals the history of technological development and, by implication, the “white space” in which new intellectual property might be claimed.

Regarding standards, these are an important source of best practices for quality, reliability, measurement, interoperability, and safety (Osif, 2014). Adopting best practices created by experts “results in time and money savings and the avoidance of unsuccessful or inefficient processes” (p. 119). It also provides assurances to consumers regarding the quality of the product.

Prior art research is thus important to the success of each SEED project. It is also valuable for the development of research skills that students will use when they enter professional practice, including literature searching, acknowledgment of sources, and intellectual property issues. Mosberg et al. (2005) surveyed professional engineers on what they considered the most important design activities, and “seeking information” was rated fourth most-important out of 23 options. Information seeking and source management skills align with ABET student learning outcomes and constitute transferable skills that have value for professional success (Lutz & Paretti, 2017). Practicing engineers spend a significant amount of time on information-related activities, mainly gathering information from interpersonal communication; publications, especially journals, magazines, and conference papers; and internet searching (Phillips et al., 2019; Tenopir & King, 2004). Standards and, less so, patents are also key sources of information for practicing engineers (Jeffryes & Lafferty, 2012; Tenopir & King, 2004).

Yet despite the value of in-depth prior art research, engineering students in many instances spend very little time on information gathering. Ekwaro-Osire et al. (2008) evaluated two undergraduate design projects and found that teams spent only 0.1% of their time on “library research.” Other studies have documented a tendency by students to focus on simple web searching and lower-quality publications (Denick et al., 2010; Wertz et al., 2011). Moody et al. (2012) described how upper-level engineering students were inclined to assume a problem statement would contain all the information needed to understand it and design a solution.

This is likely due to the “satisficing” research practices described above, but also in some significant measure to gaps in the instruction needed to equip students for complex searching. Searching for and accessing technical publications are complex tasks and students need an advanced knowledge of “the breadth of their institution’s literature collection and how to efficiently find information with online catalogs, subject guides, indices, and literature databases” (Clarke, 2014, p. 129). Yet Zabihian et al. (2015) observed at West Virginia University Institute of Technology that “even senior level students have not received proper training, either directly or indirectly, in information literacy” (p. 1) and rely on Google and Wikipedia for information gathering. McAdams and Glauberman (2017) conducted a curriculum mapping exercise at The University of North Carolina at Charlotte and reported a “lack of authenticity, scaffolding, and consistency that characterize the few structured encounters undergraduate engineering students have with the library and information literacy instruction” (McAdams & Glauberman, 2017, Introduction).

Capstone design courses thus tend to require students to apply complex skills for the seeking and interpretation of technical publications, that have received limited attention in their programs up to that point. Mercer et al.'s (2019) scoping review of engineering students' information-seeking behaviors confirmed that “capstone design projects are turning points for information literacy in the engineering curriculum as they incorporate information seeking at multiple points in the process and integrate a broad assortment of information resources used for a variety of purposes throughout the projects” (p. 18).

This paper describes how the foundational instruction for a PAR assignment was mostly “flipped” from in-person presentation to online tutorials. According to Phillips, Van Epps, et al.'s systematic review (2018), no difference in effect between online and in-person instruction has been demonstrated in information literacy instruction for engineering students. Zhang and Kozak (2017) compared the effectiveness of information literacy instruction for undergraduate engineering students in videos and online tutorials. Their study found that these technologies were equally effective in terms of learning outcomes and user satisfaction, and that a majority of the students who participated preferred the tutorials. These studies appear to validate the decision to flip the core instruction to online tutorials.

Interestingly, Phillips, Van Epps, et al. (2018) also found that online instruction was used only for “general” information literacy topics. “[T]his is not surprising, since learning the twists and turns of technical literature often requires a nimble response to students' individual information needs, in the form of an in-person instructor” (2018, p. 710). This suggests that the online delivery of technical information literacy instruction at UVM is a relatively innovative approach, while also indicating that online instruction for engineering students may be most effective when combined with some form of responsive in-person instruction.

One recent example of online technical information literacy instruction is Phillips, Fosmire, et al.'s (2018) development of a set of four interactive tutorials on standards for first-year design students. These tutorials were implemented in response to similar needs as at UVM; a need to flip core instruction into an online space and switch the limited in-person instruction to active learning. They were developed in-house as a result of an instructional gap: “there is little, if any, material that is not specific to a particular standard developing organization (SDO), institution, or discipline; targeted to undergraduate students; interactive; and includes information literacy components” (Phillips, Fosmire, et al., 2018, p. 3).

Research Questions

This case study describes an effort to address the knowledge gaps and weaknesses in research skills that the authors had observed in SEED teams’ research reports. This was done by examining the following research questions:

1. At the time of entering their senior year and beginning a capstone design course, to what extent do UVM's engineering students understand foundational concepts relating to prior art?
2. To what extent do they understand foundational concepts relating to specific types of publication where prior art can be found? Specifically, journal articles, conference papers, patents, and standards.
3. Are online tutorials an effective mode of instruction for addressing knowledge gaps identified by RQ 1 and RQ 2?

Methods

In Fall 2018, the instruction for the SEED course's PAR assignment was substantially redesigned. Previously, the instruction had been one 50-minute class presentation on prior art and research strategies by the engineering librarian. This presentation needed to cover a large amount of informational content, including how to use indexes and journal aggregators, patent searching with Google Patents, identifying and obtaining standards, citation management, and library support. The need to cover a lot of content meant that the session was almost entirely a one-way presentation and allowed little, if any, time for active learning or discussion. The presentation was followed by “research consultations” in which the librarian met with each team to provide project-specific guidance as they embarked on their PAR research.

In the redesigned instruction, the first step was the creation of three interactive online tutorials. This was accomplished using the open source “Guide on the Side” platform provided by the University of Arizona Libraries (UAL) and shown in Figure 1 (Sult et al., 2013). As noted by Zhang and Kozak (2017), this is an effective and user-friendly technology for online instruction. One tutorial on “Research Literature” covers journal articles and conference papers. The other tutorials cover patents and standards. These tutorials guide the students through the characteristics of each publication format: the information they contain, who publishes them, and their authority. They also explain where to search for them and how to navigate restrictions on access.

The SEED tutorials can currently be seen at http://go.uvm.edu/lib-tutorial-intro-research-literature-stem, http://go.uvm.edu/lib-tutorial-intro-patents, and http://go.uvm.edu/lib-tutorial-intro-standards. They are now hosted on Springshare's LibWizard platform due to UAL's discontinuation of support for the “Guide on the Side” platform. The American Libraries Association (ALA)/Association of College and Research Libraries (ACRL) Instruction Section recognized the original versions as its March 2018 “Site of the Month” and maintains links to them in its Peer-Reviewed Instructional Materials Online (PRIMO) repository (http://primodb.org/) (ACRL Instruction Section, 2018).

Tasks and questions are displayed in the left-side panel, while live web content is displayed in the right-side panel. Students work through each tutorial by following prompts and answering a series of unscored formative questions, then their learning is assessed by a short quiz of five questions that test their understanding of key points. A passing score of 80% (four correct answers out of five questions) is required and unlimited re-takes are allowed. Each student completes all three tutorials, which together typically require a total of 45-60 minutes to complete.

Before taking the tutorials, students completed a pre-test of the 15 quiz questions (five in each tutorial quiz). They then completed the tutorials, which contain all the information needed to answer all questions correctly. The tutorials end with a culminating quiz. These quizzes have the same questions as the pre-tests and thus serve as a post-test for the effectiveness of the tutorial instruction. Quiz questions are shown in Table 2, Table 3, and Table 4 with correct responses indicated by an asterisk.

Question #	Question	Multiple-choice Options
R01	What is research literature?	Descriptions of new ideas and practices, written by the people who created them * Descriptions of new innovations across a field of Engineering, written by an expert Descriptions of current best practices, written by a Professional Engineer
R02	What type of publication is this [URL linking to an index record for a conference paper]?	A journal article A conference paper *
R03	What is one of the key differences between journal articles and conference papers?	Journal articles have to meet higher standards of peer review * Journal articles are much longer Journal articles are published more quickly
R04	True or false: regular search engines will find the same research literature as databases.	True False * Impossible to know
R05	Your initial search in Engineering Village generates lots of non-relevant results. Which option would be appropriate in this scenario?	Add Controlled Terms to your search words * Make your search terms broader Conclude your search

Question #	Question	Multiple-choice Options
P01	True or false: patents are owned by the assignee, not the inventor.	True * False
P02	It can be difficult to find relevant patents. Which of these is one of the reasons for this?	Patents are private legal documents Patents often use obscure language *
P03a	What is prior art patent searching?	Searching for patents to check that an invention has not already been patented Searching for patents to learn about the history of a technologya * Searching for design patents that are artistic or decorative
P04	Search in Google Patents for the keywords “ethanol” and “distillation”. What is the top-ranked CPC class?	C07C * C12P Y02E
P05	Which section of a patent document explains how an invention works?	Abstract Images Description * Claims Patents don't explain how the invention works
a See Results for additional explanation of this question.

Question #	Question	Multiple-choice Options
S01	What is a standard?	A document containing specifications for technical characteristics, behaviors or processes * A measurement commonly used in industrial manufacturing A legal document specifying mandatory levels of production quality and performance
S02	Which of these is not a benefit of compliance with relevant standards?	Compliance may reduce your legal liability in the event of litigation Compliance ensures that product development is cost-effective * Compliance supports compatibility of different systems or components Compliance tells clients and consumers that you follow industry best practices
S03	Which type of organization does not publish standards?	Professional associations Government agencies Libraries *
S04	What should be your first step when trying to identify relevant standards?	Consult with the client * Search the websites of standards organizations Search Standards Infobase
S05	When might you need to use Standards Infobase?	Searching for relevant standards Searching for a document preview Requesting a standard document *

“Flipping” the foundational knowledge about core publication formats into online tutorials created a larger instructional space than the 50-minute one-shot presentation. It allowed students to cover the material at their own pace. Most importantly, the tutorials ensured that all students began the PAR assignment with a common baseline of knowledge.

The second component in the instruction for the PAR assignment is the class session. As mentioned above, this was previously a lecture-style presentation that needed to cover a wide-ranging survey of literature resources and research strategies. The presentation was largely one-way and provided little time for active learning or discussion. The instructors' observations indicated the mode of delivery was not engaging.

Covering foundational knowledge in the tutorials made it possible to shift the class session to a workshop-style approach. At the end of each tutorial, a final question asks “What is something about [research literature/patents/standards] that is unclear?” The librarian was thus able to gather and review students’ questions. The presentation was then reoriented away from basic content that students have mastered. Instead, it was focused on clarifying students’ authentic points of uncertainty and explore more advanced questions. For each type of prior art, the librarian presented on key points for reinforcement, then tasked the students with working in their project teams to discuss and answer some of the questions submitted in the tutorials. Each question was then discussed as a class with opportunities to raise other questions that had come to mind. This workshop approach is better suited to active learning and large-class engagement than a simple presentation.

The third component of instruction for the assignment is targeted support for each project team in the form of a team meeting with the librarian. Due to the variety of projects and technologies under consideration, different teams may need to use different types of publication. For example, astronautical projects need to use NASA technical reports—a type of publication not covered by the tutorials. They may need to use indexes and search engines not covered by the tutorials. Every group benefits from feedback on the effectiveness of its research and recommendations that are specific to their project.

Arrangements for these meetings were adjusted as part of the Fall 2018 redesign. Previously, meetings were scheduled when teams were beginning their research and lasted 60 minutes. This was changed to scheduling them for a time when teams have already done substantial searching and need “fine tuning” to resolve further questions and navigate access to needed materials. Teams are graded on their preparation for the meeting, including the articulation of questions about outstanding information needs. Due to an increase in the number of projects in recent years, meetings were shortened to 30 minutes, placing more importance on conducting the meetings efficiently.

Teams are graded for members’ completion of the tutorials and preparation for the research meeting. In addition, there are two deliverables:

1. Research log: A collaborative document in which team members record search terms, tool/site used, what was found, and the effectiveness/usefulness of the search (Figure 2). The log demonstrates the team’s due diligence and completion of work for the purpose of assessment. It also allows the team to coordinate on the assignment, avoid duplication of effort, reflect on the process, and improve its searches as its research progresses. For example, it may help teams to identify remaining information gaps and determine what further help is needed.


Figure 2. Sample entry for a PAR research log


2. Report: Each team is required to reflect on the organization and effectiveness of its search strategies. The report should then summarize the relevant sources that were found, discuss how the sources might influence the project’s design considerations, and re-evaluate the novelty of their project in the context of existing prior art. Reports also need to meet specifications for the citation of sources and professional-grade language and document presentation.

Grading is done collaboratively. Team deliverables are divided between the course instructor, the course teaching assistant, and the librarian for grading according to a rubric designed by the course instructor with input from the librarian. See Appendix for the grading rubric.

The UVM Research Protections Office has determined that this quality improvement study is exempt from IRB review.

Results

Figure 3 shows the pre- and post-test scores for each tutorial. The pre-test assessment confirmed significant knowledge gaps concerning the characteristics of the different publication types. Overall, students had better knowledge of research literature with a mean tutorial score of 3.7/5 or 74% (N = 104). Pre-test scores were lower for patents, with a mean score of 3.0/5 or 60% (N = 104). Pre-test scores were lowest of all for standards, with a mean score of 2.1/5 or 42% (N = 103).

In the “Research literature” pre-test (Figure 4), students scored higher on questions about the use of search engines and databases (R04: 91% correct and R05: 97%). They scored somewhat well on a question about the distinctive characteristics of journal articles and conference papers (R03: 76%). However, they scored much lower on a question that tested their ability to distinguish between these two formats (R02: 43%). Scores were also problematically low on a question that assessed students' understanding of the scope of research literature (R01: 57%).

In the “Patents” pre-test (Figure 5), students scored moderately high on some questions and very low on others, specifically P01 on patent ownership (47% correct) and P03 on the definition of prior art patent searching, as opposed to other kinds of patent searching, such as “freedom to operate” searching (39%).

Regarding P03 on the definition of prior art patent searching, it should be noted that some definitions of this term include searching for patents to check that an invention has not already been patented—which is an incorrect response for this question. In the UVM SEED course, the students are directed to approach prior art searching as an exploratory exercise. As noted above, the assignment objectives are to expand their understanding of the design problem, reveal a range of possible solutions, and develop their research skills with learning outcomes that map to these objectives. The students are explicitly informed that their prior art research is not intended to be a comprehensive intellectual property search with legal implications.

In the “Standards” pre-test (Figure 6), a high proportion of students scored correctly on a question about which types of organizations publish standards (S03: 85%). However, very low proportions scored correctly on all other questions about standards, including the definition of a standard (S01: 24%), benefits of compliance (S02: 38%), identifying relevant standards (S04: 19%), and using a standards database (S05: 46%).

The post-test scores were substantially higher than the pre-test scores (Figure 3). Mean post-test scores rose to 4.6/5 or 92% (N = 104) for “Research literature,” 4.4/5 or 88% (N = 104) for “Patents,” and 3.6/5 or 72% (N = 103) for “Standards.” Mean post-test scores remained highest for “Research literature” and lowest for “Standards.”

In the “Research literature” post-test (Figure 4), scores increased for all individual questions. The percentages for correct responses rose from a range of 43%-97% to 82%-99%, indicating a major increase in knowledge and one that spanned all of the key issues being assessed. The proportion of students who correctly identified a conference paper increased by 42 percentage points.

In the “Patents” post-test (Figure 5), scores again increased for all questions with the percentages for correct responses rising from a range of 39%-81% to 76%-94%. This was another major improvement, though also an indication that further attention to the definition of prior art searching is needed (P03: 76%).

In the “Standards” post-test (Figure 6), scores increased from a range of 19%-85% to 56%-88%. As noted, post-test scores were not as high as those for the other two tutorials, but the pre-test scores had been much lower and “Standards” questions accounted for most of the larger improvements in scores. In particular, there was a 51 percentage point improvement on S01 (definition of a standard) and a 44 percentage point improvement on S04 (identifying relevant standards).

Looking at the research logs, anecdotal observations by course faculty identified several areas of improvement. Teams tended to begin their prior art research earlier in the assignment timeframe. They searched more widely, exploring all the required publication formats and using a fuller range of recommended search engines, indexes, and websites. They applied recommended search strategies, including the formulation of complex search queries with keywords, controlled vocabulary, and patent classification. Importantly, research logs showed greater persistence in searching for relevant material, by revising search terms and iterative searching.

Informal review of team reports and grades also highlighted some improvements in literature searching practices. Teams were more strategic in their organization of the assignment, for example in the allocation of responsibilities and the compilation of sources. Reports contained more reflection on the purpose and process of prior art research, such as insights into why some publications and searched tools were important for their project, and why others were not. Sources tended to include more publications that were authoritative and relevant, notably more peer-reviewed journal literature, and fewer articles from commercial or non-technical websites. Reports also conveyed a better understanding of how a project stands in relation to existing products or technologies: which aspects of a project are truly novel and the ways in which the project may be considered innovative.

Discussion

The pre-test provided clear indicators of areas where student knowledge tended to be strong and areas where there were knowledge gaps. Prior knowledge levels were highest for research literature, most likely reflecting greater exposure to journal articles than other formats in previous courses. Scores were higher for questions on the search tools used to find research literature than those on the concepts underpinning the reasons for engaging with research literature. Students appeared to understand the distinctions between journal articles and conference papers, but struggled to identify a conference paper from an index record, perhaps due to not having worked with conference papers previously.

As noted above, students scored moderately high on some questions in the “Patents” pre-test and very low on others; and, on balance, lowest on the “Standards” pre-test. Conversations with SEED students support the impression that most had no prior exposure to these formats. “Technical standards are probably the least familiar type of technical literature for capstone design teams, and some students may never have read a standard prior to their first major design project” (Clarke, 2014, p. 133). These curricular gaps exist despite the importance of these formats to professional practice.

The post-test scores indicated that the interactive online tutorials can be a highly effective medium for delivering engineering information literacy instruction and covering the gaps in capstone student knowledge. Post-test scores were in many cases substantially higher than pre-test scores: in the pre-tests, seven of 15 questions had a correct response rate below 50%; in the post-tests only two questions had a correct response rate below 75%. There is still room for improvement, especially regarding standards, where the mean post-test score was 72%. Again, this likely reflects the fact that most students were working with standards for the first time, and it is probably appropriate to extend and reinforce the tutorial's subject content.

The research logs and assignment reports showed that SEED students completed the tutorials, then moved on to their prior art research and applied the techniques and strategies that the tutorials had recommended. Most logs and reports contained a satisfying depth of reflection on the process of working with technical literature and documentation, while the questions elicited by the tutorials also demonstrated that many SEED students are thoughtful and inquisitive about the nature of engineering information. For example:

• How much access will I have to research literature once I graduate?
• How to know when to stop looking for patents and conclude that one does not already exist?
• Who determines these standards are correct?
• Why is it necessary to pay for standards?

These are sophisticated questions that warrant discussion with the class as a whole. The “flipping” of foundational knowledge from the class presentation into online tutorials created time in the class session to explore them. It also had the additional benefit of changing a “sage on the stage” class presentation to a more engaging workshop-style session in which teams were able to consider and discuss complex questions.

A multifaceted approach like the one used in the redesigned PAR assignment involves a substantial amount of preparation. The design, production, testing, and maintenance of the tutorials constitute a significant investment of time, as are the team meetings with the librarian. Each year, the tutorials need to be checked for being up-to-date and technical issues like broken links. These investments of time have been mitigated in some ways, such as assigning the processing of tutorial scores to the course teaching assistant and scaling back the consultation meetings from one-hour “getting started” sessions to 30-minute “fine-tuning” sessions. However, the time investment seems justified by the impact on students’ information-seeking competencies, their prior art research, and their project work

Conclusion

Capstone design courses are an integral component of engineering undergraduate programs. In these courses, students learn about all aspects of the design process and undertake an authentic, client-based project in which they need to apply that learning as they work towards the development of a functional design solution. As a result, they need to learn quickly and early, so they can work efficiently and effectively.

This is equally true for prior art research. The pre-test in this study confirmed that UVM engineering students commonly enter their senior year unfamiliar with the concept of prior art and having only modest experience of journal articles and negligible familiarity with other types of literature like conference papers, patents, or standards—all which are rich sources of prior art. These are substantial learning areas that present a challenge for course faculty, who need to ensure all students have a baseline of foundational knowledge at the outset of the PAR assignment, which was difficult to accomplish in a one-shot library presentation and time-consuming to do in one-hour orientational meetings with the engineering librarian for each team.

Online tutorials are a new and, at the time of writing, little-used means of delivery for engineering information literacy instruction. However, the post-test in this case study demonstrates that they can be used both to provide effective instruction and to free up in-person instruction for deeper exploration of the role of prior art in design practices. The result has been that SEED students learn the process for a professional-grade search for prior art. They adopt better information-seeking practices, acquiring behaviors that will serve them well as they enter the profession. They also find more high-quality technical sources that will likely help them to understand the client's problem, evaluate the current state of the art, and develop a superior design solution.

Acknowledgments

Data collection and reporting in Guide on the Side was made possible with the addition of custom code scripted by UVM programmer Wesley Wright.

References

ABET. (2020). Criteria for accrediting engineering programs. https://www.abet.org/wp-content/uploads/2021/02/E001-21-22-EAC-Criteria.pdf

ACRL Instruction Section. (2018, March). March 2018 site of the month. https://acrl.ala.org/IS/instruction-tools-resources-2/pedagogy/primo-peer-reviewed-instruction-materials-online/primo-site-of-the-month/march-2018-site-of-the-month/

Allard, S., Levine, K. J., & Tenopir, C. (2009). Design engineers and technical professionals at work: Observing information usage in the workplace. Journal of the American Society for Information Science and Technology, 60(3), 443–454. https://doi.org/10.1002/asi.21004

Bourbon, R. (2006, February 21). Standards as prior art at the European Patent Office. https://www.etsi.org/images/files/SOSInteroperability/SOSinteropIIIpresentation3-01.pdf

Brown, D. P. (2016, June 26-29). Teaching patents and design novelty to engineering students: A narrative case study based approach [Paper presentation]. 2016 ASEE Annual Conference & Exposition, New Orleans, LA, United States. https://doi.org/10.18260/p.26041

Clarke, J. (2014). Draw on existing knowledge: Taking advantage of prior art. In M. Fosmire & D. Radcliffe (Eds.), Integrating information into the engineering design process (pp. 125–135). Purdue University Press. http://docs.lib.purdue.edu/purduepress_ebooks/31

Denick, D., Bhatt, J., & Layton, B. (2010, June 20-23). Citation analysis of engineering design reports for information literacy assessment [Paper presentation]. 2010 ASEE Annual Conference & Exposition, Louisville, KY, United States. https://doi.org/10.18260/1-2--16508

Ekwaro-Osire, S., Afuh, I., & Orono, P. (2008, June 22-25). Information gathering activities in engineering design [Paper presentation]. 2008 ASEE Annual Conference & Exposition, Pittsburgh, PA, United States. https://doi.org/10.18260/1-2--4242

European Patent Office. (n.d.-a). Prior art searching. https://www.epo.org/learning/materials/inventors-handbook/novelty/searching.html

European Patent Office. (n.d.-b). What is prior art? https://www.epo.org/learning/materials/inventors-handbook/novelty/prior-art.html

Jeffryes, J., & Lafferty, M. (2012). Gauging workplace readiness: Assessing the information needs of engineering co-op students. Issues in Science and Technology Librarianship, 69. DOI: https://doi.org/10.5062/F4X34VDR

Lutz, B., & Paretti, M. C. (2017). Exploring student perceptions of capstone design outcomes. International Journal of Engineering Education, 33(5), 1521–1533. https://www.ijee.ie/contents/c330517.html

McAdams, J., & Glauberman, J. (2017, June 24-28). Information literacy portfolio for curriculum mapping [Paper presentation]. 2017 ASEE Annual Conference & Exposition, Columbus, OH, United States. https://doi.org/10.18260/1-2--28528

Mercer, K., Weaver, K. D., & Stables-Kennedy, A. J. (2019, June 15-19). Understanding undergraduate engineering student information access and needs: Results from a scoping review [Paper presentation]. 2019 ASEE Annual Conference & Exposition, Tampa, FL, United States. https://doi.org/10.18260/1-2--33485

Moody, N., Fouch, K., Kelley, T., Purzer, S., & Fosmire, M. (2012, March 17). Innovation differentiation: Examining the problem-solving approaches of engineering and technologist students [Paper presentation]. American Society for Engineering Education IL/IN Sectional Conference, Valparaiso, IN, United States. http://ilin.asee.org/Conference2012/Papers/Fosmire.pdf

Mosberg, S., Adams, R., Kim, R., Atman, C., Turns, J., & Cardella, M. (2005, June 12-15). Conceptions of the engineering design process: An expert study of advanced practicing professionals [Paper presentation]. 2005 ASEE Annual Conference & Exposition, Portland, OR, United States. https://doi.org/10.18260/1-2--14999

Nazemetz, J., Rossler, P., High, M., & High, K. (2007, June 24-27). Why reinvent the wheel? The U.S. Patent and Trademark Office as a design tool [Paper presentation]. 2007 ASEE Annual Conference & Exposition, Honolulu, HI, United States. https://doi.org/10.18260/1-2--2284

Osif, B. (2014). Make it safe and legal: Meeting broader community expectations. In M. Fosmire & D. Radcliffe (Eds.), Integrating information into the engineering design process (pp. 115–124). Purdue University Press. https://docs.lib.purdue.edu/purduepress_ebooks/31/

Phillips, M., Fosmire, M., & McPherson, P. B. (2018). Standards are everywhere: A freely available introductory online educational program on standardization for product development. Standards Engineering, 70(3), 1–6.

Phillips, M., Fosmire, M., Turner, L., Petersheim, K., & Lu, J. (2019). Comparing the information needs and experiences of undergraduate students and practicing engineers. The Journal of Academic Librarianship, 45(1), 39–49. https://doi.org/10.1016/j.acalib.2018.12.004

Phillips, M., Van Epps, A., Johnson, N., & Zwicky, D. (2018). Effective engineering information literacy instruction: A systematic literature review. The Journal of Academic Librarianship, 44(6), 705–711. https://doi.org/10.1016/j.acalib.2018.10.006

Phillips, M., & Zwicky, D. (2017). Patent information use in engineering technology design: An analysis of student work. Issues in Science and Technology Librarianship, 87. https://doi.org/10.5062/f4zs2tr8

Prabha, C., Silipigni Connaway, L., Olszewski, L., & Jenkins, L. R. (2007). What is enough? Satisficing information needs. Journal of Documentation, 63(1), 74–89. https://doi.org/10.1108/00220410710723894

Sult, L., Mery, Y., Blakiston, R., & Kline, E. (2013). A new approach to online database instruction: Developing the Guide on the Side. Reference Services Review, 41(1), 125–133. https://doi.org/10.1108/00907321311300947

Tenopir, C., & King, D. W. (2004). Communication patterns of engineers. John Wiley & Sons.

U.S. Patent and Trademark Office. (n.d.). Patent searching and search resources: An introduction. https://www.uspto.gov/sites/default/files/documents/Basics-of-Prior-Art-Searching.pdf

Wertz, R., Ross, M. C., Fosmire, M., Cardella, M. E., & Purzer, S. (2011, June 26-29). Do students gather information to inform design decisions? Assessment with an authentic design task in first-year engineering [Paper presentation]. 2011 ASEE Annual Conference & Exposition, Vancouver, BC, Canada. https://doi.org/10.18260/1-2--17789

Zabihian, F., Strife, M. L., & Armour-Gemmen, M. G. (2015). Integration of information literacy skills to mechanical engineering capstone projects [Paper presentation]. 2015 ASEE Annual Conference & Exposition, Seattle, WA, United States. https://doi.org/10.18260/p.24335

Zhang, Q., & Kozak, K. A. (2017, June 24-28). Watch it or read it: Understanding undergraduate engineering students’ learning effectiveness and preference for video tutorials versus Guide-on-the-Side tutorials [Paper presentation]. 2017 ASEE Annual Conference & Exposition, Columbus, OH, United States. http://doi.org/10.18260/1-2--29111

Appendix. Grading Rubric for Prior Art Review Deliverables

Criterion	Percentage Points (%)	Did Not Complete – 0	Well Below Expectations – 1	Below Expectations – 2	Meets Expectations – 3	Exceeds Expectations – 4
Tutorial Completed > 75% grade – reduces grade by 10 percentage points for team members that don't complete on time.	–10	< 75% or did not complete prior to lecture	N/A	N/A	N/A	Completed prior to Prior Art Lecture.
Search
Search Process 1 – Evaluation of the library tutorials’ and tools’ effectiveness: new/familiar, clear/unclear. 2 – Description of the group’s organization and implementation of the search process: who searched, which resources/websites, evaluation of results, selection of sources. 3 – Description of how the group resolved challenges, questions, or difficulties. 4 – Give a brief evaluation of the success of your search process and results.	20	No description of the search process	Multiple items absent. The report indicates an inadequate search process, no planning, and/or no reflection.	1 or 2 items absent. The report indicates an inadequate search process, limited planning, and/or minimal reflection.	All items present. The report indicates an appropriate search process, some planning, and some reflection.	All items present. The report indicates an appropriate and effective search process, planning and effective teamwork, and reflection.
Rationale for concluding the search process 1 – Explanation of the reasons for concluding the search process. 2 – Possible directions for further research.	10	No rationale provided	Rationale is incoherent and/or possible further directions are absent.	Rationale is somewhat unclear or unconvincing, and/or possible further directions are absent.	Rationale is mostly reasonable. Possible further directions included.	Rationale is clear and logical. Possible further directions included.
Search log included 1 – Search terms used (keywords, controlled terms, CPC patent classes/subclasses). 2 – Searches of all appropriate types of information (journal articles, conference papers, patents, standards, products). 3 – Contains date, who did the search, what you were looking for and where you looked.	10	No search log provided	Log is hard to read and/or is missing several significant items.	Log is somewhat unclear and/or is missing some significant items.	Log is clear, contains most required items, indicates an extensive search process.	Log is clear, contains all required items, indicates a comprehensive search process.
Discussion of prior art
Prior Art Summary 1 – Summarize related article, patent, product or standard 2 – Discuss what you learned and how it applies to your design problem. 3 – A clear link to the citation in the References section.	20		Summary and explanation of relevance are unclear, lacking detail, and/or descriptive rather than analytical.	Summary and explanation of relevance are not entirely clear, lacking important details, and/or insufficiently analytical.	Summary and explanation of relevance are mostly clear, adequately detailed, and somewhat analytical.	Summary and explanation of relevance are clear, detailed, and analytical.
Novelty 1 – Discuss state of the art solutions to your problems based on your searches. 2 – Evaluate the project's novelty as compared to prior art. 3 – Discuss how what you learned may influence your design considerations.	20		Evaluation of novelty indicates weak analysis.	Evaluation of novelty indicates limited analysis.	Evaluation of novelty demonstrates reasonable analysis and judgment.	Evaluation of novelty demonstrates good analysis and judgment.
References 1 – In-text citations and a full “References” section. 2 – Citations contain sufficient information to be looked up. 3 – Consistent style (eg. IEEE, APA, Nature, etc).	5		Many pieces of prior art lack citations and/or many citations lack significant information.	Some pieces of prior art lack citations and/or several citations lack significant information.	All pieces of prior art are cited, most with complete information, style is consistent.	All pieces of prior art are cited with complete information and consistent style.
Professionalism
Research meeting preparation: 1 – Team has evidence of a preliminary search 2 – Team identified questions or information gaps	5	One or more items not evident	N/A	N/A	N/A	Both items evident at meeting.
Professional writing skills: 1 – Includes only relevant information 2 – Concise presentation of information 3 – No obvious spelling or grammar errors 4 – Proper use of visuals 5 – All writing in 3rd person	5	One or more items not evident	N/A	N/A	N/A	All items met in assignment
Professional Presentation: 1 – Cover page with team information, revision number and title 2 – Table of contents 3 – Consistent styles, heading numbering and form 4 – Pages numbered 5 – Dynamic links to figures, tables etc. 6 – Figure/charts labeled and numbered 7 – External references cited with links to endnotes	5	One or more items not evident	N/A	N/A	N/A	All items present in assignment.

This work is licensed under a Creative Commons Attribution 4.0 International License.