User Testing with Microinteractions: Enhancing a Next-Generation Repository COMMUNICATION User Testing with Microinteractions Enhancing a Next Generation Repository Sara Gonzales, Matthew B. Carson, Guillaume Viger, Lisa O'Keefe, Norrina B. Allen, Joseph P. Ferrie, and Kristi Holmes INFORMATION TECHNOLOGY AND LIBRARIES | MARCH 2021 https://doi.org/10.6017/ital.v40i1.12341 Sara Gonzales (sara.gonzales2@northwestern.edu) is Data Librarian, Galter Health Sciences Library & Learning Center, Northwestern University Feinberg School of Medicine. Matthew B. Carson (matthew.carson@northwestern.edu) is Head, Digital Systems/Senior Research Data Scientist, Galter Health Sciences Library & Learning Center, Northwestern University Feinberg School of Medicine. Guillaume Viger (guillaume.viger@northwestern.edu) is Senior Developer, Galter Health Sciences Library & Learning Center, Northwestern University Feinberg School of Medicine. Lisa O’Keefe (lisa.okeefe@northwestern.edu) is Senior Program Administrator, Galter Health Sciences Library & Learning Center, Northwestern University Feinberg School of Medicine. Norrina B. Allen (norrina-allen@northwestern.edu) is Associate Professor of Preventive Medicine (Epidemiology) and Pediatrics, Northwestern University Feinberg School of Medicine. Joseph P. Ferrie (ferrie@northwestern.edu) is Professor and Department Chair of Economics, Northwestern University. Kristi Holmes (kristi.holmes@northwestern.edu) is Director, Galter Health Sciences Library & Learning Center, and Professor of Preventive Medicine (Health and Biomedical Informatics) and Medical Education at Northwestern University Feinberg School of Medicine. © 2021. ABSTRACT Enabling and supporting discoverability of research outputs and datasets are key functions of university and academic health center institutional repositories. Yet adoption rates among potential repository users are hampered by a number of factors, prominent among which are difficulties with basic usability. In their efforts to implement a local instance of InvenioRDM, a turnkey next generation repository, team members at Northwestern University’s Galter Health Sciences Library & Learning Center supplemented agile development principles and methods and a user experience design-centered approach with observations of users’ microinteractions (interactions with each part of the software’s interface that requires human intervention). Microinteractions were observed through user testing sessions conducted in Fall 2019. The result has been a more user-informed development effort incorporating the experiences and viewpoints of a multidisciplinary team of researchers spanning multiple departments of a highly ranked research university. INTRODUCTION Galter Health Sciences Library & Learning Center facilitates and supports the discoverability of knowledge for the faculty, students, and staff of the Feinberg School of Medicine at Northwestern University. As an integrated unit in Northwestern University’s Clinical and Translational Sciences Institute (NUCATS) and a key partner to other institutes across Northwestern University’s two campuses, enabling maximum ease of use of library resources and support for meaningful information discoveries for researchers at all stages has been a prime motivator. These motivators helped drive the selection and development of an upgraded institutional repository infrastructure at Galter, a project which began in 2018. Discovery of resources through repository tools depends upon many factors: metadata and controlled vocabularies used, storage and retrieval capacity, and familiarity and comfort level with mailto:sara.gonzales2@northwestern.edu mailto:matthew.carson@northwestern.edu mailto:guillaume.viger@northwestern.edu mailto:lisa.okeefe@northwestern.edu mailto:norrina-allen@northwestern.edu mailto:ferrie@northwestern.edu mailto:kristi.holmes@northwestern.edu INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2021 USER TESTING WITH MICROINTERACTIONS | GONZALES, CARSON, VIGER, O’KEEFE, ALLEN, FERRIE, AND HOLMES 2 the tool on the part of researchers and students. Is the institutional repository link easy to find on the website? More importantly, is it easy to use? Can records be created and files uploaded with ease? Do searches bring meaningful results, and can they be filtered and organized for maximum impact? From early on in the repository upgrade project, Galter Library partnered with Northwestern’s Institute for Innovations in Developmental Sciences (DevSci), both to answer these questions and to find practical ways to serve researchers who aimed to discover relevant datasets through a repository. The work of the interdisciplinary DevSci group is focused on human development across the lifespan in all areas, including physical, emotional, psychological, and socioeconomic, providing a multidisciplinary perspective for the collaboration. Through this partnership, DevSci’s goal was to develop a data repository or index through which they could discover the datasets of their fellow researchers and find new collaborators, thus providing an ideal perspective from which to provide critical feedback. Galter Health Sciences Library & Learning Center selected the InvenioRDM (Research Data Management) extensible institutional repository (IR) platform as its local IR code upgrade. InvenioRDM is a Python-based, modular and scalable IR developed by CERN (the European Organization for Nuclear Research) and collaborators.1 The first version of the Invenio framework was developed in 2000. In 2018, Invenio 3.02 was released with significantly improved software and code rewritten to make it a modular framework. This new framework now serves as a foundation for modern research data management and scholarly communications through a trusted digital repository. InvenioRDM is being collaboratively developed, with its many partners and robust developer community ensuring the framework’s maintenance, improvement, and preservation capacity into the foreseeable future. To Galter Library and its partners at DevSci, the development timespan required to build a local instance of InvenioRDM presented the perfect opportunity to address one of the major stumbling blocks of IR adoption: the user experience. If a repository were designed with users’ needs in mind, and took into account their behaviors and interactions with every aspect of the tool, it had the potential to increase adoption and usability far beyond numbers generally observed for university IRs. Designing for user behaviors was the goal of Galter Library’s repository development team as we launched a round of user testing of the repository’s alph a version in Fall 2019. LITERATURE REVIEW It is an exciting time for IRs serving researchers in the sciences and particularly in the field of biomedical research. New robust repository frameworks capable of storing and preserving data for decades into the future are being developed to meet widely articulated researcher needs, including those user behaviors and technologies highlighted by the Confederation of Open Access Repositories (COAR) in detailed guidelines for next generation repository features.3 These features include interoperable resource transfer, metadata-enhanced discovery by navigation, and exposure of permanent identifiers. Researcher-focused organizations such as COAR endorse and promulgate FAIR principles to make deposited and shared data findable, accessible, interoperable, and reusable.4 Meanwhile, federal agencies such as the National Institutes of Health are increasingly incorporating policies to encourage best practices for data management and data sharing of grant funded research.5 These policies often recommend depositing data in a robust, secure, and accessible IR which can be maintained by the researchers’ own institution. In recent years, the majority of the deposited products of research have been stored in subject repositories INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2021 USER TESTING WITH MICROINTERACTIONS | GONZALES, CARSON, VIGER, O’KEEFE, ALLEN, FERRIE, AND HOLMES 3 or made available via social media platforms which carry no guarantee of long-term curation and preservation of shared resources. This happens even though institutional repositories are well- represented on the overall repository landscape, suggesting that these institutional assets are underutilized in critical data workflows.6 The reasons for slow adoption and use of institutional repositories (IR) by researchers in routine workflows are many: IRs can be perceived as adding to researchers’ administrative work burden through the need to clean, deposit, and catalog data and other research outputs; many feel trepidation about open science practices and their effects on citation counts; and researchers may feel unsure about copyright restrictions on materials they might deposit.7 Narayan and Luca, in their study of one university’s IR adoption challenges, outline some of the deep-seated motivations behind this trepidation, such as the social and psychological barriers imposed by researchers’ own, and university-encouraged, traditional views of scholarly publishing, as well as the ways in which these views are heavily supported by university systems in their tenure and promotion policies. In addition, many researchers perceive the content contained in IRs as restricted or of limited use compared to the volume of resources that can be found through a Google search.8 The IR is often seen as a small island within the larger digital research landscape. The degree to which repository managers are attuned to their local users’ professional and personal needs with regard to a repository will have a large impact on adoption rates among hesitant user populations. As Witt and Betz & Hall point out, professional motivators have arisen in multiple disciplines to deposit in IRs not only preprints, but datasets, data dictionaries, README files, and other reproducibility-supporting resources, in order to provide open access to the products of federally funded research.9 Building on funder and publisher mandates for making both publications and datasets open access, IR builders and maintainers can employ various methods to increase the motivation momentum towards IR adoption. They can highlight repository champions, faculty users of IR tools who can provide use cases and success stories about the benefits the IR brings to them, as both depositors and searchers.10 They can help to allay fears and confusion around depositors’ rights with regard to deposited materials by carefully explicating license types and definitions and by consulting with researchers on the correct licenses to choose for their deposits. They can work with their repository’s developers to create value-added modifications to the repository, including user-friendly browsing, featured collections, and researcher pages, which highlight the most current research at the institution.11 Importantly, if the IR maintainers are able to modify the repository’s interface to suit local needs, they can help ensure that the majority of users have a positive experience with the repository’s interface, one in which every interaction is intuitive and in which there are no wasted steps or unnecessary clutter. Such usability can be achieved through examining repository users’ microinteractions, that is, interactions with each small part of the software’s interface that requires human intervention. For those engaged in library technology projects in recent years, user experience (UX) design will be a familiar concept. UX design seeks to make a user’s interaction with a product—often a web- based tool—easier and more intuitive, frequently through manipulating the behavior of the user.12 However, in the recent trend toward designing with a focus on microinteractions, software developers are influenced by the data they glean from observing users’ interactions with each part of the software’s interface that requires human intervention, noting from the users themselves the intuitive and non-intuitive parts of an interaction in order to determine where changes should be made.13 The development team for InvenioRDM has taken an approach that combines traditional INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2021 USER TESTING WITH MICROINTERACTIONS | GONZALES, CARSON, VIGER, O’KEEFE, ALLEN, FERRIE, AND HOLMES 4 user-experience design based around collaborative, open source code and tools and common dataset metadata standards such as DataCite (https://schema.datacite.org); observations from Invenio’s 20 years of serving as CERN’s IR; and examination of microinteractions at certain key stages of development. These microinteractions revolve around common user actions within an IR, including depositing items, searching, browsing, and creating a user account.14 The result of combining these approaches has allowed Galter Health Sciences Library & Learning Center to put users’ needs at the forefront of its new IR. INVENIORDM: A NEXT-GENERATION REPOSITORY Galter Library’s selection of a new IR solution was carefully considered and motivated by the organization’s need for a robust, forward-looking, and feature-rich repository that could support best practices in research data management and sharing as realized through the Invenio framework. The Python framework incorporates community-built Python libraries, while also leveraging Flask, a PostgreSQL or MySQL back-end database, the React JS user interface, and the extremely fast Elasticsearch JSON-native distributed search engine. The resulting tool is eminently scalable, securely housing petabytes’ worth of easily discoverable records. Galter Library began our collaboration with CERN to build a local instance of InvenioRDM, while contributing to the overall repository source code, in late 2018. Since that time, a local developer has worked on the code and contributed to the repository’s project roadmap, updating GitHub issues and pushing releases.15 A project manager, data librarian, and the library leadership have also been involved throughout the project in the areas of general guidance and management, oversight, assessment, dissemination and outreach, and requirements gathering. Many requirements were gleaned from the DevSci community through conversations and informal interviews around data storage practices. In early 2019, to ensure that the repository was meeting the initially envisioned requirements, the Galter Library repository team analyzed the requirements thus far gathered for the project, which had been translated into GitHub issues and added to by team members and collaborators. The requirements gathered from DevSci collaborators and Galter librarians were found to map directly into key IR functional categories outlined separately by IR stakeholders throughout the globe such as the National Institutes of Health, the Confederation of Open Access Repositories, the Digital Repository of Ireland, the Department of Computer and Information Sciences at Covenant University, Nigeria, and others. Those requirements included record creation and ingest, robust metadata for accessing a record, user account and permissions, user authentication, search functionality, resource access/download, and community pages and features (see fig. 1).16 https://schema.datacite.org/ INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2021 USER TESTING WITH MICROINTERACTIONS | GONZALES, CARSON, VIGER, O’KEEFE, ALLEN, FERRIE, AND HOLMES 5 Figure 1. Local repository requirements mapped to IR functional categories. With the repository’s key functional requirements defined, the development team needed real- world data to help inform the microinteractions that would bring the functions to life, both for repository managers and users. To acquire microinteraction data, Galter Library’s data librarian designed and organized a round of user testing of the alpha release of the repository in autumn 2019. The alpha release of InvenioRDM was completed by September 2019, meeting a deadline established by one of the project’s key funders, the National Center for Data to Health (CD2H), INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2021 USER TESTING WITH MICROINTERACTIONS | GONZALES, CARSON, VIGER, O’KEEFE, ALLEN, FERRIE, AND HOLMES 6 through a grant funded by the National Center for Advancing Translational Sciences (NCATS). This early alpha release enabled record creation and file upload, application of seven metadata elements (title, authors, description, resource type, subjects, visibility, and license), user authentication, search, faceting/filtering, and download of resources. To make the experience of searching the alpha release repository as realistic as possible, the data librarian asked colleagues from DevSci to provide data for seed records for the repository, based on their own research. Colleagues willingly obliged, and over a dozen seed records based on real- world clinical studies and other studies focused on human development were created in early October 2019. While conducting email and word-of-mouth recruitment with members of the DevSci community, the data librarian worked on a testing script designed to require the maximum amount of microinteractions possible as each user worked with the repository. The script asked users to complete the following list of tasks (see fig. 2), while thinking aloud and noting anything that they found unusual or anything that they would have expected to see in the user interface of the repository. Figure 2. IR user testing script tasks. The user testing tasks conform to many of the functional requirements for institutional repositories identified from our requirements gathering (fig. 1), including user authentication and account, search functionality, resource access/download, record creation/ingest, and robust metadata for accessing a record (detailed record page). By October 2019, ten Northwestern University faculty members, mainly from DevSci, and two information professionals had agreed to test the alpha version of InvenioRDM. The data librarian arranged to securely host testing sessions through web conferencing software. Testers agreed to have the sessions recorded and shared their screens as they worked through the test scenarios, allowing the data librarian to observe their movements through the repository and to review the recordings later in case anything was missed. Testing sessions generally lasted between twenty and thirty minutes, although some lasted from forty-five minutes to one hour. After sessions were completed, the data librarian recorded in text documents a description of all microinteractions and verbal observations testers made about the repository. She later transferred that data to a spreadsheet, listing each criterion individually and manually adding a count of how many testers either reported or were observed to experience the same phenomenon INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2021 USER TESTING WITH MICROINTERACTIONS | GONZALES, CARSON, VIGER, O’KEEFE, ALLEN, FERRIE, AND HOLMES 7 (appendix 1). Reported phenomena and observed difficulties that users experienced in testing the repository were aggregated and included in the final reported data if at least two testers reported or had the same experience. Through these counts the data librarian was able to identify which microinteractions proved most challenging to the testers. DISCUSSION Manual, qualitative analysis of the user testing data revealed challenges that users faced with InvenioRDM that were best captured and expressed when observed as microinteractions. Though user experience design had been employed extensively in the design of the database, it was the nuances of the interactions that showed where improvements in the design could still be made. Almost every functional area of the repository demonstrated a need for increased user input in its design. The results of user exercises testing the various functional areas are described below. User Profile Screens Ease-of-use Exercise While most testers did not experience difficulties in locating the login button on the repository’s home page that allowed them to access the user profile portions of the site (9/12 located the button in less than three seconds), most testers (7/12) requested clearer instructions for which username and password to use (e.g., LDAP or Shibboleth-based credentials), and three-quarters of testers (9/12) inquired about where and how to add information about themselves to their profiles (e.g., professional title, contact information, department and other affiliations, etc.). While the required task consisted simply of logging successfully into one’s profile, the testers rightly discovered and acknowledged that the robust, CRIS (Current Research Information System)-like features of many repositories’ user profile pages had not yet been fully implemented in InvenioRDM. Finding Datasets Exercise The next task required testers to perform a search any way they liked in the repository, locate a dataset record, and download the associated data file. Whether searching using filters or by entering keywords to find a known item, users were always able to easily identify a data file within a record and download it. A special feature of InvenioRDM that occasionally made finding a data file to download challenging was that the repository was designed to serve as both a repository for digital files and a data index. The main feature of a data index is that it will store records representing datasets without necessarily storing the datasets themselves. The data index option is crucial for health sciences researchers, who are often motivated to share data files as openly as possible for motivations of reproducibility, open access to scientific data, and compliance with funder mandates, but who cannot always safely deposit data files due to the presence of personally identifiable information (PII) or protected health information as defined by HIPAA17 of the human subjects who are involved in their studies. This phenomenon spurred us to create seed records in the repository that represent real clinical studies, for which the data could be made available upon request from the researchers, but for which a data file was not uploaded to the repository. By following the protocol of making this clinical data public as safely as possible, these records were created and tagged with a visibility level of Open Access. Two of the testers stated that they believed that a record tagged as Open Access implied that a data file was available for download, and many others (10/12) expected either a visual cue or another filter to allow users to hone in on only the dataset record results that contained a deposited data file. INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2021 USER TESTING WITH MICROINTERACTIONS | GONZALES, CARSON, VIGER, O’KEEFE, ALLEN, FERRIE, AND HOLMES 8 Filter Searching Exercise Searching the repository, particularly via the filters, resulted in some of the most interesting results of the entire testing process. In the testing exercises, users were asked to search for anything they wanted, either narrowing their results from a direct term search or beginning a search from the full record set using the filter options on the left side of the screen. Opinions differed among the repository team members and some testers as to whether applying two filters at once to the results of a search would result in an AND or OR union of the two subsets of results. Another way to phrase this scenario would be, that upon the application of one filter, would the search results and other filters update in real time? For instance, if I filtered my search results to include only the deposits of one particular author, and if the file type filter choices still contained, after the filtering action, types including PDFs, XLSX files, DOC files, MOV files, etc., is it safe to assume that my chosen author deposited all those types of files? (See fig. 3.) Or do the filters behave independently of each other? One third of the testers (4/12) said they expected the filtering choices to update in real time and that the application of two filters should result in an AND union of results. Figure 3. Filters available in InvenioRDM. Seven of the twelve testers said that the most helpful of the of the five filters available in the repository at the time were resource type, file type, and subjects, while slightly less found author and license filters helpful. Three of the twelve asserted that it would not occur to them to filter on a resource’s license. INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2021 USER TESTING WITH MICROINTERACTIONS | GONZALES, CARSON, VIGER, O’KEEFE, ALLEN, FERRIE, AND HOLMES 9 Specific Record Search Exercise In the specific record search, users were asked to find a specific paper by a particular author. None of the testers experienced any difficulties or significant delays in bringing up the requested record, which served as a testament both to their searching abilities and to the robustness of the Elasticsearch framework utilized in the repository’s infrastructure. Ten of the twelve testers mentioned that they found the preformatted citations available with each record helpful, and two of these testers requested an easy way to export the citation in their desired format to EndNote. Three of the twelve testers experienced a brief initial delay in locating the known record because they still had filters applied in the repository from a previous search. They requested an easy way to clear all previous filters when starting a new search. Creating a Record Exercise As one of the larger user testing tasks, users were provided with a dummy file representing a dataset and asked to deposit and describe it with appropriate metadata using the repository’s cataloging form. The first part of this task involved finding the button that brings the user to the cataloging form, a button which is available from several places in the InvenioRDM layout (home page, search results page, and profile page). Eight of the twelve testers took longer than three seconds to locate this button, termed the “Catalog your research” button. Two of these eight reported that “catalog” was not the verb they would associate with depositing and describing a data file; to some, “catalog” seemed a library-centric term. On the repository’s record creation page, a space exists to either upload or drag and drop a file, and when this task is done, a large, blue “Start upload” button appears that the user must click to begin the file upload. (See fig. 4.) Yet despite its size and color, almost half the testers (5/12) did not notice that they had to click it in order to complete the upload of their file and, worse, they often completed the record creation process and published their record without noticing that the file had not uploaded. Visual cues were needed to confirm for the user whether a file was successfully uploaded or not. In addition, automatic upload upon browsing and attaching a file or dragging and dropping a file was reported as an expected behavior by many users. INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2021 USER TESTING WITH MICROINTERACTIONS | GONZALES, CARSON, VIGER, O’KEEFE, ALLEN, FERRIE, AND HOLMES 10 Figure 4. Users often missed the blue “Start upload” button just beneath the file name. INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2021 USER TESTING WITH MICROINTERACTIONS | GONZALES, CARSON, VIGER, O’KEEFE, ALLEN, FERRIE, AND HOLMES 11 Most users applied descriptive metadata successfully and easily, but some experienced trouble while appending subject metadata to describe the subject matter of their deposits. As the repository is being customized for a health sciences library, subject fields are offered in InvenioRDM to allow appending both Medical Subject Heading (MeSH) terms and terms from Faceted Application of Subject Terminology (FAST), a vocabulary derived from the Library of Congress Subject Headings (LCSH). Since the two vocabularies’ terms are offered in separate fields (FAST serving as a more universal set of terms to complement the biomedically oriented MeSH), users became confused, not knowing which, if either, subject field they should complete. A solution involving a single subject field that queries both the MeSH and FAST APIs is warranted in order to simplify the subject-tagging experience for the user. Editing a Record Exercise Editing the record they had just created, the final testing task, proved to be unproblematic for testers. Eleven of the twelve found the Edit button on their record pages in less than three seconds, and the editing process was reported to be straightforward. One tester observed that they were unable to change their file (i.e., make a version-level change to their record), but this was only because version change functionality had not yet been implemented in the repository. Reflection/Unguided Feedback Once all the testing exercises outlined above were completed, testers were asked to talk about the uses for which they might employ a tool like InvenioRDM. Without prompting or suggestions of uses, the testers overwhelmingly stated that they would use the completed repository for the very functions for which most IRs are built: storing data files (two users), sharing data for open science or to fulfill funders’ mandates (two users), searching for others’ datasets (four users), creating gray literature collections to showcase their conference presentations and posters (three users), embedding repository-issued DOIs from their datasets into manuscripts and posters (three users), and storing data in private collections to share with trusted collaborators (four users). This data shows that university faculty and researchers have various specific needs for repository solutions, which can be met if the repositories are designed with these needs in mind. AFTER TESTING: NEXT STEPS The user testing experience for InvenioRDM proved to be a highly enjoyable process for all involved. The tester participants expressed enthusiasm in the process and appreciated the opportunity to share their ideas about the functionality of an IR while it was still in the design stages. The testers’ enthusiasm reinforced the notion that many university faculty members are eager for an intuitive, user-friendly tool that will allow them to store, retrieve, and share their research outputs, as long as the tool is designed with their needs in mind. Observation of testers’ microinteractions with Galter Library’s new institutional repository has helped the local development team to better understand what those needs are. The results of the user testing were presented at the InvenioRDM product meeting at CERN in January 2020. The results were well received and resulted in immediate adjustments to the repository’s development. As development continues into 2021, the repository team at Galter Health Sciences Library & Learning Center will design and manage at least one additional user testing round to ensure that the repository continues to meet its goals of serving key functional requirements of IRs while also providing users the best possible experience through each interaction they have with the tool. As user testing sessions demonstrate, there is much room for growth in the achievement of a truly intuitive interface design in even some of the seemingly INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2021 USER TESTING WITH MICROINTERACTIONS | GONZALES, CARSON, VIGER, O’KEEFE, ALLEN, FERRIE, AND HOLMES 12 simplest functions of the repository, such as intuitively placing a deposit button or honing in on the right combination and placement of filters. The Galter Library development team is committed to continuing to work toward an intuitive and seamless user experience. On this journey the repository team acknowledges and thanks the testers and future users of its repository and the researchers and support staff for whom the tool is being built and without whom it could not be built half as well. ACKNOWLEDGEMENTS The project team would like to acknowledge Northwestern University’s Institute for Innovations in Developmental Sciences and the Northwestern University Clinical and Translational Sciences Institute (NUCATS). InvenioRDM project team members Sara Gonzales, Guillaume Viger, Matthew B. Carson, Lisa O’Keefe, and Kristi Holmes were partially funded by the CTSA Program National Center for Data to Health, Grant U24TR002306 and NUCATS, Grant UL1TR001422. APPENDIX 1. INVENIORDM USER TESTING AGGREGATE DATA, DIVIDED BY TASK Reporting criteria A phenomenon or observation was noted if it was reported by, or observed in the behavior of, two or more testers. For four tasks seconds were counted as a mark of how easy it was to find a repository element that enabled the task: 1. Finding the login button to access user account 2. Finding the Citation button after the search for a specific record 3. Finding the "Catalog your research" button 4. Finding the "Edit record" button. Counting of seconds was done with an iPhone stopwatch while reviewing the recorded sessions. If finding the required button took the user longer than a generous count of three seconds, it was deemed that the user had a hard time locating the item. User Profile screens results 9/12 testers wanted to add information about themselves and their appointment (dep artment, title, contact information, etc.) 7/12 wanted clearer instructions for the username and password they use to log in 3/12 testers took three seconds or more to locate the user login button on the home page Finding Datasets exercise results 10/12 testers expected a (sortable, filterable) cue on the search results screen to show whether record has a file to download 3/12 testers wanted grayed out instructions or search tips/suggestions in the search box 2/12 testers believed that the Open Access pill in search results implied there would be a file to download INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2021 USER TESTING WITH MICROINTERACTIONS | GONZALES, CARSON, VIGER, O’KEEFE, ALLEN, FERRIE, AND HOLMES 13 2/12 testers believed that the subject pills in Full Record view should be clickable to enable direct search on the subjects Filter Searching exercise results 7/12 testers said the most helpful filters were resource type, file type, and subjects, followed by author, then license 4/12 testers expected filter choices to update in real time based on initial filter chosen 3/12 testers expected option to expand beyond the top 10 authors in the authors filter 3/12 testers were not familiar with the choices of MeSH and FAST terms 3/12 testers would not think of filtering on license 2/12 testers expected guidance on the licenses' meanings if browsing/filtering by license is offered 2/12 testers expected greater filter collapsing/expanding options than what was offered 2/12 testers expected to apply two filters at once 2/12 testers wanted to filter on sample or demographic information of study subjects Specific Record Search exercise results 10/12 testers found the preformatted citations helpful 7/12 testers found the citation button in less than three seconds 3/12 testers had trouble with their known-record searching because filters were on when they started; needed an easy way to clear all filters 2/12 testers expected an option to download the found record’s citation to EndNote Creating a Record exercise results 8/12 testers took longer than three seconds to find the “Catalog your research” button; of those two would have used a different phrase [“’catalog’ is too library-centric”] 5/12 testers did not see “Start upload” button after dropping their files, and an additional two said they expected auto-upload immediately upon dropping their files, with no “Start upload” button necessary 3 of the 5 testers who missed the “Start upload” button did not notice that their file did not get saved to their record 5/12 testers did not notice at first that the “Save draft” step was needed before clicking Publish, and one additional tester said they expected record auto-save, which would help in filling out a longer record 5/12 testers wanted guidance on which license to choose 4/12 testers expected some kind of instructions for filling out the cataloging page, even if only for specific fields like description or title INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2021 USER TESTING WITH MICROINTERACTIONS | GONZALES, CARSON, VIGER, O’KEEFE, ALLEN, FERRIE, AND HOLMES 14 3/12 testers found the resource type interface intuitive 3/12 testers thought the arrow in the MeSH (subject) field implied the availability of a drop-down list of options 2/12 testers did not see the drop-down choices under resource type umbrella categories at first 2/12 testers expected terms entered in the MeSH (subject) fields to stay there, or a warning that they will disappear if there is no match 2/12 testers wanted more guidance on choosing visibility level (private, public, etc.) 2/12 testers wanted more definitions/assistance about difference between medical and topical subject terms 2/12 testers wanted more definitions/assistance with FAST terms 2/12 testers said they would prefer a default license option Editing a Record exercise results 11/12 testers found the Edit button in less than three seconds Reflection/unguided feedback results 4/12 testers would use the repository to search for data 4/12 testers would store data files in private collections to be shared only with trusted collaborators 3/12 would embed the repository-issued DOIs from their datasets into their manuscripts and papers 3/12 testers would create their own grey literature collections of conference abstracts and posters 2/12 testers would use the repository for storing data files 2/12 testers would use the repository for open access/open science/data sharing complian ce motivations INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2021 USER TESTING WITH MICROINTERACTIONS | GONZALES, CARSON, VIGER, O’KEEFE, ALLEN, FERRIE, AND HOLMES 15 REFERENCES 1 “InvenioRDM: the turn-key research data management repository,” CERN (European Organization for Nuclear Research), accessed March 11, 2020, https://invenio- software.org/products/rdm/. 2 Lars Holm Nielsen, “Invenio v3.0.0 Released,” Invenio Blog (blog), Invenio, June 7, 2018, https://invenio-software.org/blog/invenio-v300-released/. 3 “Next Generation Repositories: Behaviours and Technical Recommendations of the COAR Next Generation Repositories Working Group,” Confederation of Open Access Repositories (COAR), November 28, 2017, https://www.coar-repositories.org/files/NGR-Final-Formatted-Report- cc.pdf. 4 “The FAIR Data Principles,” FORCE11, accessed March 11, 2020, https://www.force11.org/group/fairgroup/fairprinciples. 5 National Institutes of Health, “Final NIH Policy for Data Management and Sharing,” NIH Office of Extramural Research, accessed January 15, 2021, https://grants.nih.gov/grants/guide/notice- files/NOT-OD-21-013.html. 6 Gary E. Gorman, Jennifer Rowley, and Stephen Pinfield, “Making Open Access Work: The ‘State- of-the-Art’ in Providing Open Access to Scholarly Literature,” Online Information Review 39, no. 5 (September 2015): 604–36. 7 Bhuva Narayan and Edward Luca, “Issues and Challenges in Researchers’ Adoption of Open Access and Institutional Repositories: A Contextual Study of a University Repository,” in Proceedings of RAILS – Research Applications, Information and Library Studies, 2016, School of Information Management, Victoria University of Wellington, New Zealand, 6–8 December (2016); Information Research: An International Electronic Journal, 22, no. 4 (December 2017), http://hdl.handle.net/10453/121438. 8 Beth St. Jean, Soo Young Rieh, Elizabeth Yakel, and Karen Markey, “Unheard Voices: Institutional Repository End-users,” College & Research Libraries 72, no. 1 (January 2011): 21–42. 9 Michael Witt et al., “Connecting Researchers to Data Repositories in the Earth, Space, and Environmental Sciences,” in Digital Libraries: Supporting Open Science, IRCDL 2019, ed. Leonardo Candela and Gianmaria Silvello (2019); Communications in Computer and Information Science 988, 86–96; Sonya Betz and Robyn Hall, “Self-archiving with Ease in an Institutional Repository: Microinteractions and the User Experience,” Information Technology and Libraries 34, no. 3 (September 2015): 43–58. 10 Betz and Hall, “Self-archiving with Ease,” 43–58. 11 St. Jean, Rieh, Yakel, and Markey, “Unheard Voices,” 21–42. 12 “User Experience Design,” Wikipedia, last modified January 12, 2021, https://en.wikipedia.org/wiki/User_experience_design. 13 Betz and Hall, “Self-archiving with Ease,” 43–58. https://invenio-software.org/products/rdm/ https://invenio-software.org/products/rdm/ https://invenio-software.org/blog/invenio-v300-released/ https://www.coar-repositories.org/files/NGR-Final-Formatted-Report-cc.pdf https://www.coar-repositories.org/files/NGR-Final-Formatted-Report-cc.pdf https://www.force11.org/group/fairgroup/fairprinciples https://grants.nih.gov/grants/guide/notice-files/NOT-OD-21-013.html https://grants.nih.gov/grants/guide/notice-files/NOT-OD-21-013.html http://hdl.handle.net/10453/121438 https://en.wikipedia.org/wiki/User_experience_design INFORMATION TECHNOLOGY AND LIBRARIES MARCH 2021 USER TESTING WITH MICROINTERACTIONS | GONZALES, CARSON, VIGER, O’KEEFE, ALLEN, FERRIE, AND HOLMES 16 14 A. O. Adewumi, N. A. Omoregbe, and Sanjay Misra, “Usability Evaluation of Mobile Access to Institutional Repository,” International Journal of Pharmacy and Technology 8, no. 4 (December 2016): 22892–905. 15 “InvenioRDM Project Roadmap,” CERN (European Organization for Nuclear Research), accessed March 17, 2020, https://invenio-software.org/products/rdm/roadmap/. 16 National Institutes of Health, Office of the Director, “Supplemental Information to the NIH Policy for Data Management and Sharing: Selecting a Repository for Data Resulting from NIH- Supported Research,” last modified October 29, 2020, https://grants.nih.gov/grants/guide/notice-files/NOT-OD-21-016.html; “COAR Community Framework for Good Practices in Repositories, Public Version 1,” Confederation of Open Access Repositories (COAR), last modified October 8, 2020, https://www.coar- repositories.org/coar-community-framework-for-good-practices-in-repositories/; Sharon Webb and Charlene McGoohan, The Digital Repository of Ireland: Requirements Specification (National University of Ireland Maynooth, 2015), https://doi.org/10.3318/DRI.2015.6; Adewumi, Omoregbe, and Misra, “Usability Evaluation of Mobile Access to Institutional Repository,” 22892–905; Suntae Kim, “Functional Requirements for Research Data Repositories,” International Journal of Knowledge Content Development & Technology 8, no. 1 (March 2018): 25–36. 17 U.S. Department of Health and Human Services, “Summary of the HIPAA Security Rule,” last modified July 26, 2013, https://www.hhs.gov/hipaa/for-professionals/security/laws- regulations/index.html. https://grants.nih.gov/grants/guide/notice-files/NOT-OD-21-016.html https://www.coar-repositories.org/coar-community-framework-for-good-practices-in-repositories/ https://www.coar-repositories.org/coar-community-framework-for-good-practices-in-repositories/ https://doi.org/10.3318/DRI.2015.6 https://www.hhs.gov/hipaa/for-professionals/security/laws-regulations/index.html https://www.hhs.gov/hipaa/for-professionals/security/laws-regulations/index.html ABSTRACT Introduction Literature Review InvenioRDM: a Next-Generation Repository Discussion User Profile Screens Ease-of-use Exercise Finding Datasets Exercise Filter Searching Exercise Specific Record Search Exercise Creating a Record Exercise Editing a Record Exercise Reflection/Unguided Feedback After Testing: Next Steps Acknowledgements Appendix 1. InvenioRDM User Testing aggregate data, divided by task Reporting criteria User Profile screens results Finding Datasets exercise results Filter Searching exercise results Specific Record Search exercise results Creating a Record exercise results Editing a Record exercise results Reflection/unguided feedback results REFERENCES