Measuring Library Broadband Networks to Address Knowledge Gaps and Data Caps ARTICLE Measuring Library Broadband Networks to Address Knowledge Gaps and Data Caps Chris Ritzo, Colin Rhinesmith, and Jie Jiang INFORMATION TECHNOLOGY AND LIBRARIES | SEPTEMBER 2022 https://doi.org/10.6017/ital.v41i3.13775 Chris Ritzo, MSLIS (critzo@afutures.xyz) is Consultant/Owner, Anemophlious Futures LLC. Colin Rhinesmith, PhD (crhinesmith@metro.org) is Director, Digital Equity Research Center, Metropolitan New York Library Council. Jie Jiang, MSLIS (jie.jiang@simmons.edu) is a doctoral student at the Simmons University School of Library and Information Science. © 2022. ABSTRACT In this paper, we present findings from a three-year research project funded by the US Institute of Museum and Library Services that examined how advanced broadband measurement capabilities can support the infrastructure and services needed to respond to the digital demands of public library users across the US. Previous studies have identified the ongoing broadband challenges of public libraries while also highlighting the increasing digital expectations of their patrons. However, few large-scale research efforts have collected automated, longitudinal measurement data on library broadband speeds and quality of service at a local, granular level inside public libraries over time, including when buildings are closed. This research seeks to address this gap in the literature through the following research question: How can public libraries utilize broadband measurement tools to develop a better understanding of the broadband speeds and quality of service that public libraries receive? In response, quantitative measurement data were gathered from an open-source broadband measurement system that was both developed for the research and deployed at 30 public libraries across the US. Findings from our analysis of the data revealed that Ookla measurements over time can confirm when the library’s internet connection matches expected service levels and when they do not. When measurements are not consistent with expected service levels, libraries can observe the differences and correlate this with additional local information about the causes. Ongoing measurements conducted by the library enable local control and monitoring of this vital service and support critique and interrogation of the differences between internet measurement platforms. In addition, we learned that speed tests are useful for examining these trends but are only a small part of assessing an internet connection and how well it can be used for specific purposes. These findings have implications for state library agencies and federal policymakers interested in having access to data on observed versus advertised speeds and quality of service of public library broadband connections nationwide. INTRODUCTION The COVID-19 pandemic exposed the severity of the digital divide in the United States. During this time, lack of access to computers and the internet has been highlighted among individuals and families with limited monthly incomes in tribal, rural, and urban communities where broadband is neither available nor affordable. Decades of research has shown that this digital divide is further deepened along racial and ethnic lines. Wealthier, white, and more educated individuals consistently have higher rates of home computer and broadband ownership. Many without th is societal privilege rely on their local public libraries and other community spaces to fill these gaps. The pandemic has also underscored just how significant public libraries have been in addressing people’s need for computers and high-speed internet. Last year, for example, mainstream news mailto:critzo@afutures.xyz mailto:crhinesmith@metro.org mailto:jie.jiang@simmons.edu INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 2 organizations shared several stories about children, parents, and teachers all relying on wireless internet access while seated outside in school and public library parking lots, which happens both during and outside library hours.1 Much less attention has been paid, however, to the broadband infrastructure and technical support that public schools and libraries need to meet the digital demands of their communities. In 2018, our research team, composed of researchers and practitioners at the Simmons University School of Library and Information Science, Measurement Lab (M-Lab), and Internet2, received a grant (award #71-18-0110-18) from the US Institute of Museum and Library Services (IMLS). The purpose was to investigate how advanced broadband measurement capabilities can inform the capacity of the nation’s public libraries to support the online software applications and social and technical infrastructure needed to promote the National Digital Platform.2 In this paper, we present findings from this study, which seeks to address a gap in understanding, particularly among researchers, practitioners, and policymakers, about the speeds and quality of service of public library internet connections across the United States. Through our research we learned that there are significant gaps in knowledge about broadband speeds and quality of service measures that are impacting the ability of public libraries to support their communities’ digital needs. In this context, we hope the quantitative data and analysis presented in this paper contributes to the scholarship on broadband measurement in libraries, as well as to expanding awareness and understanding of broadband data. More concretely, we hope this paper helps to raise awareness of the urgent need for shared knowledge about broadband data and infrastructure that supports digital services in public libraries. We begin the paper with a brief review of key studies that have highlighted the important role of public libraries in promoting digital equity, as well as studies that have discussed the importance of measuring broadband connectivity in public libraries. We concentrate particularly on those studies that have sought to elucidate the opportunities and challenges of both connecting public libraries with high-speed internet connections and educating public librarians, other researchers, and policymakers about what is meant by broadband infrastructure and services. We then present our findings from the quantitative analysis of our broadband measurement data, which highlights the ways in which ongoing, locally collected measurements can enhance libraries’ understanding of their internet service and help inform interactions with patrons and IT service providers. The paper concludes with a discussion of the contribution of our research to the scholarship, and we briefly discuss the implications for state and federal policymakers interested in better understanding the role that library broadband measurement data can play in promoting healthy digital equity ecosystems. LITERATURE REVIEW Digital Inclusion and Broadband Measurement in Public Libraries Public libraries in the United States have been committed to bridging the digital gap by providing free public access to computers, internet, and digital literacy skills for decades.3 For example, in their study of how public libraries respond to inquiries about the digital divide through participatory forms, Schenck-Hamlin, Han, and Schenck-Hamlin found that public libraries have been recognized as the “first and last” resort for internet access particularly “for those unable to afford high-speed connections at home.”4 Further, Bertot, Real, and Jaeger affirmed this idea with their Digital Inclusion Survey data, collected over several years, by stating, “America’s public libraries are an important force for bridging this (digital) divide, with 62.1% of these outlets INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 3 reporting that they are the only free providers of Internet access inclusive of computers in their communities.”5 In addition to providing public access to computers and the internet, US public libraries have placed an emphasis on promoting the general public’s awareness and skills around broadband through delivering free digital literacy training sessions, as well as hosting civic discussions around the topic of broadband connections with their patrons.6 To further illustrate how public libraries narrow the digital divide, DeGuzman, Jain, and Loureiro explained that telemedicine has become a new norm in today’s medical visits, which quickly became a reality during the COVID-19 pandemic. In their article, the authors show how public libraries can play a critical role in bridging this “digital health divide” that exists in many communities.7 As a bottom- up means to promote digital inclusion in the US, the role that public libraries have played to promote digital inclusion and equity cannot be overlooked. However, as Jaeger et al. explained in their study on how public libraries address the digital divide and digital inclusion, “one curious constant across policy approaches to digital divides in many, though not all, nations has been the failure to involve librarians in the formulation of definitions, policies, or other aspects of the policy-making process.”8 It is within this space that public librarians and the technological staff who support them can play an important role in co-designing the tools, skills, and knowledge needed to better understand broadband in public libraries. Broadband Measurement in Public Libraries Many public libraries, particularly small, rural, and tribal libraries, face ongoing challenges in gaining accurate information about their broadband speeds and quality of service. This lack of information can limit their capacity to provide a wide range of applications and services to the community. As Bertot, Real, and Jaeger concluded, one of the big challenges that public libraries have been dealing with is that the speed of public library internet connections “can vary significantly according to local population density.”9 In reaction to this situation, public librarians have shown great interest and need to acquire knowledge about their libraries’ current broadband performance.10 Digital inclusion scholars have proposed topics that future research on public libraries and broadband measurement should explore.11 These topics include how to better inform public librarians in order to assist them in planning, as well as how to deliver sufficient and quality broadband connections to the community. Other topics include looking at how to help public libraries justify the need for more workstations and bandwidth using data coming from “empirical measures, especially longitudinal measures.”12 These and other questions remain largely unanswered in the academic literature. THE MEASURING LIBRARY BROADBAND NETWORKS (MLBN) PROJECT AND RESEARCH DESIGN Research Questions and Significance of Study Our research sought to address this gap in the scholarship on broadband measurement in public libraries through the following research question: How can public libraries utilize broadband measurement tools and training materials to develop a better understanding of the broadband speeds and quality of service that public libraries receive? In response, our research team gathered quantitative data from an open-source broadband measurement system that was both developed for this study and deployed at 30 public libraries across the US. Our research is significant because answers to these questions can help strengthen public libraries as essential anchor institutions and partners in providing data to address the digital needs of their communities. The findings from our study can also assist public libraries in responding to the challenges of developing a more integrated, equitable, and dynamic set of infrastructures for delivering public computing access and digital library services. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 4 Project Overview and Research Design The Measuring Library Broadband Networks (MLBN) project (https://slis.simmons.edu/blogs/mlbn/) was originally conceptualized to be completed in four phases during the two-year grant period. During the first phase, we organized a “participatory design” workshop with our 10 first-year public libraries who agreed to serve as part of our research panel on the project.13 Findings from our analysis of the qualitative data gathered during the workshop revealed that public libraries wanted access to broadband measurement data in order to: (1) better communicate with their patrons about their library’s broadband connectivity, (2) respond to their communities’ digital needs, and (3) justify the importance of robust internet connectivity to their funders.14 Our analysis revealed early on in the project that knowledge gaps existed around the performance of public library broadband networks, patron and staff experiences using the library’s internet connection, and the meaning and value of measurements such as speed tests. During the second phase of the project, we applied what we learned from insights gained during the workshop to our site visits with the 10 participating first-year public libraries. During our fieldwork at the libraries, we sought to interview four different groups of people: (1) library staff, (2) library administrators, (3) IT staff, and (4) IT administrators. The purpose was to gain multiple perspectives on the same sets of questions, which would provide additional qualitative data to help answer our research questions. In a few of the libraries, the library administrator was also the primary IT professional on site. In other words, depending on the size of the organization, librarians often wore several hats, which is certainly not uncommon for small, rural, and tribal libraries. In addition to conducting interviews with these four groups, we also held focus groups with patrons on site at each of the libraries. During this process, we were able to learn more about the context, character, and communities of our partner libraries and gain a better sense of what it is like to work at and/or be a patron of each library, as well as why public libraries might need an open-source broadband measurement system. The other main goal during this phase was to learn more about and document the process of installing our broadband measurement devices. Through this process, we gained additional insights into the nuances of the network configurations at each location and refined our device configurations and setup instructions in response. Ultimately, we sought to identify potential barriers to the measurement devices working properly in the networks of our second-year libraries, when we would not have the luxury of being there in person. At the conclusion of the research program in March 2021, we asked participating library and/or IT staff to complete a final evaluation survey. Twenty libraries responded to a range of questions, two of which related to their understanding of the library’s internet connectivity and network management practices: “Is there an overall download and/or upload cap on the connection to the entire library building?” and “Is there a cap on individual devices using the internet at the library?” Eight libraries responded to one or both of the above questions; their responses are in table A.2 in Appendix B. Training Manual During phases 2 and 3, we worked with Carson Block, a well-known library technology expert and consultant who helped us to develop our MLBN Training Manual (https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/8XXXZQ). The https://slis.simmons.edu/blogs/mlbn/ https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/8XXXZQ INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 5 development of the manual was led by Chris Ritzo, Carson Block, and Colin Rhinesmith to assist our second-year participating public libraries in being able to install the measurement devices on their own. The manual provides a comprehensive overview of our MLBN project, including what we learned in the first year of the project about why public libraries would want to measure broadband at their libraries. Section 2 focuses on the setup instructions that public libraries would need to install the devices to measure both wired and wireless internet connections. We also provided details on the hardware used, device management, and data collection, as well as the data visualization platform developed for the project, which allows public libraries to access and use the broadband data gathered from the devices installed on their network. The manual includes complete information about how the measurement platform used in this study can be set up independently for future use by any library, institution, or individual. We knew this manual would be essential for scaling to the 60 total public libraries for our project and for any additional library after the end of our grant. Final Cohort of Participating Libraries Libraries participating in this research were recruited primarily through the suggestion of the project’s advisory board, many of whom represented state library agencies, regional research and education networks, or other intermediary organizations working with or supporting public libraries. Though the COVID-19 pandemic limited our ability to scale up to our goal of 60 libraries, ultimately 30 libraries were recruited to participate in the research. Appendix A lists the final cohort of participating libraries, the specific branch where measurements were conducted, the city, state, and the library’s IMLS Code. Broadband Measurement Data Collection Quantitative measurements of the network connections at participating libraries were collected using the Murakami software developed by M-Lab, running from a dedicated, on-premise measurement computer/device.15 This software provides tests from two large platforms for crowdsourced speed tests: M-Lab’s Network Diagnostic Tool versions 5 and 7 (NDT) and speedtest-cli, an open source client using the Ookla platform.16 NDT is a network performance test of a connection’s bulk transport, conforming to the Internet Engineering Task Force’s (IEFT) RFC 3148.17 M-Lab provides two NDT testing protocols (ndt5 and ndt7), each measuring different aspects of the Transmission Control Protocol (TCP).18 All versions of NDT measure upload and download speeds and latency using a single TCP stream, between the computer running the test and the nearest M-Lab server. Ookla is a commercial company that created the network performance test speedtest.net.19 Ookla’s test also measures upload and download speeds, as well as latency, but provides the option to measure using a single TCP stream or using multiple streams. The primary differences between these two platforms’ tests are the use of single or multiple TCP streams and the location of testing servers.20 At each location (with a few exceptions), two devices were configured using network details supplied by library or IT staff and shipped to the library with setup instructions (in some cases, depending on network complexity, only one device was installed). One device was connected to the switch or router where internet service connected the location (egress). The other device was connected to an available switch port on the same virtual local area network (VLAN) as WiFi access points. The intention was to measure the capacity of the entire location using the egress device, and the capacity of a single WiFi access point (AP) to serve multiple patrons using the WiFi AP device. Once connected, each device ran tests approximately six randomized times within each 24-hour period. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 6 Each Murakami device ran four tests: NDT 5, NDT 7, Ookla single-stream, and Ookla multi-stream. Each test result was exported to an archive in Google Cloud. This data was imported into BigQuery and analyzed in DataStudio.21 Data from the 2019 Public Libraries Survey (PLS) from IMLS was also included to describe each library’s locale, service population, and number of public computers, annual computer use sessions, and annual WiFi sessions reported.22 Public data provided by both Ookla and M-Lab for the counties in which each MLBN library was located were loaded to compare each platform’s reported aggregate measurements for the surrounding area to the measurements conducted at the libraries.23 Aggregate public data for the surrounding counties in our analyses excluded all measurements from the libraries themselves. Along with the data itself, specific details on our data import, cleaning, and analysis are provided in our publicly available MLBN Dataverse (https://dataverse.harvard.edu/dataverse/mlbn), hosted by Harvard University. Limitations The COVID-19 pandemic created challenges for the research team in scaling up to 60 libraries, as was planned at the beginning of the project. Therefore, we had to limit our outreach and engagement during 2020. When we asked the final 30 public libraries that were able to participate in the research whether they had closed their doors during the pandemic, all of them said yes. However, all the libraries reported that they continued to provide wireless internet access, even though their buildings were closed to the public during this time. Although we were unable to scale up to 60 public libraries, we were still absolutely thrilled with the response we received from the libraries that were able to participate. The NDT 7 tests in our program uncovered a now-resolved bug where measurements were limited by the performance of our selected premise devices, which lack proper support for encryption.24 This is observable in some of our data as a large jump in measured speeds from NDT 7 after November 1, 2020. The jump in measured throughput from NDT 7 tests after November 1, 2020 reflected when encrypted NDT 7 tests were disabled and began running unencrypted.25 FINDINGS Individual Libraries’ Data Data collected at each library is provided in an interactive MLBN DataStudio report, along with summary information about the library from the 2019 Public Libraries Survey.26 Aggregated download, upload, and latency metrics from measurements conducted at each library can be viewed on page 2, Individual Library Data (see figure 1 for an example).27 https://dataverse.harvard.edu/dataverse/mlbn INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 7 Figure 1. Individual Library Data page for Andover, Massachusetts.28 A unique feature of the report is a map of server locations to which tests were conducted. This feature demonstrates the different topologies of the Ookla and M-Lab platforms and enables analysis of measurements to specific servers. If a library’s internet service provider (ISP) hosts an Ookla server, it can be selected to display only measurements of the ISP’s network, as shown in figure 2. The Federal Communications Commission (FCC) distinguishes this topology as on-net, when the server and client are both within the same network, in contrast to off-net, where the server and client are in different networks.29 Figure 2. Individual Library Data page for Clarkston, Michigan, MERIT Networks’ nearest Ookla server selected.30 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 8 By selecting all servers, we can observe the wide geographic range of Ookla servers used. Conversely, when we select one of the NDT tests, we can see that M-Lab servers are only hosted in large metropolitan areas, as seen in figure 3. This demonstrates key differences in the server locations of these two measurement platforms and how the data from each relates to the FCC’s national broadband standard.31 Clarkston, Michigan—all Ookla servers selected32 Clarkston, Michigan—all M-Lab servers selected33 Figure 3. Test server locations for Clarkston, Michigan—all Ookla and M-Lab servers selected. Additional aggregate speeds are provided on the Individual Library Data page to communicate general measurement trends over time: maximum upload and download speed by month, day, hour, and weekday (see figures 4–7). This allows a library to confirm advertised speeds, as seen in figure 4 where the connection at Clarkston, Michigan, was measured consistently at just under 100 Mbps symmetric download and upload. We also observe where measurements are not always consistent, as seen below in figures 5 and 7. In figure 5 we observe a dip in the upload median for Westchester County, New York, in late June 2020, and a drop in upload median in late October 2020. With additional information, a library could correlate these observations with network outages, service changes, or network management changes. For example, the change in October 2020 could have been a network management change or service change to 200 Mbps symmetric. In figure 7 we observe a trend that many librarians will recognize: a slight dip in median speeds over the peak hours of use. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 9 Figure 4. Max speeds by month—Clarkston, Michigan.34 Figure 5. Daily aggregate speeds—Westchester County, New York.35 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 10 Figure 6. Weekday aggregate speeds—Pasco County, Florida.36 Figure 7. Hourly aggregate speeds—Estherville, Iowa.37 With additional local knowledge about network use, conditions, and events, library and IT staff can use ongoing measurements to confirm and explain service changes or uncover issues that are not previously known. Many MLBN libraries sought ongoing measurements of internet service to confirm service delivery levels, and some shared their expected service speeds in our final program evaluation survey. The list of libraries that shared their expected service levels are listed at the end of this paper. Using these reported speeds as an example, we can observe where the overall measured speeds were consistent with the service levels and where they were not, using the Ookla multi-stream measurements. Bennington (VT) Free Library reported a 100 Mbps symmetric connection as their expected service level, and the monthly maximum speeds range between 93 and 98 Mbps.38 Similar results were seen in Live Oak, Georgia; Monroe County, Michigan; and Sheridan, Arkansas.39 In other cases, the reported service levels did not match the measurements. In Pasco County, Florida, measurements indicate a 50 Mbps symmetric connection where the reported service level was 100 Mbps download and 25 Mbps upload.40 And in Ventura County, California, our INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 11 measurements confirm an ~300 Mbps symmetric connection but the reported service level was 1 Gbps symmetric. Finally, in several cases measurements may confirm anomalies or changes in the library’s internet service. In these examples we do not have local knowledge of changes in service or events that might explain anomalies observed in the data, but we can nonetheless observe that a change happened and make an inference about the causes. Some examples include: • Graham County, Arizona—possible service delivery change in May 2020 from ~100/10 (download/upload Mbps) to ~300/3041 • Traverse City, Michigan—possible service delivery change in January 2021 from ~80/5 to ~300/2042 • Waltham, Massachusetts—change to symmetric download and upload in November 2020 from ~50/25 to ~50/5043 • Truro, Massachusetts—observed changes in symmetry of upload and download measurements in June 2020 and March 2021 are perhaps indicators of testing changes in network management to adapt to changing needs44 • Westchester County, New York—observed dip in some upload measurements in late June 2020 at specific times of day for unknown reason45 Comparing Average Monthly Maximum Speeds The final two pages (5 and 6) of our Data Studio report display the maximum overall speeds and the average monthly maximum speeds measured for each library, filterable by IMLS code, access media, and type of ISP.46 Figure 8 shows a report for the average maximum speeds per test at libraries connected with fiber. Figure 8. Average maximum speeds per test measured at MLBN Libraries connected with fiber. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 12 Comparison of Measurements and Related Data To support increased understanding of network measurement within the public library community, we also compared measurements from the public libraries that participated in our study to the public data of crowdsourced measurements from the two large scale internet measurement platforms used in our research measurements, Ookla and M-Lab. We can observe the differences or similarities in measurements between the tests conducted from the libraries’ premise devices and the publicly released data in aggregate for the surrounding county. The weighted average speeds and latency are provided by quarter, since Ookla’s public data limits more granularity. Figure 9. Comparing individual library data to public datasets—Twin Falls, Idaho, 2020 Q4.47 On page 4 of the Data Studio report we can observe whether the measured speeds in MLBN libraries were lower or higher than measurements from the surrounding county (see figure 9), along with the percentage difference between the two sources (see figure 10).48 While these differences are interesting to observe, and in some cases seem quite pronounced, this is not a finding that explains whether libraries are getting better or worse speeds than their communities. This is a coarse comparison that we might think of as a kind of litmus test for further inquiry, rather than findings that tell a definitive story. Ookla public data aggregates all measurements from all ISPs together, while measurements from MLBN libraries are from one. A subscription to Ookla Speedtest Intelligence might enable more direct comparisons on a per ISP basis. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 13 Figure 10. MLBN Data Studio Report, page 4—Was public data or library data higher or lower? DISCUSSION Our research builds on the Digital Inclusion Survey that used a version of the Ookla test in a supplemental speed test study in which, “libraries were asked to run the speed test when the library was closed, during regular hours of operation, and when usage was light, normal, or heavy, by the librarians’ estimation.”49 The software created by MLBN to collect ongoing, randomized measurements extends the idea of the Digital Inclusion Survey’s supplemental speed test, making it a source of monitoring data that can be correlated with each location’s connection plans, service tiers, costs, and other metadata. This approach aligns with the methods used by the FCC by using a dedicated, on-premise device.50 The MLBN system also goes further, providing a framework for any open-source measurement to be added as an available test. Since the conclusion of our research, several new tests have either been added or are being considered. 51 As new measurement tools and analyses are developed by the research community, the MLBN system can incorporate them and bridge network science researchers’ understandings to anchor institutions and the general public. While speed tests have been used in this research and its predecessors, we find there is a gap in public understanding about what these tests can tell us. One important outcome of our study is that understanding the experience of using the internet, measuring it, and regulating it all need additional measurements and approaches that go beyond speed tests alone. Speed tests and the platforms that support them are very different. Internet service plans may focus on upload and download speeds, as does telecommunications regulation at the FCC, but these tests offer only simple and incomplete assessments. The Ookla and M-Lab platforms provide two different controlled experiments designed to measure internet protocols and performance for different segments of the internet’s topology. They both use data generated specifically for the measurement itself. But these tests do not measure our experiences using the internet in general. For that we need network science researchers to develop new measurement methods and analyses. We advocate for even more nuance and additional metrics in the measurement and understanding of internet service beyond speed. This perspective aligns with the research er and network science community who are designing new measurements to account for user experience, content delivery, and latency issues, all of which are often incorrectly assumed to be measured by speed tests.52 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 14 We took the approach of using multiple tests and platforms to provide complementary measurements of different aspects of delivered internet service. If we need to confirm advertised service levels, we can look at the Ookla test data. In this analysis, we used the average monthly maximum speeds as measured by the Ookla multi-stream test since it was used in the Digital Inclusion Survey and most closely aligns with ISPs’ terms of service.53 M-Lab’s NDT, on the other hand, provides a diagnostic measurement for how well the underlying TCP protocol is performing over the measured path. That measurement path for M-Lab tests always traverses the boundaries between networks. If we want to assess our ISP’s connectivity to the internet beyond the last mile network it maintains, we can examine the NDT data. However different the methodologies and measurements of Ookla’s speed test and M-Lab’s NDT test are, we agree with the internet measurement community that measurements of throughput or speed should not be the only focus of assessing a connection’s quality, and a broader public understanding that includes other measurements and nuances needs to be cultivated. 54 Researchers’ understanding of the usefulness of both Ookla and NDT measurements is not static, as recent analyses of their public datasets have shown.55 The research community is also exploring new metrics derived from various data sources that may eventually provide analyses that speak to user experience of using the internet, as well as other technical factors that can influence performance such as latency, bufferbloat, and responsiveness.56 What Does This All Mean for Public Libraries? Building on the speed test used in the Digital Inclusion Survey, the MLBN measurement system enables communities to collect ongoing measurements using a dedicated premise device, leveraging available open-source measurement tools, instead of running periodic or occasional tests. Using this longitudinal data, libraries can confirm expected service levels using Ookla test results, uncover where there is a mismatch in understanding of service levels or network management practice, or compare a library’s measured service to the surrounding community using public data sources. Additional measurements like M-Lab’s NDT can assess a library’s connectivity beyond the ISP’s network. The resulting measurements are useful for interrogating the differences in platforms, their tests, and regulatory or funding benchmarks. But while speed tests can provide useful metrics for understanding general trends and anomalies, an appropriate understanding of what they do and do not measure is also needed. Speed tests demonstrating advertised levels do not necessarily mean that users of that network will not experience slowness as content is delivered to their computers over the same connection. As the federal government prepares to outlay infrastructure dollars to states to improve internet access and service quality, libraries and other public institutions in their states will need specific data and understanding of its nuances and differences. CONCLUSION In this paper, we sought to promote greater understanding about the speeds and quality of service of public library internet connections, an understudied area within library and information science, as well as among broadband policymakers. Library staff and administrators need information to understand and communicate about a library’s network capacities, management practices, and diagnostic or monitoring information. The availability of measurements from different sources can help build shared understanding about a library’s internet connectivity between library staff and IT or network administration personnel. And while speed tests are admittedly limited in what they can tell us about internet capacity, library staff who have access to INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 15 these types of measurements, as well as other information provided by a library’s IT staff, would be better equipped to engage with patrons around questions of internet stability and capacity in support of the library’s mission. From our observations of the data collected, the MLBN measurement system can be used to enhance understanding of the library’s internet service and network management. For the subscribed service levels identified at MLBN libraries, Ookla measurements confirm whether the library’s measured connection speeds matched the expected service levels and when they did not. When measurements are not consistent with expected service levels, libraries can observe the differences and correlate this with additional local information. Ongoing measurements conducted by the library enable local control and monitoring of this vital service and support critique and interrogation of the differences between internet measurement platforms, their topologies, tests, and data, from the perspective of the library doing the measurement. Speed tests are useful for examining these trends but may not always be indicative of a user’s experience accessing and using internet content and services. New research and leadership from the internet measurement community are needed to provide more nuanced and authentic assessments of both network performance and user experience. Emerging research and analyses published openly can be added to the MLBN system to support increased public understanding of internet connection quality and user experience. We hope this paper and our research will help support public libraries interested in ongoing measurement and assessment of their internet service, as well as contribute to discussion of the implications for state and federal policymakers interested in better understanding that public libraries play a key role in their local digital equity ecosystems. ACKNOLWEDGMENTS Funding Statement This work was supported by an award (#LG-71-18-0110-18) from the US Institute of Museum and Library Services National Leadership Grants for Libraries Program. Data Accessibility The datasets supporting this article have been uploaded to the Harvard Dataverse, located here: https://dataverse.harvard.edu/dataverse/mlbn https://dataverse.harvard.edu/dataverse/mlbn INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 16 APPENDIX A: ALL PARTICIPATING LIBRARIES Table A.1. The final cohort of participating libraries, the specific branch where measurements were conducted, the city, state, and the library’s IMLS Code Library Branch (if applicable) City State IMLS region code Andover Memorial Hall Library — Andover MA 21 - Suburb, Large Arkansas River Valley Regional Library Arkansas River Valley Dardanelle AR 33 - Town, Remote Avery Mitchell Yancy (AMY) Regional Library Avery Morrison Library Newland NC 42 - Rural, Distant Bennington Free Library — Bennington VT 32 - Town, Distant Caruthersville Public Library — Caruthersville MO 33 - Town, Remote Cherokee Public Library — Cherokee IA 33 - Town, Remote Clarkston Independence District Library Main Branch Clarkston MI 21 - Suburb, Large Cochise County Library District Elfrida Elfrida AZ 32 - Town, Distant Denver Public Library Central Library Denver CO 11 - City, Large Estherville Public Library — Estherville IA 33 - Town, Remote Mid Arkansas Regional Library System Grant County Library Sheridan AR 33 - Town, Remote Casewell County Library Gunn Memorial Public Library Yanceyville NC 42 - Rural, Distant Hall County Library System Gainesville Gainesville GA 13 - City, Small Hollis Public Library — Hollis AK 43 - Rural, Remote Live Oak Public Libraries Bull Street Library Savannah GA 13 - City, Small Monroe County Library System Bedford Branch Library Temperance MI 21 - Suburb, Large INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 17 Library Branch (if applicable) City State IMLS region code Multnomah County Library St. Johns Branch Portland OR 12 - City, Midsize Pasco County Library Regency Park Library New Port Richey FL 21 - Suburb, Large Pryor Public Library — Pryor OK 32 - Town, Distant Union County Library System The Public Library for Union County Lewisburg PA 32 - Town, Distant Safford City-Graham County Library — Safford AZ 32 - Town, Distant Saline County Library — Benton AR 33 - Town, Remote Saint Paul Public Library Rondo Branch, Central Branch Saint Paul MN 11 - City, Large The Ferguson Library — Stamford CT 12 - City, Midsize Traverse Area District Library Kingsley Branch Library Kingsley MI 21 - Suburb, Large Truro Public Library — Truro MA 21 - Suburb, Large Twin Falls Public Library — Twin Falls ID 33 - Town, Remote Ventura County Public Library EP Foster Branch, Admin Branch Ventura CA 21 - Suburb, Large Waltham Public Library Main library Waltham MA 21 - Suburb, Large Westchester County Public Library Hendrick Hudson Free Library, Library system datacenter Montrose NY 21 - Suburb, Large INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 18 APPENDIX B: FINAL PROGRAM EVALUATION SURVEY Table A.2. Final evaluation responses on internet connectivity and network management Library Survey respondent role(s) Service tier from survey (download/upload) Per device limit imposed Bennington Free Library Library staff, IT staff 100/100 - Live Oak Public Libraries IT staff 300/300 - Monroe County Library System Network administrator 50/20 - Pasco County Library Network administrator 100/25 - Grant County Library Library administrator 15/15 - Public Library for Union County IT staff - 10 Mb/s Ventura County Public Library IT staff 1000/1000 - Waltham Public Library Library staff, IT staff 100/100 50 Mb/s INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 19 ENDNOTES 1 The Editorial Board, “Doing Schoolwork in the Parking Lot Is Not a Solution,” The New York Times, July 18, 2020, https://www.nytimes.com/2020/07/18/opinion/sunday/broadband- internet-access-civil-rights.html; Kathleen Gray, “These Buses Bring School to Students,” The New York Times, December 17, 2020, https://www.nytimes.com/interactive/2020/12/17/us/school-bus-remote-learning- wifi.html; Cecilia Kang, “Parking Lots Have Become a Digital Lifeline,” The New York Times, May 5, 2020, https://www.nytimes.com/2020/05/05/technology/parking-lots-wifi- coronavirus.html; Dan Levin, “In Rural ‘Dead Zones,’ School Comes on a Flash Drive,” The New York Times, November 13, 2020, https://www.nytimes.com/2020/11/13/us/wifi-dead-zones- schools.html. 2 Institute of Museum and Library Services, “LG-71-18-0110-18,” accessed August 25, 2021, https://www.imls.gov/grants/awarded/lg-71-18-0110-18-0. 3 John Carlo Bertot, Brian Real, and Paul T. Jaeger, “Public Libraries Building Digital Inclusive Communities: Data and Findings from the 2013 Digital Inclusion Survey,” Library Quarterly 86, no. 3 (2016): 270–89, https://doi.org/10.1086/686674; Donna Schenck-Hamlin, Soo-Hye Han, and Bill Schenck-Hamlin, “Library-Led Forums on Broadband: An Inquiry into Public Deliberation,” Library Quarterly 84, no. 3 (July 2014): 278–93; Sharon Strover, Brian Whitacre, Colin Rhinesmith, and Alexis Schrubbe, “The Digital Inclusion Role of Rural Libraries: Social Inequities Through Space and Place,” Media, Culture & Society 42, no. 2 (2020), https://doi.org/10.1177/0163443719853504. 4 Schenck-Hamlin, Han, and Schenck-Hamlin, “Library-Led Forums on Broadband,” 280. 5 Bertot, Real, and Jaeger, “Public Libraries Building Digital Inclusive Communities,” 271. 6 Schenck-Hamlin, Han, and Schenck-Hamlin, “Library-Led Forums on Broadband.” 7 Pamela B. DeGuzman, Neha Jain, and Christine G. Loureiro, “Public Libraries as Partners in Telemedicine Delivery: A Review and Research Agenda,” Public Library Quarterly 41, no. 3 (May 2022): 294–304. 8 Paul T. Jaeger, John Carlo Bertot, Kim M. Thompson, Sarah M. Katz, and Elizabeth J. DeCoster, “The Intersection of Public Policy and Public Access: Digital Divides, Digital Literacy, Digital Inclusion, and Public Libraries,” Public Library Quarterly 31, no. 1 (January 2012): 4. 9 Bertot, Real, and Jaeger, “Public Libraries Building Digital Inclusive Communities,” 276. 10 Colin Rhinesmith et al., “Co-Designing an Open Source Broadband Measurement System with Public Libraries,” in eds. Larry Stillman, Misita Anwar, Colin Rhinesmith, and Vanessa Rhinesmith, Proceedings—17th CIRN Conference 6-8 November 2019, Monash University Prato Centre, Italy: “Whose Agenda: Action, Research, & Politics” (Department of Human Centred Computing, Monash University, 2020): 153–76, https://www.researchgate.net/publication/341882544_Co- Designing_an_Open_Source_Broadband_Measurement_System_with_Public_Libraries. https://www.nytimes.com/2020/07/18/opinion/sunday/broadband-internet-access-civil-rights.html https://www.nytimes.com/2020/07/18/opinion/sunday/broadband-internet-access-civil-rights.html https://www.nytimes.com/interactive/2020/12/17/us/school-bus-remote-learning-wifi.html https://www.nytimes.com/interactive/2020/12/17/us/school-bus-remote-learning-wifi.html https://www.nytimes.com/2020/05/05/technology/parking-lots-wifi-coronavirus.html https://www.nytimes.com/2020/05/05/technology/parking-lots-wifi-coronavirus.html https://www.nytimes.com/2020/11/13/us/wifi-dead-zones-schools.html https://www.nytimes.com/2020/11/13/us/wifi-dead-zones-schools.html https://www.imls.gov/grants/awarded/lg-71-18-0110-18-0 https://doi.org/10.1086/686674 https://doi.org/10.1177/0163443719853504 https://www.researchgate.net/publication/341882544_Co-Designing_an_Open_Source_Broadband_Measurement_System_with_Public_Libraries https://www.researchgate.net/publication/341882544_Co-Designing_an_Open_Source_Broadband_Measurement_System_with_Public_Libraries INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 20 11 John Carlo Bertot and Charles R. McClure, “Assessing Sufficiency and Quality of Bandwidth for Public Libraries,” Information Technology and Libraries 26, no. 1 (March 2007): 14–22; Lauren H. Mandel, Bradley W. Bishop, Charles R. McClure, John Carlo Bertot, Paul T. Jaeger, “Broadband for Public Libraries: Importance, Issues, and Research Needs,” Government Information Quarterly 27, no. 3 (January 1, 2010): 280–91. 12 Mandel, Bishop, McClure, Bertot, and Jaeger, “Broadband for Public Libraries,” 388. 13 Douglas Schuler and Aki Namioka, eds., Participatory Design: Principles and Practices (Hillsdale, NJ: Lawrence Erlbaum Associates, Inc., 1993). 14 Rhinesmith et al., “Co-Designing.” 15 Measurement Lab, “M-Lab/Murakami: Run Automated Internet Measurement Tests in A Docker Container” (2021), https://github.com/m-lab/murakami/. 16 Measurement Lab, “M-Lab/Ndt5-Client-Go: Ndt5 Reference Client Implementation in Go” (2021), https://github.com/m-lab/ndt5-client-go; Sivel, “Sivel/Speedtest-Cli: Command Line Interface for Testing Internet Bandwidth Using Speedtest.net” (2021), https://github.com/sivel/speedtest-cli. 17 Measurement Lab, “NDT (Network Diagnostic Tool),” https://www.measurementlab.net/tests/ndt/. 18 Lai Yi Ohlsen, Matt Mathis, and Stephen Soltesz, “Evolution of NDT,” Measurement Lab (blog), August 5, 2020, https://www.measurementlab.net/blog/evolution-of-ndt/. 19 Ookla, “Speedtest,” https://www.speedtest.net/. 20 Measurement Lab, “Where Are M-Lab Servers Hosted?”, https://support.measurementlab.net/help/en-us/9-platform/2-where-are-m-lab-servers- hosted. 21 MLBN Data Studio Report, page 1—Overview of MLBN Libraries, https://datastudio.google.com/u/0/reporting/0dff817b-0e0e-446e-a3b3- 406121291124/page/gxXiB. 22 Institute of Museum and Library Services, “Public Libraries Survey,” https://www.imls.gov/research-evaluation/data-collection/public-libraries-survey. 23 Ookla, “Ookla’s Open Data Initiative,” https://www.ookla.com/ookla-for-good/open-data; Measurement Lab, “Data Overview,” https://www.measurementlab.net/data/. 24 Measurement Lab, “Detect CPU Capabilities and Set Scheme Accordingly by robertodauria · pull request #62 · m-lab/ndt7-client-go” (2021), https://github.com/m-lab/ndt7-client- go/pull/62; Measurement Lab, “M-Lab/Ndt7-Client-Go: Ndt7 Reference Client Implementation in Go” (2021), https://github.com/m-lab/ndt7-client-go. https://github.com/m-lab/murakami/ https://github.com/m-lab/ndt5-client-go https://github.com/sivel/speedtest-cli https://www.measurementlab.net/tests/ndt/ https://www.measurementlab.net/blog/evolution-of-ndt/ https://www.speedtest.net/ https://support.measurementlab.net/help/en-us/9-platform/2-where-are-m-lab-servers-hosted https://support.measurementlab.net/help/en-us/9-platform/2-where-are-m-lab-servers-hosted https://datastudio.google.com/u/0/reporting/0dff817b-0e0e-446e-a3b3-406121291124/page/gxXiB https://datastudio.google.com/u/0/reporting/0dff817b-0e0e-446e-a3b3-406121291124/page/gxXiB https://www.imls.gov/research-evaluation/data-collection/public-libraries-survey https://www.ookla.com/ookla-for-good/open-data https://www.measurementlab.net/data/ https://github.com/m-lab/ndt7-client-go/pull/62 https://github.com/m-lab/ndt7-client-go/pull/62 https://github.com/m-lab/ndt7-client-go INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 21 25 Measurement Lab, “Add to ndt7 runner to force all tests to be non-TLS” (2020), https://github.com/m- lab/murakami/commit/3770d6b63ebd9ad62b3754e0642bd0e9216e171e. 26 MLBN Data Studio Report, Page 1—Overview of MLBN Libraries, https://datastudio.google.com/s/gHTW2GM-VfI. 27 MLBN Data Studio Report, Page 2—Individual Library Data, https://datastudio.google.com/s/kPpdGF3I8C4. 28 MLBN Data Studio Report Page 2—Individual Library Data page, Andover, Massachusetts, https://datastudio.google.com/s/uNPY5Z1M21g. 29 FCC Office of Engineering and Technology, “Technical Appendix to the Tenth MBA Report” (n.d.) 24, accessed 2021-06-22, http://data.fcc.gov/download/measuring-broadband- america/2020/Technical-Appendix-fixed-2020.pdf. 30 MLBN Data Studio Report Page 2—Individual Library Data for Clarkston, Michigan, MERIT Networks’ Nearest Ookla Server Selected, https://datastudio.google.com/s/pQPcybmHAj8. 31 Ookla, “The Speedtest Server Network,” accessed August 16, 2021, https://www.ookla.com/speedtest-servers; M-Lab, “NDT Data in NTIA Indicators of Broadband Need,” accessed February 15, 2022, https://www.measurementlab.net/blog/ntia/. 32 MLBN Data Studio Report Page 2—Clarkston, Michigan—All Ookla Servers Selected, https://datastudio.google.com/s/oILgppL47Rc. 33 MLBN Data Studio Report Page 2—Clarkston, Michigan—all M-Lab Servers Selected, https://datastudio.google.com/s/j79KG_WMyRc. 34 MLBN Data Studio Report Page 2—Max Speeds by Month—Clarkston, Michigan https://datastudio.google.com/s/hyljK_mPVL4. 35 MLBN Data Studio Report Page 2—Daily Speeds—Westchester County, NY https://datastudio.google.com/s/ixGn8V4R3IA. 36 MLBN Data Studio Report Page 2—Weekday Aggregate Speeds—Pasco County, FL https://datastudio.google.com/s/gAdtydi2g9k. 37 MLBN Data Studio Report—Hourly Speeds—Estherville, IA https://datastudio.google.com/s/n5Pe2Y9RjYg. 38 MLBN Data Studio Report page 2—Individual Library Data (Bennington, VT and speedtest- multi-stream selected) https://datastudio.google.com/s/nJreLTxeJGE. 39 MLBN Data Studio Report page 2—Individual Library Data—Ookla Multi-stream Measurements for Live Oak, GA, https://datastudio.google.com/s/kiLD8kkltZA; MLBN Data Studio Report page 2—Individual Library Data—Ookla Multi-stream Measurements for Monroe County, MI, https://datastudio.google.com/s/p140sWJIHjM; MLBN Data Studio Report page 2—Individual https://github.com/m-lab/murakami/commit/3770d6b63ebd9ad62b3754e0642bd0e9216e171e https://github.com/m-lab/murakami/commit/3770d6b63ebd9ad62b3754e0642bd0e9216e171e https://datastudio.google.com/s/gHTW2GM-VfI https://datastudio.google.com/s/kPpdGF3I8C4 https://datastudio.google.com/s/uNPY5Z1M21g http://data.fcc.gov/download/measuring-broadband-america/2020/Technical-Appendix-fixed-2020.pdf http://data.fcc.gov/download/measuring-broadband-america/2020/Technical-Appendix-fixed-2020.pdf https://datastudio.google.com/s/pQPcybmHAj8 https://www.ookla.com/speedtest-servers https://www.measurementlab.net/blog/ntia/ https://datastudio.google.com/s/oILgppL47Rc https://datastudio.google.com/s/j79KG_WMyRc https://datastudio.google.com/s/hyljK_mPVL4 https://datastudio.google.com/s/ixGn8V4R3IA https://datastudio.google.com/s/gAdtydi2g9k https://datastudio.google.com/s/n5Pe2Y9RjYg https://datastudio.google.com/s/nJreLTxeJGE https://datastudio.google.com/s/kiLD8kkltZA https://datastudio.google.com/s/p140sWJIHjM INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 22 Library Data—Ookla Multi-stream Measurements for Sheridan, AR, https://datastudio.google.com/s/peHRfXoSX6s. 40 MLBN Data Studio Report page 2—Individual Library Data—Ookla Multi-stream Measurements for Pasco County, FL, https://datastudio.google.com/s/uxGJLFV44ZE. 41 MLBN Data Studio Report page 2—Individual Library Data—Graham County, AZ—Observing Maximum Ookla Measured Speeds by Month, https://datastudio.google.com/s/ieal3B2vBu8. 42 MLBN Data Studio Report page 2—Individual Library Data—Traverse City, MI – Observing Maximum Ookla Measured Speeds by Month, https://datastudio.google.com/s/tljy900Tx-Q. 43 MLBN Data Studio Report page 2—Individual Library Data—Waltham, MA—Observing Maximum Ookla Measured Speeds by Month, https://datastudio.google.com/s/pxzDRb0chb0. 44 MLBN Data Studio Report page 2—Individual Library Data—Truro, MA—Observing Maximum Ookla Measured Speeds by Month, https://datastudio.google.com/s/oKcWP-_f6qs. 45 MLBN Data Studio Report page 2—Individual Library Data—Westchester County, NY— Observing Daily Aggregate Speeds between June 17-29, 2020 and Hourly Aggregate Speeds for tests in the 8 a.m., 10 a.m., 3 p.m., and 10 p.m. columns, https://datastudio.google.com/s/vlEG- yNkoQc. 46 MLBN Data Studio Report page 5—Comparison of Overall Maximum Speeds by Test (fiber access media selected), https://datastudio.google.com/s/ii2F7ONUcHI; MLBN Data Studio Report page 6—Average Monthly Maximum Speeds per Test (all libraries selected), https://datastudio.google.com/s/oO7XAl_KV4k. 47 MLBN Data Studio Report page 3—Comparing Individual Library Data to Public Datasets—Twin Falls, ID, 2020 Q4, https://datastudio.google.com/s/vQQ38wyrAHo. 48 MLBN Data Studio Report Page 4—Was Public Data or Library Data Higher or Lower?, https://datastudio.google.com/s/mEB4S-CflCw. 49 Bertot, Real, and Jaeger, “Public Libraries Building Digital Inclusive Communities,” 271; American Library Association, “Library Broadband Speed Test Shows Increased Capacity; Room Still for Improvement” (press release), https://www.ala.org/news/press- releases/2015/04/library-broadband-speed-test-shows-increased-capacity-room-still- improvement. 50 Federal Communications Commission, “Measuring Broadband America,” https://www.fcc.gov/general/measuring-broadband-america. 51 Measurement Lab, “Add new runner for ooniprobe-cli,” https://github.com/m- lab/murakami/pull/103; Measurement Lab, “Add fast.com test runner,” https://github.com/m-lab/murakami/issues/48; “Data Science Institute,” University of Chicago, https://cdac.uchicago.edu/; University of Chicago Data Science Institute, “Netrics – Active Measurements of Internet Performance,” https://github.com/chicago-cdac/nm-exp- active-netrics/. https://datastudio.google.com/s/peHRfXoSX6s https://datastudio.google.com/s/uxGJLFV44ZE https://datastudio.google.com/s/ieal3B2vBu8 https://datastudio.google.com/s/tljy900Tx-Q https://datastudio.google.com/s/pxzDRb0chb0 https://datastudio.google.com/s/oKcWP-_f6qs https://datastudio.google.com/s/vlEG-yNkoQc https://datastudio.google.com/s/vlEG-yNkoQc https://datastudio.google.com/s/ii2F7ONUcHI https://datastudio.google.com/s/oO7XAl_KV4k https://datastudio.google.com/s/vQQ38wyrAHo https://datastudio.google.com/s/mEB4S-CflCw https://www.ala.org/news/press-releases/2015/04/library-broadband-speed-test-shows-increased-capacity-room-still-improvement https://www.ala.org/news/press-releases/2015/04/library-broadband-speed-test-shows-increased-capacity-room-still-improvement https://www.ala.org/news/press-releases/2015/04/library-broadband-speed-test-shows-increased-capacity-room-still-improvement https://www.fcc.gov/general/measuring-broadband-america https://github.com/m-lab/murakami/pull/103 https://github.com/m-lab/murakami/pull/103 https://github.com/m-lab/murakami/issues/48 https://cdac.uchicago.edu/ https://github.com/chicago-cdac/nm-exp-active-netrics/ https://github.com/chicago-cdac/nm-exp-active-netrics/ INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 MEASURING LIBRARY BROADBAND NETWORKS | RITZO, RHINESMITH, AND JIANG 23 52 Internet Architecture Board, “Measuring Network Quality for End-Users, 2021, https://www.iab.org/activities/workshops/network-quality/; David D. Clark and Sara Wedeman, “Measurement, Meaning and Purpose: Exploring the M-Lab NDT Dataset (August 2, 2021), https://ssrn.com/abstract=3898339 or http://dx.doi.org/10.2139/ssrn.3898339. 53 Bertot, Real, and Jaeger, “Public Libraries Building Digital Inclusive Communities.” 54 Internet Architecture Board, “Measuring Network Quality for End-Users, 2021” (call for papers), accessed August 26, 2021, https://www.iab.org/activities/workshops/network-quality/; Lai Yi Ohlsen and Chris Ritzo, “NDT Data in NTIA Indicators of Broadband Need,” Measurement Lab (blog), July 15, 2021, https://www.measurementlab.net/blog/ntia/. 55 Clark and Wedeman, “Measurement, Meaning and Purpose.” 56 Lai Yi Ohlsen, “M-Lab Research Fellows – Sprint 2022,” Measurement Lab (blog), January 13, 2022, https://www.measurementlab.net/blog/research-fellow-announcement/; Lai Yi Ohlsen, “Upcoming M-Lab Community Call Discussing Latency, Bufferbloat, Responsiveness,” Measurement Lab (blog), August 18, 2021, https://www.measurementlab.net/blog/community-call-announcement/; Broadband Internet Technical Advisory Group (BITAG), “Latency Explained,” https://bitag.org/latency- explained.php; Internet Architecture Board, “Measuring Network Quality for End-Users 2021,” https://www.iab.org/activities/workshops/network-quality/; CAIDA, “NSF Workshop on Overcoming Measurement Barriers to Internet Research (WOMBIR 2021), https://www.caida.org/workshops/wombir/2101/; CAIDA, “2nd NSF Workshop on Overcoming Measurement Barriers to Internet Research (WOMBIR-2), https://www.caida.org/workshops/wombir/2104/; IETF, “Responsiveness under Working Conditions” (draft), https://datatracker.ietf.org/doc/draft-cpaasch-ippm-responsiveness/. https://www.iab.org/activities/workshops/network-quality/ https://ssrn.com/abstract=3898339 http://dx.doi.org/10.2139/ssrn.3898339 https://www.iab.org/activities/workshops/network-quality/ https://www.measurementlab.net/blog/ntia/ https://www.measurementlab.net/blog/research-fellow-announcement/ https://www.measurementlab.net/blog/community-call-announcement/ https://bitag.org/latency-explained.php https://bitag.org/latency-explained.php https://www.iab.org/activities/workshops/network-quality/ https://www.caida.org/workshops/wombir/2101/ https://www.caida.org/workshops/wombir/2104/ https://datatracker.ietf.org/doc/draft-cpaasch-ippm-responsiveness/ Abstract Introduction Literature Review Digital Inclusion and Broadband Measurement in Public Libraries Broadband Measurement in Public Libraries The Measuring Library Broadband Networks (MLBN) Project and Research Design Research Questions and Significance of Study Project Overview and Research Design Training Manual Final Cohort of Participating Libraries Broadband Measurement Data Collection Limitations Findings Individual Libraries’ Data Comparing Average Monthly Maximum Speeds Comparison of Measurements and Related Data Discussion What Does This All Mean for Public Libraries? Conclusion Acknolwedgments Funding Statement Data Accessibility Appendix A: ALL PARTICIPATING LIBRARIES APPENDIX B: FINAL PROGRAM EVALUATION SURVEY Endnotes