key: cord-0809558-1tiilo27
authors: Lazar, Jonathan
title: Managing digital accessibility at universities during the COVID-19 pandemic
date: 2021-03-08
journal: Univers Access Inf Soc
DOI: 10.1007/s10209-021-00792-5
sha: 668202cc6e2ac7c3e1a2061e55ef3437c689205f
doc_id: 809558
cord_uid: 1tiilo27

Students, faculty and staff with disabilities at universities have faced many digital accessibility barriers during the COVID-19 pandemic. While the legal requirements for digital accessibility at universities are clear, the mechanisms for managing digital accessibility are not yet well established. This study reports on a longitudinal series of 18 interviews throughout 2020 involving directors of digital accessibility, starting before the COVID-19 pandemic was officially declared by the World Health Organization in March 2020. These interviews took place with directors of digital accessibility at a small-, medium-, and large-sized university in the USA, respectively, as well as the director of digital accessibility at a comparative US state government agency. Patterns throughout the year, common challenges, and emerging best practices are all discussed, as well as implications for managing digital accessibility during non-pandemic times.

Over the past decade, universities have been struggling with ensuring that all students, faculty, and staff with disabilities can access digital technologies and content [6] . Technological advances have increased the pervasiveness of technology in university settings. Digital technologies and content can include websites, e-books, content management systems for courses, registration mechanisms, technologies for participating in a physical classroom (such as clickers), and video and other multimedia content. Lawsuits and similar administrative complaints filed against universities in the 2010s brought more attention to the topic of digital accessibility, as well as an increase in compliance [13] . Arizona State University, Penn State University, Florida State University, Louisiana Tech University, University of Montana, Wichita State University, and Miami University of Ohio, were among the US universities involved in legal actions, due to inaccessible technology used on campus and in education [8] . The COVID-19 pandemic has only increased the importance of digital accessibility, as many universities have moved their operations to either be fully or partially on the web. For many students with disabilities, barriers in technology accessibility used to be barriers to the digital portion of their education, and those students could still take part in the in-person aspects of education which remained accessible. However, as university education in 2020 became primarily or exclusively online, existing accessibility barriers could lead to a complete exclusion from the university experience.

This issue of digital accessibility is a global problem, which people with disabilities have faced at campuses across the world, well documented in papers such as [4, 9, 10] . This paper focuses specifically on universities in the USA during the year 2020.

While the legal requirements for digital accessibility for a US-based university are very clear, the mechanisms for managing digital accessibility are not yet well established [13] . How does managing accessibility change over the course of a year? What challenges are faced? How do policies change and how are these policies implemented? What trade-offs are made? The goal of this paper is to describe the experiences of three digital accessibility managers within universities, and also to describe the comparison experience of an individual with similar managerial responsibilities within a state government agency, during the COVID-19 pandemic year of 2020.

This paper describes 18 interviews carried out in 2020 with four individuals who manage digital accessibility at a small-sized, a medium-sized, and a large-sized university in the USA, respectively, as well as a manager of digital accessibility at a state government agency in the USA. Originally, this study was designed to understand the experiences of managing digital accessibility over the course of a calendar year. Since the interviews started in January 2020, and continued through the COVID-19 pandemic, this study had the ability to examine and document the changes in digital accessibility that occurred at universities during the pandemic. This study can provide a unique view of how universities managed digital accessibility in 2020 during a pandemic, and offer advice for addressing digital accessibility during 2021 and future pandemic experiences. Commonalities and themes are described, while the paper ends with suggestions for managing digital accessibility during the current and any future pandemics.

Universities have had well-documented problems with digital accessibility. Web pages have often been the focus of the research studies, since they are public and easy to access and evaluate (see [2] or [14] as examples). Yet there are many technologies utilized at universities, and all of them must provide equal access for students, faculty, and staff with disabilities. For instance, the software shells for content management systems (such as WordPress and Drupal) and course management systems (such as Blackboard, Moodle, and Canvas) need to be accessible, as does the content posted (such as course content created by faculty) [6] . In-classroom technologies, such as clickers and electronic whiteboards, need to be accessible. Recorded videos and online events need to have accurate captioning [3] . The online systems used during the lifecycle of being a student (applying for admission, registering for classes, paying student bills, and applying to graduate) need to be accessible, as do human resource systems for faculty and staff, such as payroll and benefit applications. Library and class materials need to be made accessible for students with print-related disabilities [1] . Ironically, although the number of classes teaching topics related to accessibility at universities has increased over time, that has not impacted the accessibility of the classes themselves [11] .

Public media often focuses on digital accessibility for businesses (e.g., the Domino's Pizza or Winn-Dixie lawsuits). Yet, it is important to note that, legally, universities (and state government) have different legal coverage for topics such as digital accessibility than businesses. The relevant law for digital accessibility for businesses is Title III of the Americans with Disabilities Act (ADA), 1 and there are existing legal questions about the necessary nexus between the physical location (such as a store), and the website, for Title III coverage of public accommodations. Some circuit courts of appeals currently require that a website has a nexus to a physical location (such as a store), while others do not [7] . Yet, due to the different legal coverage, these legal questions related to the "nexus to a physical location" are not applicable to universities or state government.

State universities, as well as state governments, are covered under Title II, rather than Title III, of the Americans with Disabilities Act. Private universities are covered under Title III of the ADA as a public accommodation (specifically, public accommodation category (10) "Places of education (e.g., nursery schools, elementary, secondary, undergraduate, or postgraduate private schools)." Regardless of the public or private nature of a university, nearly all universities are covered under Section 504 of the Rehabilitation Act, because they receive federal funding. 2 Section 504 of the Rehabilitation Act (29 U.S.C. § 794) prohibits discrimination against people with disabilities by any recipients of federal funds, so none of the "nexus to a physical location" issues involving business, are relevant to universities. The US Department of Justice has taken a number of actions over the years, to clearly specify that campus technologies must be accessible for people with disabilities, including a well-known "Dear Colleague" letter jointly published with the US Department of Education in 2010 which laid out the requirements for accessible technology under the ADA and Section 504 [15] .

While the legal requirements for digital accessibility on a university campus are very clear, the mechanisms for managing digital accessibility are not yet well established [13] . Universities are generally run in a decentralized manner, employment, Title II of the ADA prohibits discrimination by state and local government, and Title III prohibits discrimination by public accommodations. There are 12 categories of public accommodations described in the ADA. While the law and the regulations do not specifically mention websites, since 1996, the Department of Justice has said that websites of public accommodations are covered under the "effective communication" requirement of the ADA (a viewpoint fully described in the State of Interest in New v. Lucky Brand Jeans, available at: https ://www.ada.gov/brief s/lucky _brand _soi.pdf.). More information about the ADA is available at https ://www.ada.gov/2010_ regs.htm., and the legal coverage of web site accessibility for public accommodations is described in [7] .

with a majority of decisions being made at the level of a department or college, rather than university-wide. Academic freedom and shared governance are well-established concepts, and they further limit the perceived ability to do centralized management. Yet, many aspects of the university community, such as housing, food service, security, disability accommodations, procurement, and IT security, are typically centrally managed, and IT accessibility generally needs to be centrally coordinated. This is in large part due to how IT accessibility involves multiple divisions of a university, such as the division of academic affairs due to the coursework, the division of student affairs due to the office of disability student services, and the division of administration and finance due to the IT involved. Problems related to IT accessibility cannot be solved by one division, or one department. Techniques that are common for managing digital accessibility on a diverse campus include having a centralized person for coordinating IT accessibility (a champion who is proactive in reaching out to people throughout the university), and the compliance monitoring approaches of procurement, IT accessibility plans, and IT accessibility audits [6, 12] . Campus-wide initiatives (such as the Assistive Technology Initiative at George Mason University 3 ), or university system-wide initiatives (such as the California State University Accessible Technology Initiative 4 ), provide some best practices on management structures for coordinating digital accessibility on a campus. However, there is no previously published research which attempts to understand, from those charged with managing digital accessibility, what the overall experience of managing accessibility looks like.

A research project was planned in 2019, to evaluate how digital accessibility was managed on US university campuses throughout the 2020 calendar year. Obviously, when the project was planned, no one knew about the coming COVID-19 pandemic. The original goal of the project was to interview those responsible for managing digital accessibility on a campus, multiple times throughout the year, to get a sense for how policies change and how digital accessibility is managed. Quantitative surveys, which collect shallow data from a large number of people, are good for understanding newer research problems, for understanding the "what" rather than the "why." [5] . The issue of digital accessibility on university campuses is a well-known problem, but little is known about how digital accessibility is managed, where the barriers occur, and what challenges are faced throughout the year. Rather than a one-point-in-time survey or even a one-point-in-time interview, the goal of this research was to have an ongoing conversation, to learn about the challenges and best practices in managing digital accessibility in a large organization throughout a year-long period. For instance, a survey could document that captioning videos on campus is an ongoing issue, and that's already well-known (see the National Association of the Deaf lawsuits against Harvard and MIT for examples), but what is not known are the best practices in successfully managing accessibility. To put it another way, surveys could help document that digital accessibility is a problem, but longitudinal interviews could help in understanding potential solutions! By January 2020, the project had received Institutional Review Board (IRB) 5 approval from the university, and four individuals had agreed to participate. Participants were not paid for their participation. Three of these individuals were the director of digital accessibility for their respective campus (more information about the campuses is given in the following section). A fourth individual was director of digital accessibility for a state government agency, included in the project to enable a comparison of how digital accessibility is managed outside of the university setting, in the same year-long period. The interviews began in January, and by March it became clear that a pandemic was occurring. All four interviewees agreed to continue their interviews, with the understanding that the focus of the interviews might change slightly. Because this data collection effort was already in place in January 2020 and continued regularly throughout 2020, these interviews provide a unique view to the challenges that occurred during the COVID-19 pandemic, and what policies and management strategies were used (successfully and unsuccessfully), related to digital accessibility.

While the original plan was for the directors of digital accessibility to participate in interviews bimonthly (a total of 6 interviews), due to the ongoing stress and scheduling concerns during the pandemic, each participant was interviewed fewer than 6 times throughout the year. Two participants were interviewed five times each and two participants were interviewed four times each, for a total of 18 interviews. The first interview for each individual started out with a standard list of questions, about the individual's role in the 3 https ://ati.gmu.edu/. 4 https ://www2.calst ate.edu/impac t-of-the-csu/techn ology /acade mictechn ology -servi ces/Pages /acces sible -techn ology .aspx. 5 Institutional Review Boards exist primarily in US universities and organizations to review the appropriateness of research involving human participants. In many other countries, a similar role is taken by groups titled "research ethics committees" or similar. The protection of human participants in research is a core part of human rights found in policies and documents as diverse as the Declaration of Helsinki and the European Union's General Data Protection Regulation (GDPR). organization, existing organizational policies related to digital accessibility, current challenges, methods for both evaluating for accessibility (usability testing, expert reviews, and automated testing), and manager-level compliance monitoring, and plans for the year 2020. Follow-up interviews were much more open-ended and less structured, asking about their current successes, challenges, and follow-up questions based on previous interviews.

One of the key concerns of the interviewees was confidentiality and anonymity, ensuring that their personal and organizational participation would not be made aware to anyone. In fact, only one of the four participants made their bosses in the organization aware of their participation in the interviews. Universities are under ongoing threats of lawsuits due to inaccessible technology, so there is an extreme sensitivity to revealing information about accessibility or revealing participation in this study (for the one participant coming from state government, the legal action might not take the form of a lawsuit, it might instead take the form of actions coming from the US Department of Justice, but there are concerns regardless). As in any study, protecting the privacy and anonymity of research participants is important. What differed in the present case is the ease in identifying or "unmasking" or "outing" participants since they come from a very small pool of potential participants. The community of people who have the role of "university director of digital accessibility" is relatively small (perhaps only a few hundred people across the USA), and even simple demographic information about the participants may make them relatively easy to identify. Because of that, no demographic information of the individuals (age, gender, previous work, number of years in position, degree information) will be given in this article. Furthermore, no information about the size of the digital accessibility team, current or pending legal action, the title of the role, or any similar organizational-level details were given.

Due to the sensitive nature of their work, none of the four participants were willing to be audio-or video-recorded, and only agreed for the researcher to take handwritten notes. Therefore, the type of content analysis typically done in recorded interviews was not possible. There are two additional notes related to the extreme level of concern about anonymity. First, multiple times during the year, participants would make a comment, and then say, "wait a minute, don't write that, that would identify me or my employer…". At that point, the author of this paper either crossed out what was already written, or did not make any additional notes on that topic. None of the "crossed-out" notes are included in this paper. Second, when participants made a specific point that sounded like a great potential quote, since no recordings were being made, the researcher repeated the quote back to the participant, to confirm that the quote was recorded in notes properly. The analysis will be in a format similar to that of a case study, where an overall story is told. However, case studies typically use multiple sources of data (such as existing records and direct observation) and only one or two interviews, whereas the current study uses multiple interviews over a year-long period [5] .

The participants were directors of digital accessibility at a small-sized university (approx. 5,000 students), a mediumsized university (approx. 20,000 students), and a large-sized university (approx. 40,000 students), respectively, across the USA. Two of the universities were private and one of the universities was public. The fourth participant was the director of digital accessibility at a state government agency in the USA.

For the participants from the three universities, the first interview took place in person (pre-pandemic when that was allowed), while the rest of the interviews took place via Zoom meetings or phone calls due to COVID-19 restrictions. For the participant from the state government agency, all interviews took place via Zoom meetings or phone calls for the entire year. To help describe the interviews, each participant was given a pseudonym, i.e., "Marshall" (at the small-sized university), "Denise" (at the medium-sized university) "Neil" (at the large-sized University), and "Janice" (at the state government agency).

The first interview with the director of IT accessibility, "Marshall," at the small-sized university took place face-toface, in January 2020. Marshall is responsible for IT accessibility on campus, but this is only a portion of his responsibilities: "My job is so big, and accessibility is just a small portion of it." Marshall described how there were a number of new initiatives underway at the time. The homepage of the university had recently been redesigned and made more accessible, as a part of the campus-wide rebranding. Efforts were underway to reduce the number of PDF files on campus (inspired by the "Great PDF Purge" at North Carolina State University 6 ), because of the difficulty in making the PDF files accessible. The campus had acquired a license for the SiteImprove 7 software for web accessibility testing in 2018, to use for web accessibility testing in additional to the existing use of the WAVE 8 plug-in for testing. In addition to the automated web accessibility testing, Marshall and his assistants use NVDA 9 and JAWS 10 for testing new software applications.

Marshall said that his main goal for 2020 was to educate the content editors on campus about accessibility, all of the 300 or so people who have access to the content server. In 2019, Marshall and his assistants had been developing a new page template, and expected that in 2020, they would be greatly increasing the amount of on-campus accessibility training. Not only were Marshall and his assistants hoping to educate more content contributors, but were hoping that through the use of the SiteImprove tool, the reduction of PDF files on campus, the use of the new web page template, a new requirement from the provost's office that course syllabi be accessible, and reaching out to the full campus with a message of "the 5 main barriers to digital content accessibility," that there would be major improvements in digital accessibility in the year 2020. One hope is that by the end of 2020, Marshall and his assistants would be sending out specific accessibility reports about the testing that they did and the barriers that they discovered, to content/software owners.

These goals at the university were not driven by legal complaints or even a large number of students, faculty or staff with disabilities. Marshall estimated in Jan 2020 that the campus had fewer than 10 students who had declared visual or hearing impairments, but noted that a larger number of students had learning disabilities, and that number seemed to be increasing. While the campus has an "accessibility feedback form" on the web, Marshall noted that his team receives almost no feedback. Marshall indicated a close working relationship with the Americans with Disabilities Act coordinator, who in many cases was proactively pushing the topic of digital accessibility, even without complaints or legal problems on the campus.

In interview #2 (pre-pandemic), Marshall noted that since the new homepage was launched, he and his team were hoping to start working on the rest of the website. While accessibility was considered in the development of the website, no faculty, staff, or students with disabilities were involved with testing, due to the low number of individuals with disabilities on campus. Marshall wondered whether perhaps they should bring in an outside contractor who could perform accessibility involving people with disabilities. While course content and web content in general has been getting more attention for web accessibility, there were applications, e.g., the applications for undergraduate admissions, and for students to evaluate instructors, which were known to have accessibility problems. Marshall noted that procurement is an area that he is actively working on. At this point, Marshall is requesting VPATs [Voluntary Product Accessibility Templates] 11 from vendors where there are procurements, and he personally reviews the VPATs.

Marshall noted that funding accessibility is a problem: "everyone wants to own their piece of accessibility, but no one wants to fund it…everyone needs to buy into the part that they own." Potential stakeholders include the offices of the Provost, the VP of diversity and inclusion, the Disability Student Services office, human resources, and the IT division. For instance, getting enhanced tools for online course accessibility; who takes responsibility for that, academic affairs, diversity and inclusion, DSS, or the IT division? While captioning is done proactively for public content, it is only done reactively for courses upon request, and it is not clear that any of the stakeholders will take budgetary responsibility for doing that. Marshall noted his concern: "We're really paying attention first to the public website, since that's available to everyone, and then, we are paying attention to students with disabilities, but not faculty or staff."

One of Marshall's tools for getting more attention for IT accessibility is to do an "end-of-year accessibility report" which goes out to relevant stakeholders in the university. Longer term, Marshall hopes to add content about IT accessibility at both new faculty orientation and new staff orientation. Marshall gave the example of how they used to have a faculty member with physical disabilities, where technology accommodations were made, though currently, Marshall could not remember a single faculty member who needed technology accommodations.

Marshall noted that he was hoping to send out compliance reports to content owners during the summer of 2020 (earlier than the original goal of Fall 2020), and worried that web accessibility "slips easily because we have a rotating door of content owners." Marshall also discussed that he hopes to make a dent in PDF remediation, but "it can be a giant endeavor, because people don't want to learn about Adobe Acrobat Pro 12 because it takes time. We can start with teaching people how to do accessible markup in MS-Word, since they prefer MS-Word." Marshall said that while the content creators will be required to make any remaining PDF files on the website accessible, "We won't force PDF accessibility, just like we don't force web accessibility. We will guilt them into it and follow up when we see that they are not following the protocol." Marshall noted the distinctions between guidelines and policies (policies are more 9 https ://www.nvacc ess.org/. 10 https ://www.freed omsci entifi c.com/produ cts/softw are/jaws/. 11 A Voluntary Product Accessibility Template (VPAT) is a document describing how a hardware or software product conforms to (or does not conform to) accessibility guidelines such as the Web Content Accessibility Guidelines. VPATs can be useful in determining the potential accessibility level of products, but vendors are known to offer confusing or misleading information in their VPATs. See https ://www.secti on508 .gov/sell/vpat. for more information about VPATs. 12 https ://acrob at.adobe .com/us/en/acrob at/acrob at-pro.html. enforceable). The idea stated was that, if the university does not have the resources to check everything for compliance with a policy, then it should not be a policy because there is no way to enforce it.

Interview #3 took place once the pandemic was already underway, and Marshall said that he and his team were struggling to keep up during the pandemic, based on their increased responsibilities. Marshall noted the large number of technology procurements that were made as the campus transitioned to fully online learning in March. Among the technology procurements made the following were noted: software to supporting virtual presentations, software for online testing, a chatbot for the admissions office since campus tours were off-limits, software for planned giving, and a greatly expanded Zoom 13 license (for more seats and more time). Marshall feared that, with so many moving parts at all times, some items will not get the necessary attention, and more digital accessibility problems will occur.

As of June 2020, it was expected that the Fall 2020 semester would be hybrid, with most classes being a combination of online and face-to-face meetings. While no specific digital accessibility policy changes were made related to COVID-19, much more information is going out to faculty members on a regular basis about digital accessibility. Also, the multiple committees involved in planning for the Fall experience have been discussing digital accessibility. Marshall also noted that the incoming Fall 2020 Freshman class is expected to have a few students with visual and/or hearing impairments, and wondered, "how do you figure out how to meet the accommodation needs of the students when they are not on campus?" Marshall was especially concerned about the hybrid nature of the classes being problematic for students with executive functioning problems and anxiety.

Marshall noted that due to the large number of tasks to address during the pandemic, no further work has been done yet on either rolling out the new web page template to more of the university website, or addressing the problems of PDF accessibility. Overall, Marshall and his team felt that during the time of COVID-19, "digital accessibility has been status quo. The only thing that's gotten more attention is the needs of the incoming freshman with disabilities, and we have already been involved with the online Freshman orientation." Marshall still hoped to roll out more online training related to digital accessibility, and to start the PDF accessibility work, before the Fall semester starts in September. Marshall also noted that he hopes to implement a more formal testing process for accessibility, but was unsure if that would happen in 2020.

Interview #4 took place after the Fall 2020 semester had started, and Marshall was struggling with the non-stop changes and the workload. Marshall noted that "every time that I feel like things are going back to normal, I feel like the rug is pulled out from under me. It comes in waves, and you ride the wave the best that you can. We are all just waiting for the next wave to hit." Marshall noted that the ticket volume (the number of requests) for problems with the online learning system "was outrageous starting in the 2 nd week of August." Furthermore, there were two large technical problems related to the campus Zoom software.

In planning for the Fall semester, Marshall had taken a number of proactive steps. Marshall had regular conversations with the director of Disability Student Services. Multiple students had requested captioning on Zoom, and Marshall and his team had to plan for that. The plan included integrating REV 14 into Zoom for auto-captioning, and licenses for REV were given to 40 faculty members, a cost that came from the COVID-19 emergency fund on campus. All campus-wide events in fall were online (e.g., orientation, convocation, etc.), and live captioners, not REV, were used for the events, which went smoothly from an accessibility point of view.

At this point, Marshall noted that they still did not have training specific to digital accessibility, but hoped to have training in place at some point in the future. Marshall noted a number of other areas that saw no progress due to the nonstop workflow of the pandemic: no progress has been made on the upgrading of the campus website to the new page template, no work has been done on the annual accessibility report, no work has been done on improving PDF accessibility (aside from Marshall's team writing up some guides), and no work has been done on incorporating testing from people with disabilities into the process. Data from the automated SiteImprove testing tool is still the only major form of testing for web accessibility. One major improvement did take place between January and September: Marshall and his team did VPAT reviews for all technology purchases. While this had not been the case historically, Marshall and his team were able to insert themselves successfully into the procurement process. Currently, the process is that when Marshall receives the procurement documents and the associated VPAT, he has two weeks to review it.

Marshall noted one other bright spot: "the general understanding of the need for digital accessibility review has greatly improved during the pandemic." He noted that due to the increased awareness of accessibility, they have received more requests for help. For instance, Marshall noted that a faculty member, for the first time, contacted them proactively to ask for an accessibility evaluation of their course. Marshall stated that the approach that they always used with faculty was the "early adopter" model, where they would train faculty who wanted to be early adopters, and then have the technically savvy faculty train other faculty, noting that those early adopters become the technology leaders on campus. He also noted that of the various departments on campus, sciences and education are generally the leaders in digital accessibility, with the humanities lacking far behind.

Interview #5 took place in December 2020, as the Fall semester was coming to an end. Marshall started by saying that the Fall semester has been like playing "whacka-mole." Marshall continues to feel that procurement has been a success, noting that for some procurements, he or his staff are able to do basic testing with a screen reader, not just reviewing the VPAT documents. He noted that two tools related to tracking COVID-19 on campus were purchased without him first reviewing the documents. These tools were "fast-tracked" so they were purchased without going through the normal procurement process in which he is involved. COVID-19 has still been the major focus, with multiple new websites being rolled out in fall, all of which required accessibility testing.

Some of the plans for captioning did not pan out as expected. Students reported to Disability Student Services (DSS) that accuracy was a major problem with the autocaptioning provided by REV, so for a majority of the classes which required captioning, the campus switched to CART 15 services. It is yet unclear for the Spring semester whether CART will continue to be used, or if another option, such as otter.AI 16 for Zoom, will be used for captioning. Marshall stated that he looks to DSS to provide feedback on what technology is working well and what is not. Marshall also noted an increase in requests to DSS, primarily relating to students with executive functioning challenges.

The previously mentioned plans for working on PDF accessibility never came to fruition, and Marshall perceives that in fact, PDF accessibility on campus actually got worse in the fall, as campus offices moved more paper forms online, and many of them are not in accessible format. Marshall stated that, "it gave us the opportunity to talk more with campus offices about form accessibility, but it was not received well." Marshall said that there is currently no PDF remediation process in place on campus, and stated that "we need to address PDF accessibility when things settle. It's definitely not great right now."

Summarizing the year, Marshall said that "COVID-19 solidified the need for accessibility procurement processes, and it focused us to test them in real time. Furthermore, it shined light on the fact that faculty will need to be proactive in making online course content accessible." He also stated that "more than any other semester, flexibility has been the key word for the semester." Looking toward the future, Marshall noted that the Spring semester will be hybrid, with a combination of online and in-person learning, with the goal of ensuring that all information is understandable regardless of format, and classes will generally be recorded and available to students. However, faculty will be discouraged from using any whiteboards in the physical classrooms.

The first interview with the director of IT accessibility, "Denise" at the medium-sized university took place faceto-face, in January 2020. Denise's position as director of IT accessibility is a full-time position, where she supervises web accessibility, captioning, and is involved with (though does not supervise or control) procurement of accessible technology. Denise indicated that 2019 was an active year for digital accessibility on the campus, as there was a big increase in digital accessibility training across campus. As of 2019, each of the campus units (colleges and large administrative units) had a technology/IT accessibility liaison, though none of these full-time positions, as these are responsibilities added on to existing positions. The campus uses multiple automated tools for web accessibility testing, and anyone from the campus can request access to the tools. Denise notes that no long-term trend analyses have ever been done using either of the automated testing tools. While Denise's team does not do usability testing, another group on campus occasionally conducts usability testing specifically involving people with disabilities.

The campus website includes an accessibility complaint form, and Denise's team has received a fair number of recent complaints. While Denise's team technically has the right to remove inaccessible content, her team has never done so and would first need to develop a process to do so. Currently, when complaints are received about inaccessible content, Denise's team notifies the content managers and encourages them to make it accessible, advising them when necessary. The current policy on captioning is that it will be done within a week of the request. Denise notes that a longer-term goal is to make accessibility training be a requirement to use any of the content management systems, although that is not currently the situation.

Interview #2 took place in April 2020 after the pandemic had started, and was conducted via Zoom. Denise stated that while no policies or expectations related to digital accessibility had changed since the start of the pandemic, that her team had given more scrutiny to specific websites, such as those that provided data about COVID-19 or about campus adjustments due to COVID-19. Denise noted that videos that would be viewed by the entire campus, such as those that are COVID-19 related, were given priority on captioning. Denise and her team made proactive efforts to reach out to very specific content contributors on campus during the start of the pandemic to ensure that their material was accessible.

Denise noted a number of positive things related to the pandemic experience so far: Denise noted that many of the tools used for campus teaching and research during the pandemic (e.g., Zoom, MS-Teams, 17 Slack 18 ) had already been in use by the campus IT division, and so they were not "new to campus," just new to some people. Denise also stated that "when everything is online but inaccessible, and a person with a disability doesn't have access to anything, it highlights the need for accessibility. I no longer need to make this argument for why digital accessibility is important." Denise said that a fair number of people have contacted her team about making events and content accessible, and she takes that as a good sign of her team's visibility and communication on campus.

Denise noted that "we are in the early days of this [referring to the pandemic], and we're just getting started," also noting that many of the challenges related to digital accessibility are "long runway" problems that take more than a few months to solve. Denise stated, "we eventually need to move beyond the emergency crisis mode, and think longer-term about the fall semester." Denise also noted with concern that for the first 2 weeks of the pandemic, she had staff members appropriated to other IT groups to assist with other tasks, but by the 3 rd week, they were all allowed to return to their core work on accessibility. Denise stated: "we have been very much in react mode in March, but I hope that we'll be back to a more normal operation soon, so that we can work on our normal goals. But hopefully we can use stories and examples from this time to strengthen the business case for digital accessibility. It demonstrates the need to fund these accessibility services, and in an online-only environment, the role of our team is critical."

Denise also noted an uptick in the number of complaints received, as "people on campus have been reporting accessibility problems when their bosses don't listen. We don't want to be the accessibility police, but we do reach out to noncompliant units, which magnifies the stakes a bit for the offending unit." Denise noted that when reaching out, her team states that an accessibility problem was reported, and provides specific suggestions on how to fix it, such as for web pages or PDF files. Denise stated that some of the accessibility fixes, such as for third-party software, are really hard to accomplish and out of scope for her team. However, in those cases, Denise's team notifies the campus unit which owns the technology procurement contract, and offers their help in terms of expertise and meeting with the vendor. Denise is very sensitive about the political nature of her job, stating, "it doesn't help us to do the work over the heads of others. We are better when we assist and support the work of others."

Interview #3 with Denise took place after the Spring 2020 semester had ended, and Denise reported that there had been no policy changes made related to digital accessibility, and her focus and the focus of her team continued to be on public websites and videos, not internal content. Denise stated that she uses her team's work on external content, to basically advertise that they are available to help with internal content, as well. Denise suggested that she and her team are receiving more requests for help with making online meetings and events accessible, than they are for classes. The number of requests for help on both web accessibility, as well as captioning, continued to increase throughout the Spring semester. Denise noted that perhaps this is not surprising, because she and her team continue to provide more training and more presentations throughout the campus, and she feels that the ongoing outreach has had a major impact.

Since the previous interview #2, Denise noted that there has been a large increase in the number of technology procurements, many being done without checking for accessibility. Denise stated: "frequently, we receive calls from other offices on campus saying, 'we're going to sign a contract with a vendor today, can you please look over the paperwork before we sign' which is completely last-minute and unrealistic." She noted that the procurement processes on campus are "not mature yet" related to digital accessibility, and there are no policies in place to enforce that. Denise stated, "we only vet products when people ask. It's pull, not push. It only occurs when someone is a good university citizen."

Denise summarized her goals for the next few months: to get out of the "spring emergency mode" and plan ahead for the fall semester, to scale up for more students and faculty to use these new technologies in their fall classes, and to be overall more thoughtful in the approaches that they use. Denise also noted that she's still trying to educate and evangelize to people, and that she can't investigate every procurement contract, and perhaps they don't even know if they are getting involved in the right contracts, so long-term she and her team need to find a way to become more strategic about dealing with procurements.

Interview #4 took place right as the Fall semester was beginning. Denise stated that over the summer, the amount of online training that they provided has continued to increase, with larger numbers of faculty and even graduate assistants signing up for the classes (whereas the majority of previous participants in accessibility training were staff members). Denise said that her approach was to let people know, "if you can get a group of people together, we will put together special training for your group!" Denise's team increased the range of training, instead of offering general introductory concepts, focusing more on role-based concepts needed for content editors, developers, document authors and purchasers. The number of people taking training, and the training offerings had increased during summer 2020, and the number of accessibility complaints also continued to increase. Denise thought that an increased number of complaints was actually a sign of success: people are feeling more comfortable making complaints, and confident that their complaints are followed up on. The number of requests for captioning and help making virtual events accessible, continued to increase.

Denise stated that in a few areas, no progress had been made between interview #3 and interview #4: procurement was still problematic, no further work had been done on PDF accessibility, and no additional resources had been given to the accessibility coordinator in each campus unit. Denise worried that the number of forms used on campus had increased this fall, without any outreach or infrastructure from her team on form accessibility being in place. She was also aware that a lot of technology procurement had been done without checking with her team. Denise hoped that at least the communication and relationship building that her team had been doing, might have benefits in the long-term if not the short term.

Thinking ahead, Denise was optimistic that progress had been made in terms of communication, outreach, and that next steps could be to improve the processes in place and make them easier. Denise's team had recently put together campus guidelines for how to put together accessible online events, and was wondering what steps could make the captioning request easier. Perhaps better tools for captioning, or making it easier for campus units to request CART service? While Denise felt comfortable with the progress made over the summer in terms of event accessibility, she was still concerned about procurement, and the "blow-back that will occur soon from all of the inaccessible technology that was procured." Denise stated one additional concern: "It's hard to keep the focus on accessibility among decision makers, when there is so much coming at them right now."

Interview #5 took place with Denise at the end of the Fall 2020 semester. Her overall assessment was that, "this fall, things went OK. We didn't do everything perfectly everywhere, but people [in the university community] were generally proactive." Denise noted that more faculty and staff had reached out to her team, and described the increase in requests as "dramatic." She stated, "as far as I know, nothing blew up, although it's possible that things did blow up and the problem was handled by the Disability Student Services office on campus." She noted that attendance in the training courses has continued to increase.

Denise acknowledged that technology procurement continues to be a problem, stating: "so much was procured this fall, we know that we missed a lot. A lot of agreements were made on an 'emergency basis' outside of normal procurement challenges, often with contracts signed in a week instead of three months." Denise gave an example of when she tried to block a technology procurement and was told by the person responsible for the procurement that, "accessibility is one of many procurement checks that I'm bypassing because I need to get an online lab installed right now, today." Denise opined that even if her team tried, it would not be effective to try to block any technology procurements, but she expressed concern about the future: "we are going to be reaping the rotten effects of those contracts signed without accessibility, for years to come."

Denise said that the need for captioning has increased greatly, noting that captioning generally went well, and her team only received one complaint related to captioning in the fall. She stated: "there may be content that isn't captioned, but we haven't heard about it and overall we think that we're doing ok." One of the challenges in Fall (and expected in Spring 2021 as well) is that captioning vendors have had a much slower turnaround time than normal, due to the high demand from universities (and other organizations) elsewhere. Denise discussed how her team educates people on the differences between human-based captioning and AIbased captioning, explaining for instance, why the costs of AI captioning are low, which often is the only consideration for decision makers concerned about the bottom line. Denise noted that "it's hard to educate people on what's sufficient, on what can be considered quality captioning. We spend time deconstructing misconceptions." Denise further noted: "people say, 'oh, I didn't know, you're saying that AI-based captioning doesn't count?'".

Denise also noted that the demand for live captioning for online events has also skyrocketed, and it poses challenges because live captioning is logistically hard to do and again, vendors are under high demand to provide live captioning (with limited availability to get live captioners). In the past, only those running large events asked about live captioning, but now, people all over campus are running online events and asking for live captioning, even for small groups.

Denise also noted some progress on long-term goals: while her team has not removed any inaccessible web content, they have forced people to change inaccessible content when they have received multiple complaints filed by faculty, staff, and students. There has been some basic progress on PDF accessibility, as more people take the online training on document accessibility, and her team has gotten a new license for software to assist with document remediation. Denise stated some concerns about their current approaches: "We always correct and promote accessibility whenever potential problems are pointed out to us," but she would like to make their approaches more proactive in the future. Denise noted that her team was able to proactively review most COVID-19 information for accessibility before it was posted, and hopes to take more of that proactive approach in the future. Denise did not expect any major changes in Spring 2021, but still hoped for she and her team to be able to be more proactive with their work.

Denise closed out the interview with some overall thoughts: "I was positively surprised how the remote pandemic situation has gotten more people to think about digital accessibility, more than they otherwise would have. It's been a breakthrough situation, causing people to proactively address accessibility in a way that they otherwise might not have done. Now people come forth and have joined our sphere of awareness."

The first interview with the director of IT accessibility, "Neil," at the large-sized university took place face-to-face, in February 2020. Neil described how his team is responsible on campus for coordinating web accessibility, multimedia (primarily captioning) accessible learning, and accessible technology procurement for the campus. Neil also regularly communicates with the assistive technology (AT) lab on campus (which works primarily with students), disability student services (which works only with students), and the web content managers in each academic unit on campus (none of whom have that role as a full-time responsibility). Neil mentioned that both the disability student services (DSS) office and the AT lab are under-funded and over-tasked, and some of the spill-over comes to his office.

The university has had a web accessibility policy in place since 2017 and the primary automated tool used for web accessibility testing is SiteImprove. In terms of accessible procurement, Neil described how all technology procurements are required to go through security, privacy, and accessibility testing. Neil notes that he and his team will take a quick look at the VPATs (Voluntary Product Accessibility Templates), though they generally do not do any testing, either themselves or involving people with disabilities. Neil or a member of his team will ask tech vendors for their longterm "roadmap" for accessibility, to ensure that accessibility features are not lost in future versions of software. In terms of captioning, there is a campus-wide policy for captioning, but no central funding source for captioning (each academic unit is required to pay for it). Neil noted that the administration tends to be more concerned about the accessibility of public content, than content located behind authentication (which requires use of the campus single-sign on).

In terms of longer-term planning, Neil has a number of goals for him and his team. He hopes to develop accessible web page templates, and establish best practices for procurement, accessibility testing, remediation, and online learning. Neil hopes to develop policies related to social media and live captioning at events. He also hopes to have accessibility be a stronger component of coursework and faculty in the future, specifically, to make accessibility details be a requirement in any new course proposals coming to the university-wide curriculum committee, and for accessibility to be required to be included in teaching portfolios submitted for promotion and tenure. Neil also hopes for increased funding, at a central campus level, for web accessibility and captioning. He also hopes that increased and centralized funding, and additional staff in the future, could assist in having expert reviews and usability testing involving people with disabilities, for new technology procurements and web development.

Neil noted that for the short term, he hopes to increase the amount of accessibility training given to academic units on campus. One challenge that Neil noted in terms of getting centralized funding, is that disability generally has not been viewed as a "diversity" issue by central administration. Furthermore, the structures have not been in place for long, to enable university members to file complaints related to inaccessible technology. Informal complaints have been made by faculty and staff within the university who have disabilities. So, Neil postulated that in many ways, the central administration does not have a good understanding of the many problems that exist in the university related to digital accessibility, because, "we don't have the infrastructure at [university name removed] that we need."

Interview #2 took place in June, after the pandemic had started. Neil described how the central administration had added new staff positions and funding to help his team deal with the increasing responsibilities during the pandemic. As part of that, his team will be focusing on accessibility of the top 25 most-visited websites on campus. Neil noted that there had been more requests for captioning videos; however, it is "unlikely that all public videos are captioned" and for the Fall semester, online courses will not be required to be accessible, they will only be made accessible if there is a student with a disability registered for the course.

Neil stated that the largest events on campus that were held virtually (town hall with the president, main commencement), had live captioning, though not all public events have had live captioning. Neil thought that there was likely "a slight improvement in web accessibility, as more people have become aware of accessibility, and as sites get updated and redesigned, in the last year they are more accessible." At this point, Neil was planning ahead for the Fall 2020 semester. One of the approaches that Neil and his team were using was to increase the amount of online training available in the summer, to faculty and staff. Neil noted that there have not been any changes in accessibility policies or governance due to the pandemic, but at the beginning of the pandemic, he was taken away from his normal accessibility-related responsibilities to work on the creation of COVID-19-related websites. Neil also mentioned that there has not been any progress yet on the accessible web page templates.

Interview #3 with Neil took place at the beginning of the Fall 2020 semester. Neil noted a number of successes over the summer: there was a great increase in the number of people taking online training over the summer, over 500 people took accessibility training online. Another success was that both his team and other offices on campus that were involved in digital accessibility, increased their staff during the summer. The provost made small grant awards to faculty who proposed to re-work their classes for the online environment, and a majority of those proposals included accessibility components. Neil stated that due to the proposals, "I've never had so many people asking me about captioning before!" Neil also mentioned that the summer orientations for new and transfer students all included live captioning. Moreover, his team has increased their focus on ensuring accessibility in the top 25 websites, to the top 30 websites, noting, "there has been a lot more intentionality than before." Neil stated that he and his team do not just wait to be contacted:they reach out to campus members running public events to discuss live captioning, but even some events slip by-one prominent online event run at a VP-level, was very inaccessible. Neil mentioned one other influence-a student leader with a disability wrote up a report on campus inaccessibility and submitted it to the upper administration-bringing more attention to where the university was not being inclusive.

Given all of the successes, Neil also had a lot of concerns. With the increase in captioning required, there still is not centralized funding for captioning, and a lot of campus is still not engaged with captioning. Neil thought that you cannot force the campus to caption all videos if you do not provide any funding for captioning, and mentioned that he is always telling upper administrators that "you need to be working on this!" Neil specifically mentioned the tension between pushing offices on campus to caption their videos, and the preference for AI-based captioning, which costs less than human-based captioning. Transfer students greatly increased in fall 2020 so he expects an increase in demands for accessibility, and there is a lot of uncertainty around the budget given the pandemic. Neil also expressed his concern that while the university has been great about letting his team work remotely during the pandemic, he is concerned that the university may soon force him and his team to return to working on the physical campus again.

Interview #4 took place right at the end of the Fall 2020 semester. Neil stated that nearly everything related to digital accessibility was in high demand, with a very high number of requests in Fall 2020. His team had a lot of requests for manual testing, training (both classes and one-on-one training), and captioning. Neil said that he and his team were not able to meet all of the requests, especially the requests to do thorough manual testing for accessibility. His team basically did quick reviews for accessibility basics, and told individuals instead that they could receive training on how to do a manual test.

While overall training demand was high, it was not as high as during the summer. An increased push for training is planned for the January semester break and Neil noted that over 100 faculty and staff are already signed up for January training. Neil saw a similar trend in usage of the SiteImprove automated web accessibility testing tool: there was a large increase in usage during Summer 2020, but the usage levelled off in the Fall semester.

Given the high demands for Neil and his team, he was happy about the progress made in a few areas: during the Fall semester, a stronger captioning policy was put in place, and new standardized text for accessible technology procurement was approved. In addition, half of the academic and administrative units on campus now have an IT accessibility plan in place. Neil noted that, unfortunately, two colleges which have well-staffed IT groups, have continued to pushback on some aspects of IT accessibility. Given previous problems on campus with PDF document accessibility, Neil also happily noted the procurement of two software applications, available to all campus members upon request, to assist with remediating inaccessible documents.

Given the new captioning guidelines on campus, Neil is investigating other options for captioning, noting the high demand for both video captioning and live event captioning, and also the inadequacy of automated (AI-based) captioning. Neil noted that no major changes, in policy, or staff, are expected in the Spring 2021 semester. Neil was generally happy with the progress made on digital accessibility during the pandemic, but expressed concern about high demand for accessibility services in the Spring 2021 semester, as well as potential budgetary cuts. He also noted a few areas that still need improvement: centralized funding for captioning, all campus units having an IT accessibility plan, and rolling out new initiatives such as campus-wide use of the accessible web page template and the development of best practices documents that can be widely distributed.

For a comparison of how digital accessibility is being managed outside of the university setting, interviews took place with the manager of IT accessibility, "Janice" at a state government agency. Janice has a position where her primary responsibility is managing accessible IT procurement and accessibility testing for a state agency. This would be similar to the "Section 508 coordinator" role at the Federal level, with similar responsibilities; however, Section 508 of the Rehabilitation Act does not apply, legally speaking, at the state level. Janice's work on accessibility at her agency also involves assisting some other state agencies, although she is not by any means in charge of IT accessibility for the entire state, and in fact, there is no such role of "state director of IT accessibility."

The first interview took place at the beginning of March 2020, right before the pandemic had been officially declared, when people were already starting to become aware of the coronavirus. Janice mentioned that she has been asked to assist in making the website for COVID-19-related information accessible, so that was distracting from some of her normal workload. One of the big challenges at that time was how to make maps and data visualizations related to COVID-19 accessible, something that Janice noted that she is currently working on.

Janice described how she and her small team typically run automated scans of websites using WAVE, ANDI, 19 and Dynomapper, 20 and do manual inspections of websites involving screen readers, checking that labels, buttons, and such, are fully accessible. Janice also described how she and her team have been working on improving PDF accessibility. One project that Janice was excited about, was the "accessibility testing template" that she created to help people in her agency and potentially other agencies accomplish accessibility testing themselves, and hopes that the use of the template could assist with testing and identifying errors more easily.

One of Janice's goals for this year is to get someone hired to provide training, to help increase the capacity of accessibility testing in her agency and more broadly in state government. There is a state-level department of digital technology, independent of all other state agencies, and some of the state agencies consult with that department on accessibility, while others do not, but Janice feels that the lack of central management of digital accessibility leads to many inaccessible and unwelcome outcomes. Janice said that from her position in her state agency, she is trying to be "a one-woman army" spreading the message of accessibility, engaging with as many other agencies as possible. Janice said that her common message is: "websites are easy to make accessible, if you do it from the start, and the law requires that you do it!". Janice expressed frustration with technology procurement in the state, saying that, "procurement people really don't understand IT accessibility," and she and her team are trying to work with state-level procurement officials to educate them and perhaps to "put the fear of God in them" about accessible procurement. She hoped that later in 2020 there would be some changes in procurement. Janice mentioned that she has generally found those involved with technology across the state government to be mostly sympathetic and sometimes even supportive, even though they frequently ask "how are we going to do this?" and sometimes she is even asked, "can we just skip this step?" Janice stated that vendors who promise accessible content or software and then deliver inaccessible content or software, are the biggest problem, saying, "we need to stop using these vendors [who] can't make accessible content."

Interview #2 with Janice took place in June 2020, when the pandemic was well underway. Janice was excited that since the first interview, an additional staff member was added to her team to assist with accessibility testing. But Janice discussed how, during the pandemic, she was worried that many IT procurements were slipping through without considering accessibility. For instance, one request for proposals from a technology procurement went out saying that accessibility was optional, not a requirement, which greatly upset her. In another example, a website for economic and employment-related information related to COVID-19 developed by a vendor for the state was not accessible. Janice and her team were only consulted after the site was implemented, to perform accessibility testing, and the vendor is now making updates based on feedback from Janice and her team. Janice expressed frustration that the vendor, after originally building the site inaccessibly, keeps coming back to her team to ask them to re-test it. Janice wonders why the state agreed to a procurement with a vendor who does not seem to have any accessibility expertise on their staff: "if we're doing the testing for the vendor, can the state bill the vendor for our time?" Janice states that in her opinion, the biggest problem related to accessibility in the entire state, is outside vendors who get paid well to build websites for the state, but build them inaccessibly and then get angry when you ask the vendor to make it accessible. She notes, "it's insane, we're testing it for the vendor because the vendor has done such a poor job" and notes that, "it is a lot of pressure on me and my staff" and that she has the expertise but not the authority to solve these procurement problems. Janice has encouraged her higher-level managers and administrators to allow her access to more procurements earlier in the process. Janice and her team have been creating a "procurement cheat sheet" to quickly advise procurement officers.

Janice estimates that the workload of her team is now 70% on COVID-19-related technologies, and says that the COVID-related work has distracted from some of the other goals beforehand, such as offering online training sessions for state workers (including those in other agencies), and doing automated and manual testing. Janice's team has setup a Google form to collect COVID-19 problems related to digital accessibility. She says that currently, even if they are running automated accessibility testing, they do not have the time to summarize and write reports that are useful to others, because COVID-19-related issues keep coming up. As the state continues to make changes to web content related to COVID-19, "accessibility issues keep coming up [related to] COVID-19, as they add bells and whistles non-stop."

One of Janice's big concerns right now is the use of teleconference software by employees at her agency and elsewhere within the state government. The state required the use of one of two teleconference software applications, both of which have accessibility barriers. Janice said that this is just one example of how accessibility does not influence big software decisions within the state government.

Interview #3 took place with Janice in the beginning of October. Janice noted that by now, she and her team are doing less COVID-19-related work than before. One of the major websites with a vendor that was causing headaches for her team, is now re-launched and is more accessible, although still not working properly for VoiceOver 21 users. Janice was enthusiastic that now she and her team was able to get back to working on some of her ongoing interests: training and procurement, saying, "things are looking up right now!" Accessibility training was scheduled to be offered by her team starting later in October. Four different training courses will be offered, with a goal of one training session per month: (1) accessibility basics, (2) advanced accessibility, (3) document accessibility, and 4) accessible IT procurement.

Janice felt that she has made much progress with procurement, helping to craft (with other state workers in other agencies) standard procurement language for IT accessibility, and becoming more well-known and visible to the procurement leaders in the state. For instance, after her complaints related to inaccessible teleconferencing software, the options for state employees were expanded to include additional, more accessible options. Even though she is happy to help, Janice is frustrated that often, technologies are procured, rolled out, the state receives complaints about inaccessibility, and then the requests for information about accessibility "come down the chain" to her and her team. She hopes that, with more visibility within state government, those responsible for technology would contact her and her team first, saving time, money, and frustration. For instance, multiple learning management systems are used by state agencies for training, and almost none of those are accessible. Given the existing number of learning management systems on the market that are accessible, Janice wonders why this is the case, suggesting that the guideline should be "don't hand out any money until you're sure that it's accessible." Related to this, Janice noted that when people within the state government internally create software applications or websites, they frequently are accessible, or at least accessibility is brought in earlier in the process, though when software or websites are procured from outside vendors, that is where the biggest problems occur, because most vendors do not have the expertise on accessibility, so they just ignore it. One further frustration of Janice is that when she contacts vendors, often the vendors have subcontracted out portions of the development, so that it takes five different email messages just to reach the right person. Janice suggested that in the future, perhaps the state should focus on hiring only local vendors who have all work done locally.

Interview #4 with Janice took place in late December 2020. Janice said that not much has changed or progressed since interview #3. There was a steady flow of incoming requests for doing accessibility testing. Unfortunately, there were many new technology procurements without checking for or requiring accessibility. New websites for COVID-19 information, vaccination, and medical records were built inaccessibly. The same problem as before continues to occur, where complaints were filed by either citizens or other state employees and then the requests trickled down to Janice and her team to provide insights and testing. While Janice and her team will test and chat with a vendor, they will not do any remediation, as the vendor is required to do that. Janice was frustrated, saying, "anything COVIDrelated, they ran through procurements without checking for accessibility!" Janice is further worried that even though much of the COVID-related work will trail off by Fall 2021, "certain [state] agencies may continue to use emergency COVID-related procurement powers" to avoid checking for accessibility.

The training courses offered by Janice and her team have started, although enrollment was not very strong in the past two months, and she hopes that enrollment will increase as more people become aware of the training courses. Janice continues to believe that due to planned changes, procurement will get better in the long-term. Starting in 2021, more IT accessibility training will be designed specifically for and offered to state procurement officers, involving information on what to look for in a CFP [Call for proposals] 22 and describing some common accessibility mistakes. Janice is hopeful that this new approach will help solve some of the accessibility problems in procurement. At the same time, Janice is worried that because she and her team are providing the training, procurement officers may simply call her 21 https ://www.apple .com/voice over/info/guide /_1121.html. 22 A Call for Proposal (CFP) is a document that organizations put out, seeking proposals from vendors to do the work described. CFPs need to specify accessibility requirements up-front, so that vendors include accessibility details in their proposals. Even when proposals include accessibility details, they must be documented and managed and checked for compliance throughout the entire procurement process. office about every single IT procurement, and the workload may become overwhelming. This is especially problematic, because her office does not specialize in procurement, and she does not have a large enough team to become a "stopping point" on every single IT procurement. Janice feels that both the vendors and the state procurement officers need to go through training on IT accessibility and be able to make the right decisions amongst themselves, because otherwise, "we might be tweaking every procurement CFP, every bid" and that is something that Janice and her team do not have the capacity to do (nor do they have the statutory authority to do so). In many ways, Janice is an advocate for accessibility, trying to get attention for the topic within state government, always trying to talk with "higher-ups," as Janice described her approach: "get to the highest branch on the tree that you can" and teach them about accessibility. While Janice does not expect any new policies or testing approaches in the first half of 2021, she is worried about the potential increase in workload related to procurement.

From the interviews involving Janice, Neil, Denise, and Marshall, some key themes repeatedly appeared. These are discussed in the following sub-sections.

Interviews were conducted at particular time milestones (e.g., before the pandemic, after the spring semester, before the fall semester, etc.) and some consistent patterns definitely emerged across all the three interviewees at universities (n.b.: the milestone times were not relevant for the interviewee at the state government agency). Before the pandemic, the interviewees had specific plans for what they wanted to accomplish related to digital accessibility during 2020, for instance, Marshall at the small-sized university wanted to educate more content creators about accessibility, reduce the use of PDF files, and increase the use of the accessible page template and automated testing. At the beginning of the pandemic in March, all four interviewees were struggling, overwhelmed by additional work, and pushing hard to make the initial COVID-19-related information posted online accessible. Their original work plans for 2020 were already pushed by the wayside. When campuses moved from primarily in person to exclusively virtual, this was an unplanned changed, and all three interviewees who work at universities were helping to support this change and distracted from their normal work routines. By the end of the Spring semester, the interviewees from the universities were taking account of how many technology procurements "slipped through the cracks" without accessibility checks. By summertime, those interviewed were providing summer training on accessibility, planning ahead for a strictly online Fall semester, and planning accommodations for students with disabilities, primarily the captioning needed for online classes. By the beginning of the Fall semester, enrollment in training classes decreased, while requests for help (or "tickets") increased. Toward the end of the Fall 2020 semester, those interviewed at the universities were trying to figure out how to manage both the budgetary needs and the capacity needs (for captioning and other accommodations), while trying to plan ahead for the Spring 2021 semester, which increasingly looked like it would be strictly virtual. The patterns and challenges were similar across all three universities given similar timeframes.

All four people interviewed discussed the challenges related to accessible technology procurement during the pandemic. Marshall, at the small-sized university, was actively trying to become inserted in all of the technology procurements, to evaluate for accessibility, and thought that he and his team had been able to catch most things. However, Marshall was aware that two software tools related to tracking COVID-19 on campus were purchased without him first reviewing the documents, using "fast-tracked" authority. Denise, in the medium-sized university, stated that their procurement processes were not yet mature, and she was aware of the fact that many procurements of inaccessible technology were made during 2020, fearing that "we are going to be reaping the rotten effects of those contracts signed without accessibility, for years to come." Denise also noted the "emergency procurement" powers to bypass accessibility checks, just as Janice noted similar emergency procurement powers within the state government agency, and Marshall noted the "fasttracking" authority at the small-sized university.

Neil, at the large-sized university, had been working on standardized text for procurement, and by the end of the year had gotten the standardized text approved, but at no point described procurement as either completely successful or a complete failure. Janice, in state government, was continually frustrated by inaccessible technology procurements, noting that she was contacted repeatedly, only after the state government procured something that was inaccessible. Janice was especially frustrated that one request for proposals for a technology procurement went out saying that accessibility was optional, not a requirement. All four interviewees described challenges with procurement, how they were trying to get more control and influence over the process, and how inaccessible technology procurements were being made using "emergency authority" (or equivalent). The boldness with which procurements were made, was surprising, as Denise described one individual who said, "accessibility is one of many procurement checks that I'm bypassing because I need to get an online lab installed right now, today." Two clear themes emerged related to procurement. First, the people charged with managing digital accessibility were often being bypassed when it came to procuring technologies, often being contacted only after the fact. Second, "emergency procurement powers" had been used in all four organization contexts, which made the situation worse, and the interviewees hope that those emergency powers could be curtailed when the pandemic is over.

Captioning was an ongoing challenge for the three universities, although it did not seem to come up as a major issue for Janice in the state government (perhaps due to the state government agency putting out very little multimedia content). For the universities, they all have large numbers of videos that need captioning, as well as livestream events that need live captioning. Marshall, at the small-size university, noted that captioning is done proactively for public content, but is only done reactively for courses upon request. Denise, at the medium-sized university, also noted the administrative priority on public content over content behind a single-sign on authentication, and noted an even higher administrative priority for captioning publicly posted videos related to COVID-19. Neil, at the large-sized university, echoed Denise's comments from the medium-sized university, that the administration tends to be more concerned about the accessibility of public content, than content located behind single-sign on authentication.

Marshall at the small-sized university, and Neil at the large-sized university both noted the challenge of funding for captioning: Neil noted that there was no central fund to pay for it, and Marshall noted that when there are multiple stakeholders involved in content, none of the stakeholders want to take budgetary responsibility for the captioning. Both Neil at the large-sized university and Denise at the medium-sized university describe the dramatic increases in the number of requests for captioning, for both videos and for live events, and also noted the increased delays in captioning, and the shortage of live captioners available. Marshall at the smallsized university noted that auto-captioning using REV inside of Zoom was deemed insufficient, and that they had to switch over primarily to CART services for classes instead. Denise at the medium-sized university described the need to educate stakeholders about how AI-generated captioning was not sufficient, and that human captioning was necessary, in a way that was nearly identical to what Neil reported at the large-sized university.

Overall, four themes emerged related to captioning: (1) there was a very large increase in the number of requests for captioning, (2) there was a need to differentiate between computer-generated captioning and human captioning, (3) clearer mechanisms and funding sources needed to be created for paying for captioning, and 4) there was an emphasis on the accessibility of publicly available videos over videos behind a "password wall" (requiring single-sign-on authorization).

All four people interviewed noted how the pandemic had interrupted their normal work goals and plans. Marshall, at the small university, noted how the pandemic had stopped their efforts to address PDF accessibility and work on rolling out the new accessible web page template to the entire university web site, and even to work on the annual IT accessibility report presented to the campus. Denise, at the medium-sized university, noted how during the first two weeks of the pandemic, she had staff members appropriated to other IT groups to assist with other tasks. Neil, at the large-sized university,, described how their high workload related to the pandemic limited work on accessibility testing and assisting academic units on putting together their IT accessibility plans. Janice, at the state government agency, described how she had been asked to take on additional work related to new websites for COVID-19, limiting her and her team's ability to work on testing, reporting, and outreach. One clear theme emerged: the start of the COVID-19 pandemic certainly distracted all interviewed from their normal work, and the long-term impact of projects that were stagnant for a year, or that slowed down, remains to be seen in the future. Only one of the four interviewees actually had members of their team reassigned to work on other projects at the beginning of the pandemic.

All four of the interviewees described the need for training, and their efforts toward increased training. They all described how the capacity of themselves and their teams was limited, but how they could magnify that impact by training others within their organizations to understand accessibility and do testing and remediation. Marshall, at the small-sized university, noted their "early adopter" model, where they would train faculty who wanted to be early adopters, and those early adopters would then train other faculty on campus, serving as mentors and leaders. Marshall did not get to add online training classes specifically focused on digital accessibility in 2020 as he had hoped to do. Denise at the medium-sized university, Neil at the large-sized university, and Janice at the state agency, were all able to increase their online training offerings during 2020, with multiple courses available. While Denise hoped to make accessibility training a requirement for content creators, none of the organizations involved had digital accessibility training required as of the end of 2020. Both Neil and Denise mentioned that the number of people in their organizations who enrolled in digital accessibility training greatly increased over the summer. An overall theme is that all of the interviewees are either increasing their online training offerings related to disability, or in the case of Marshall at the small-sized university, he is attempting to start offering online training.

It is interesting to note that multiple interviewees mentioned PDF accessibility as a challenge in their organization, even though the topic was not part of the original list of questions for interview #1. Marshall, at the small-sized university, mentioned at the beginning of the year that he and his team were planning to reduce the number of PDF files on campus because of the difficulty in making the PDF files accessible. By the end of 2020, Marshall thought that PDF accessibility on campus had actually gotten worse, as campus offices moved more paper forms online, and rarely created forms in an accessible format. Denise, at the medium-sized university, was aware that PDF accessibility on campus was a problem, and little progress had been made through most of the year, but by the end of the year, more community members were taking the new online training for document accessibility. Both Denise at the medium-sized university, and Neil at the large-size university, had procured licenses for software applications to help address the challenges of remediating PDF files for accessibility. Janice at the state agency also mentioned the problem of PDF accessibility, and like Denise at the medium-sized university, she had started offering training classes online about document accessibility. The overall theme is that to address PDF accessibility, three approaches were being taken: (1) try and reduce the number of PDF files used in the organization, (2) provide training on PDF/document accessibility for content creators, (3) purchase software tools for PDF remediation.

It is important to note that with minor exceptions, no new policies were created to manage IT accessibility during the pandemic. At the small and medium-sized universities, no new policies were put into place. At the large-sized university, an existing policy on captioning was strengthened. Given the limited resources and increased demands, tradeoffs were sometimes necessary, and it is surprising that there was no formal approach, no new policy, for triaging or prioritizing requests reported by any of the interviewees for managing digital accessibility.

While those managing IT security often plan for emergencies and extreme scenarios, those managing digital accessibility do not typically have similar plans in place. Before 2020, no university which was a primarily physical university, expected to move their operations fully online at any point in the future. The planning for capacity and throughput related to digital accessibility, simply did not exist. As the COVID-19 pandemic continues in 2021, and as planning is done for future pandemics, the stories emerging from the interviews may provide suggestions for how to manage digital accessibility during a pandemic: 1. Find ways to ensure that accessibility requirements remain in procurement during emergency situations; 2. Limit "emergency procurement authority" which bypasses accessibility requirements, and eliminate it as soon as it is feasible;

3. Provide as much online training as possible related to digital accessibility, and plan for increased enrollment during the semester breaks (summer and winter); 4. Provide grants or incentives to faculty and staff outside of the core digital accessibility team, who want to spend time learning about and implementing digital accessibility at the university; 5. Assume that there will be periods of time where classes will switch back and forth between being strictly virtual and partially in-person. Plan ahead to have the human and computing capacity for captioning, accessibility testing, and other accessibility activities when the university is running on a primarily or strictly virtual basis. Have agreements in place with outside accessibility vendors as necessary; 6. When "surges" of digital accessibility work occur, have clear policies in place to ensure additional budgetary resources, staff time, and to eliminate reassignment of staff working on digital accessibility; 7. During periods of relative normalcy in between shutdowns, educate the campus community about issues which require understanding during the periods of stress, such as document/PDF accessibility and differences between human-based and automated/AI captioning.

No one had predicted the COVID-19 pandemic, and so managers of digital accessibility had not planned for the massive changes in their work and the needs of the university. In examining the stories presented through the 18 interviews, there were many common problems faced across the three universities and the state government agency, related to procurement, document accessibility, accessibility training, and captioning. Understanding where the challenges occurred can help in planning for future digital accessibility needs during times of extreme stress. It is even possible, as noted by interviewee Denise, that the pandemic can assist in highlighting the need for digital accessibility: "…the remote pandemic situation has gotten more people to think about digital accessibility, more than they otherwise would have. It's been a breakthrough situation, causing people to proactively address accessibility in a way that they otherwise might not have done." To quote interviewee Marshall, "the general understanding of the need for digital accessibility review has greatly improved during the pandemic."

Everyone hopes that the COVID-19 pandemic ends soon, and that there are no similar pandemics anytime in the future. However, the knowledge gained from these interviews can help in managing digital accessibility in "normal" or nonpandemic times. From the interviews, there were common challenges that exist at universities, such as captioning, PDF files, the need for stronger control over procurement, more training, and limited funding for all of these. The COVID-19 pandemic brought more attention to these challenges, and or any universities that do not have plans in place to address each of these areas, it would make sense to start planning to address each of these. Furthermore, there seemed to be a common problem that "surge capacity" did not exist for captioning and other accessibility-related services. One could easily imagine surges in student enrollment occurring due a number of factors, such as students moving around to other universities due to climate change, political unrest, or even funding changes. For example, hurricanes and political shifts (such as Brexit) have led to unforeseen changes in student enrollments as students change universities and/or geographic areas. Universities need to plan ahead to ensure that digital accessibility can be managed when "surge situations" exist. Similarly to how companies prepare for having extra computer processing power on demand, when needed, universities need to plan ahead for having surges in demand for accessibility services, at unexpected times. Doing so would benefit universities at all times, even without a pandemic.

A case study in practice: providing accessibility for persons with disabilities at the College of Staten Island Library

A web accessibility report card for top international university web sites

Improving real-time captioning experiences for deaf and hard of hearing students

Accessibility of Turkish university Web sites

Research methods in humancomputer interaction

Ensuring digital accessibility through process and policy

Due Process and Primary Jurisdiction Doctrine: A Threat to Accessibility Research and Practice

Rulings, Filings, and Letters

Evaluation of Web content accessibility in an Israeli institution of higher education

Convenience and accessibility of library services to students with disabilities at the University of Limpopo in South Africa

Best practices for teaching accessibility in university classrooms: cultivating awareness, understanding, and appreciation for diverse users

Implementing accessibility initiatives at the Michigan State University libraries

Doing the right thing: one University's approach to digital accessibility

Web accessibility: a longitudinal study of college and university home pages in the northwestern United States

Letter: Electronic Book Readers

Acknowledgements Portions of this work were funded by a Google Faculty Research Grant. The author greatly appreciates the financial support of Google, and any opinions included herein are solely those of the author.