key: cord-0829827-bclokshg authors: Kaul, Smiti; Coleman, Cameron; Gotz, David title: A rapidly deployed, interactive, online visualization system to support fatality management during the coronavirus disease 2019 (COVID-19) pandemic date: 2020-10-11 journal: J Am Med Inform Assoc DOI: 10.1093/jamia/ocaa146 sha: 2db5a728760653e6a2e7c347c21c4039ee4369a8 doc_id: 829827 cord_uid: bclokshg OBJECTIVE: To create an online visualization to support fatality management in North Carolina. MATERIALS AND METHODS: A web application aggregates online datasets for coronavirus disease 2019 (COVID-19) infection rates and morgue utilization. The data are visualized through an interactive, online dashboard. RESULTS: The web application was shared with state and local public health officials across North Carolina. Users could adjust interactive maps and other statistical charts to view live reports of metrics at multiple aggregation levels (eg, county or region). The application also provides access to detailed tabular data for individual facilities. DISCUSSION: Stakeholders found this tool helpful for providing situational awareness of capacity, hotspots, and utilization fluctuations. Timely reporting of facility and county data were key, and future work can help streamline the data collection process. There is potential to generalize the technology to other use cases. CONCLUSIONS: This dashboard facilitates fatality management by visualizing county and regional aggregate statistics in North Carolina. By late May 2020, deaths due to the novel coronavirus disease 2019 (COVID-19) had surpassed 100 000 in the United States, 1 though public health experts suspect these counts underestimate the true mortality burden. 2 Surges in fatalities have overwhelmed hospitals and funeral homes in many areas, necessitating the deployment of mobile morgues: an unfortunate reality that has garnered national attention in the lay press. 3 Optimal fatality management during the COVID-19 response is therefore an important public health issue requiring timely and transparent data sharing and coordination between public and private entities. To support this need in North Carolina, we developed an interactive, web-based visualization 4, 5 to show the distribution of morgues and their capacity levels in real time. This visualization is primarily displayed within the state's Emergency Operations Center. Key stakeholders include governmental public health agencies at the state level, regional health-care coalitions, local funeral homes, morgues, crematories, medical examiners, and hospitals. In addition, 10 major hospital systems span North Carolina, together providing much of the infrastructure for acute and critical care where many COVID-19 deaths occur. Our team aggregated facility-level morgue data that were continuously reported to the state, which we tabulated at the county and region levels and presented through an interactive, web-based application. The purpose of this map is to visualize fatality management data to facilitate real-time surveillance and early identification of critical areas where extra support might be needed. The interactive, web-based dashboard leverages 3 sources of data: (1) a public resource containing statistics about confirmed COVID-19 cases; (2) a private, access-restricted repository of morgue facility capacity and utilization statistics that was developed for this project and maintained by the North Carolina Department of Health and Human Services (NC DHHS); and (3) static metadata describing state counties and regions, including geographic boundaries for the rendering of an interactive map. These data sources are updated regularly and made web accessible as described below. When a user visits the visualization dashboard via a standard web browser, JavaScript code embedded within the dashboard runs in the browser. This code dynamically retrieves the latest versions of all data sources, processes the data as needed to derive relevant statistics and summaries, and visualizes the results. This approach allows for asynchronous data updates during data gathering, while also ensuring that the dashboard contains the most up-to-date data available at any time. Interactive capabilities allow users to navigate the data sets to view overviews and details on demand for various statistics of interest, with customized views for different user roles. Moreover, a state-dependent bookmarking feature allows users to bookmark specific views within the dashboard, which can be useful for link sharing and collaboration. The overall architecture and data flow are illustrated in Figure 1 . Data for the number of confirmed COVID-19 cases in North Carolina is obtained from a public database maintained by USAFacts.org. 6 The resource includes county-level statistics and is made available as a URL-accessible comma-separated value (CSV) data file. In contrast to the COVID-19 case data, data regarding the capacity and status of mortuary facilities in North Carolina are neither publicly available nor readily visible. To address this, NC DHHS developed a survey instrument to capture key facility information, including the (1) facility name and address (including county); (2) total capacity; and (3) current utilization, among other details. The online survey was distributed via ReadyOp, 7 a vendor-provided platform to support emergency management and disaster response that is used by North Carolina and many other states. The survey is distributed to facilities across the state, including local medical examiners, funeral homes, and morgues, through official communications from the state health department. The survey is also distributed to hospitals via the state's Office of Emergency Management. Recipients are asked to respond a minimum of 2 times per week (on Mondays and Thursdays), although more frequent responses are encouraged. To allow for the fastest time to implementation, the project team opted for a process where survey responses and time stamps are manually copied by a DHHS employee from ReadyOp to an online Google Sheet, which is accessible to the web application in real time via Google's http-based CSV interface. 8 Metadata resources (county names, map geometry, and region definitions) were gathered during the system design process and stored as statically web-accessible data files. A web application was developed to process and visualize the gathered data. It is implemented in JavaScript (along with HTML/CSS) using Bootstrap 9 and D3js. 10, 11 All code runs entirely within a user's web browser (we use the Apache http server), and it requires no server-side execution. Whenever a user visits the visualization dashboard, their browser first retrieves all required HTML, CSS, Java-Script, and image files from the web server. The browser then begins executing the JavaScript code, which performs 2 key steps: (1) data processing, and (2) data visualization. The data processing stage begins with asynchronous GET requests using JavaScript to retrieve all required data resources each time the website is reloaded. JavaScript promises are used to synchronize data processing after the data files have been fully loaded into the browser. A multi-level aggregation is performed on the mortuary facility data to group statistics at the county, regional, and statewide levels. The COVID-19 county-level case counts are aggregated similarly. These aggregate data structures are then linked across each level of detail (county, region, and state) to produce a single, unified data structure. The visualization dashboard includes 2 mode selectors ( Figure 2D ) and 3 main display areas (Figure 2A , B, and C) as seen in Figure 2 . The mode selectors allow users to select both (1) the metric to be displayed in the visualization, and (2) the desired level of aggregation. Metrics include total capacity, occupied capacity, available capacity, occupancy rate, and confirmed case count. The aggregation level can be either county or region. Users can change either selector at any time to trigger an immediate update of the visualization. The first display area ( Figure 2A ) contains an alphabetized list of counties showing statewide and county-level statistics for the selected metric. Micro-stacked bar charts next to each county provide an overview of the occupied and available capacity (which together sum to total capacity), allowing for quick scans for county-level hotspots. Counties with missing data are represented using a distinct, short, gray bar. The second display area is a choropleth map of North Carolina. When county-level aggregation is selected, each county is displayed independently. When region-level aggregation is selected, counties are grouped by region and each region has a uniform color (see Figure 3) . The map color-codes individual counties or regions based on the metric chosen, where each metric is associated with a distinct color. Missing data are indicated with special color coding. This is shown in Figure 2 , where we distinguish between counties with 0 capacity (and therefore an undefined occupancy rate; shown with a dotted pattern) versus counties with missing data (shown in gray). Detailed tooltips showing county-and region-level statistics are displayed on mouseover. The third display area ( Figure 2C ) is a table containing facilitylevel data. It includes detailed availability, occupancy, and total capacity data; each facility's exact location; and a time stamp To support bookmarking for specific views in the visualization, the current view mode is automatically updated in the URL's hashtag fragment identifier whenever a user changes 1 of the 2 mode selectors. Users can therefore copy the URL after identifying a view of interest within the dashboard to bookmark for later or to share with collaborators. Finally, a hidden administrative feature allows data managers access to 2 additional modes that show the average and maximum times since the last survey data update for each facility. This administrative feature can help data managers identify counties where the data is most out of date and in need of a manual data collection effort. An evaluation of the visualization was conducted after initial deployment, based on real-world usage patterns and qualitative stakeholder feedback. 12 The chief users of the visualization comprised a small team of public health and emergency management officials who bookmarked the site link and accessed the dashboard daily. Early in the COVID-19 response, this occurred primarily through a central monitor at the state's Emergency Operations Center, but access patterns shifted to remote laptops as the Fatality Management team transitioned to working from home. There was variation in the frequency and consistency of survey responses among hospitals and mortuary facilities: some submitted survey data almost daily, whereas others reported weekly or less frequently. After several weeks, reporting patterns reached a "steady state" at which there were regular or semi-regular updates from 85/100 counties. Facility data remained missing for 15/100 counties; geographically, these were evenly distributed across the state. Qualitative feedback was gathered via an interview with the project's key liaison in the NC DHHS Public Health Preparedness and Response branch and through informal observations of stakeholder usage by 1 author (CC), who worked directly with users while embedded within DHHS for 4 weeks (2 remotely as the pandemic worsened). (Due to time constraints during the pandemic response and other COVID-related priorities, only 1 stakeholder responded to our invitation to provide structured feedback.) Web log data captured by Google Analytics were also reviewed for quantitative insights. Findings from this limited evaluation are summarized in Table 1 , which uses the Centers for Disease Control and Prevention's evaluation framework for public health surveillance systems 13 to organize feedback according to key system attributes. In general, the system performed favorably across the major dimensions. Through formal feedback and anecdotal reports, users consistently reported high levels of satisfaction with the system, stating they continued to access the application regularly and that it was an essential workflow tool. As of this writing, 405 unique sessions, originating in 24 cities, have accessed the system, with many sessions resulting multiple page views. Traffic was mostly desktop users (91%), though mobile and tablet sessions were also recorded. Access rates were highest in the early phases of the pandemic (average 154.5 views per week), before falling to a lower, steady state as the state began a phased reopening (average 11 views per week). This web application integrates multiple data streams into a visualization to support fatality management during the COVID-19 response in North Carolina. The project represents a collaboration between local academic researchers (including trainees), governmental public health practitioners, and emergency management organizations. The 3-person multi-disciplinary team that self-assembled to develop the application was able to identify user requirements, consolidate data sources, develop source code, and launch an interactive prototype in under 5 days. This rapid start-up was pivotal to the success of the project and required clear communication and teamwork across multiple organizational boundaries. This work emphasizes the importance of iterative design and development that can respond to quickly evolving user needs. For example, in its initial version, the visualization did not indicate when facility data were last updated, but this information was added later using time stamps and visual emphasis strategies based on stakeholder feedback. Other system capabilities were similarly iterated and improved upon. Our experience and feedback from stakeholders yielded several ideas for future enhancements to the system. Automating the data export from the survey platform to Google Sheets would improve upon the current system of manual data management. In addition, URLbased access to the visualization dashboard was limited to a handful of stakeholders, while others received static screenshots and did not benefit from the system's interactivity. Future design improvements could account for the value of static information by relying less on mouse-based interaction and displaying more static information. Furthermore, the system has the potential to integrate epidemiological data to provide more context in identifying critical areas for fatality management, as well as a "look back" feature that allows users to compare current county or region data to historical data. Through close collaboration with stakeholders and a rapid iterative development process, a statewide system was developed to gather, process, and visualize morgue facility data at facility, county, and regional levels. This system enables real-time surveillance and has been well received among public health and emergency management officials involved in the COVID-19 disaster response. This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. SK, CC, and DG all contributed to the writing and editing of the paper. CC was primarily responsible for developing the data gathering aspects of the system. SK and DG were primarily responsible for developing the overall system, including the data flow, web application infrastructure, and user-facing components (including the visualizations). An interactive web-based dashboard to track COVID-19 in real time Preliminary estimate of excess mortality during the COVID-19 outbreak New York City deploys 45 mobile morgues as virus strains funeral homes. The New York Times Improving healthcare with interactive visualization A survey of visual analytics for public health Coronavirus live map; US coronavirus cases by county Google Sheets: free online spreadsheets for personal use Responsive Web Development D3 data-driven documents D3.js; data-driven documents Evaluating visual analytics for health informatics applications: a systematic review from the American Medical Informatics Association Visual Analytics Working Group Task Force on Evaluation Updated guidelines for evaluating public health surveillance systems: recommendations from the Guidelines Working Group The authors thank Christine Mullarkey-Campbell from the North Carolina Department of Health and Human Services Public Health Preparedness and Response Branch for her close collaboration and leadership in the work reported in the article. None declared.