key: cord-0768348-ehflxb0t
authors: Boggs, Krislyn M.; Sullivan, Ashley F.; Espinola, Janice A.; Gao, Jingya; Camargo, Carlos A.
title: Evaluation of the American Hospital Association Annual Survey for health services research in emergency medicine
date: 2022-04-01
journal: J Am Coll Emerg Physicians Open
DOI: 10.1002/emp2.12704
sha: 8a35c1ba1a200bf0f17569a05177c512616e25e5
doc_id: 768348
cord_uid: ehflxb0t

OBJECTIVES: Emergency department (ED) data are often used to address questions about access to and quality of emergency care. Our objective was to compare one of the most commonly used data sources for national ED information, the American Hospital Association (AHA) Annual Survey, with a criterion database: the National Emergency Department Inventory (NEDI)–USA data set. METHODS: We compared the 2015 and 2016 AHA surveys to the following 3 criterion standards: (1) the 2015 and 2016 NEDI‐USA databases, which have information about all US EDs, including merged data from (2) Council of Teaching Hospitals (COTH) and (3) the Critical Access Hospital (CAH) program. We present descriptive results about the number of EDs in each data set; total and median visit volumes; locations in rural areas; and COTH, CAH, and freestanding ED (FSED) status. RESULTS: The AHA survey identified 3893 US EDs in 2015. These EDs had a total annual visit volume of 129,197,493 visits, with a median of 22,772 visits (interquartile range, 8311–47,938). Compared with the NEDI‐USA, the AHA included 1433 fewer EDs (−27%; 95% confidence interval [CI], −28% to −26%) and 23,615,163 (−15%) fewer visits. Specifically, AHA was missing 245 (−22%; 95% CI, −24% to −19%) of those located in rural areas, 268 (−20%; 95% CI, −22% to −18%) in a CAH, and 240 (−47%; 95% CI, −51% to −42%) FSEDs. We saw similar results using 2016 data. CONCLUSIONS: Although several aggregated results were similar between the compared data sources, the AHA data set excluded many US EDs, including many rural EDs and FSEDs. Consequently, the AHA underreported total ED visits by 15%. We encourage data users to be cautious when interpreting results from any 1 ED data source, including the AHA.

Emergency departments (EDs) play a critical role in the US healthcare system. As EDs open and close, tracking EDs on the national level is challenging. Administrative ED data are often used by policy researchers to address questions about access to care, quality of care, and ED staffing. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] Thus, reliance on inaccurate data sources could hinder or even misinform such research as well as subsequent efforts to improve access to and quality of care.

One of the most commonly used data sources for national hospital and ED information is the American Hospital Association (AHA)'s Annual Survey database. 13 This database includes many details about hospital and ED characteristics and can easily be linked with other databases (eg, the Centers for Medicare and Medicaid Services [CMS] data 14 ) to investigate associations between facility characteristics and patient outcomes. Although the AHA data set is uniquely able to facilitate important health services research about emergency care, it has several limitations. Notably, the AHA often groups EDs within the same health system under a single identification (ID) number, and it is difficult to attribute facility characteristics to individual EDs. 15 

Our objective was to compare the AHA database to a criterion standard and determine the magnitude and direction of any identified errors. Specifically, we aimed to compare it with the National Emergency Department Inventory (NEDI)-USA database, 16 which was selected as a criterion database because of its comprehensive inclusion of EDs and annual survey response rate of >80% over many years.

The AHA database is commercially available on an annual basis and contains hundreds of data elements, including ED-related variables.

The AHA collects data directly from >6000 US hospitals via the voluntary AHA Annual Survey (Chicago, IL), with a response rate of >75% each year. 17 For non-reporting hospitals and for incomplete responses, data are estimated based on the missing hospital's most recent information using various statistical methodologies. 17 We compared commonly used ED data from the 2015 AHA database to (1) the 2015 NEDI-USA database, 16 which includes information taken directly from (2) the Council of Teaching Hospitals (COTH) 18 and (3) the Critical Access Hospital (CAH) program. 19 To examine the reliability of the findings, we repeated the comparison using the 2016 AHA and 2016 NEDI-

Robust public health databases support high-level health ser- 

The NEDI-USA contains information on all non-federal, non-specialty, 19 ; and total and satellite FSED status. 24 The analysis used descriptive statistics and reported frequencies with proportions and medians presented with interquartile ranges (IQRs). We compared distributions of ED characteristics presented in each data set using chisquare and Kruskal-Wallis tests as appropriate. Percentage changes relative to the criterion database were calculated with 95% confidence intervals (CIs) to illustrate variation in differences between the AHA and NEDI-USA.

The AHA survey identified 3893 US EDs compared with 5326 in the NEDI-USA in 2015 (Table 1) We again found differences when comparing FSED characteristics in the AHA versus characteristics among satellite FSEDs only in the NEDI-USA (all P < 0.05; Table 1 ).

When using COTH and CAH data as the criterion standards, we identified differences in both the AHA and NEDI-USA data. Specifically, there were 433 total hospitals/EDs listed in the COTH data set, of which the AHA and NEDI-USA both identified 262 (61%). There were 1333 EDs listed in the CAH data set. Of these, the AHA identified 1065 (80%), whereas the NEDI-USA identified 1333 (100%; Figure 1 ).

When repeating this analysis using data for the year 2016, we found similar patterns. Briefly, the 2016 AHA identified 3789 EDs ( Figure 1 ).

The 2015 consolidated data set included 4658 observations. There were 149,969,604 total visits to these EDs, ranging from 1 to 476,187, and a median of 20,388 (IQR, 7000-45,255; Table 3 ). There were 1103 (24%) located in rural areas, 1324 (28%) in a CAH, and 214 (5%) identi-fied as affiliated with satellite FSEDs. Using the consolidated data set, 241 (56%; Figure 1 ) of the COTH hospitals/EDs were identified and 1324 (99%) of the CAH hospitals/EDs were identified.

We found that EDs that were individually listed in the full NEDI-USA data set, but grouped in the AHA, were more likely to be FSEDs (43%; an FSED (53%) and were less likely to be rural (2%; all P < 0.001). Those excluded completely from the AHA were also more likely to be an FSED (90%) and less likely to be rural (4%; all P < 0.001). There were 143 EDs in the AHA that were not represented by the consolidated data set.

The current analysis has several limitations. Information in both the AHA and NEDI-USA is self-reported by individuals at the respective healthcare facilities. Also, although we identify differences between the AHA and NEDI-USA data sets, some of those differences may be attributed to how these databases define an ED (eg, exclusion of federal EDs in the NEDI-USA but not in the AHA). However, even with these differences, it is clear that the AHA does not include a large proportion of general EDs in its annual data set.

When reviewing 2 consecutive years of national data, we found that the AHA was similar to the other criterion databases but did have some 

None of the authors report any potential conflicts of interest.

All authors conceived and designed the study. Krislyn M. Boggs and 

Patient insurance status is associated with care received after transfer among pediatric patients in the emergency department

Factors associated with potentially avoidable interhospital transfers in emergency general surgery: a call for quality improvement efforts

Geography, not health system affiliations, determines patients' revisits to the emergency department

National differences in regional emergency department boarding times: are US emergency departments prepared for a public health emergency?

Ambulance diversions following public hospital emergency department closures

Accountable care hospitals and preventable emergency department visits for rural dementia patients

Disparities in access to emergency general surgery care in the United States

The crisis in United States hospital emergency services

Hospital and community characteristics associated with pediatric direct admission to hospital

Associations of emergency department length of stay with publicly reported quality-of-care measures

Hospital markets in the United States

Trends and variation in the utilization and diagnostic yield of chest imaging for medicare patients with suspected pulmonary embolism in the emergency department

American Hospital Association. Annual Survey Dataset Documentation Manual. American Hospital Association

Systems/Research-Statistics-Data-and-Systems

Consolidating emergency department-specific data to enable linkage with large administrative datasets

A profile of US emergency departments in 2001

AHA data & insights

Association of American Medical Colleges. Council of Teaching Hospitals and Health Systems (COTH)

Flex Monitoring Team. Critical access hospital locations

Types of hospitals in the United States

Emergency department visit counts (in thousands) in the United States

Availability of pediatric emergency care coordinators in U.S. emergency departments

What is a freestanding emergency department? Definitions differ across major United States data sources

State regulation Of freestanding emergency departments varies widely, affecting location, growth, and services provided

Distance from freestanding emergency departments to nearby emergency care

Core-based statistical areas

Number of emergency department visits in the U.S

Factors associated with closures of emergency departments in the United States

The year in numbers. https: //mnareview.kaufmanhall.com/the-year-in-numbers?

National trends in emergency department closures, mergers, and utilization

Agency for Healthcare Research and Quality. Healthcare Cost and Utilization Project

The Emergency Department Benchmarking Alliance