key: cord-0693067-ku7akv8f authors: Heyne, T. F.; Geisler, B. P.; Negishi, K.; Choi, D. S.; Al Saud, A. A.; Marinacci, L. X.; Smithedajkul, P. Y.; Devaraj, L. R.; Little, B. P.; Mendoza, D. P.; Flores, E. J.; Petranovic, M.; Toal, S. P.; Shokoohi, H.; Liteplo, A. S. title: A Simplified Point-of-Care Lung Ultrasound Protocol to Detect Coronavirus Disease 2019 in Inpatients: A Prospective Observational Study date: 2021-04-20 journal: nan DOI: 10.1101/2021.04.19.21254974 sha: 142e9496045f3b6554eefec20fc2802bb4b00eb4 doc_id: 693067 cord_uid: ku7akv8f Objectives: To assess the diagnostic performance of lung point-of-care ultrasound (POCUS) compared to either a positive nucleic acid test (NAT) or a COVID-19-typical pattern on computed tomography (CT) and to evaluate opportunities to simplify a POCUS algorithm. Methods: Hospital-admitted adult inpatients with (1) either confirmed or suspected COVID-19 and (2) a completed or ordered CT within the preceding 24 hours were recruited. Twelve lung zones were scanned with a handheld POCUS machine. POCUS, CT, and X-ray (CXR) images were reviewed independently by blinded experts. A simplified POCUS algorithm was developed via machine learning. Results: Of 79 enrolled subjects, 26.6% had a positive NAT and 31.6% had a CT typical for COVID-19. The receiver operator curve (ROC) for a 12-zone POCUS protocol had an area under the curve (AUC) of 0.787 for positive NAT and 0.820 for typical CT. A simplified four-zone protocol had an AUC of 0.862 for typical CT and 0.862 for positive NAT. CT had an AUC of 0.815 for positive NAT; CXR had AUCs of 0.793 for positive NAT and 0.733 for typical CT. Performance of the four-zone protocol was superior to CXR for positive NAT (p=0.0471). Using a two-point cutoff system, the four-zone POCUS protocol had a sensitivity of 0.920 and 0.904 compared to CT and NAT, respectively, at the lower cutoff; it had a specificity of 0.926 and 0.948 at the higher cutoff, respectively. Conclusion: POCUS outperformed CXR to predict positive NAT. POCUS could potentially replace other chest imaging for persons under investigation for COVID-19. Rapid and accurate diagnosis of Coronavirus Disease 2019 (COVID-19) can be challenging. The most widely used diagnostic test to detect Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) infection is a nucleic acid amplification test (NAT) via reversetranscription polymerase chain reaction. NAT via nasopharyngeal swab has a very high specificity but a sensitivity of only 70-85%. 1 Thus, a significant number of patients may be infected even with a negative NAT, especially later in the course of infection. 2, 3 Computed tomography (CT) of the chest, which can detect typical patterns of lung findings for COVID-19, has a higher sensitivity (86-97%) but lower specificity (25-81%) compared to NAT. [4] [5] [6] [7] Several validated reporting systems are used by radiologists in assessing likelihood of COVID-19 pneumonia on chest CT. A commonly used consensus guideline from the Radiological Society of North America (RSNA) classifies CT morphologies as "typical" (highest suspicion), "indeterminate", "atypical", or "negative" for COVID-19 pneumonia. 8 However, CT scan has drawbacks; it requires significant healthcare resources and time, exposes the patient to ionizing radiation, poses infection control risks, and may be unsafe for unstable, hypoxemic patients. Chest X-ray (CXR) reduces some of these drawbacks but has a low sensitivity for detecting COVID-19, particularly early in the disease course (55-83%). 9 Thus, hospitals often rely on more than one category of diagnostic test, as well as patient history and risk factors, to attempt to rule out COVID-19 infection. Imaging, therefore, can be a helpful adjunct. 6, 10 In our hospital system, either a CXR or a CT is required for all admitted patients with a clinical suspicion for COVID-19 (based on history and demographics) and a negative swab. 11 If CXR findings are concerning for COVID-19, a CT scan and second NAT are often recommended. If the CT scan does not show a "typical" pattern for COVID-19, and the second NAT is negative, All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted April 20, 2021. ; https://doi.org/10.1101/2021.04.19.21254974 doi: medRxiv preprint then COVID-19 is usually ruled out. While the case remains uncertain (while patients are still undergoing diagnostic evaluation for suspected , patients are flagged as Persons Under Investigation (PUIs) and placed under the same enhanced isolation precautions as patients with COVID-19. Thus, both CXR and CT scan can help in ruling out and ruling in COVID-19, but they have drawbacks. Point-of-care ultrasound (POCUS) may be an alternative diagnostic modality. POCUS is inexpensive, rapid, safe (no radiation), widely available, and does not require travel to and possible infectious exposure of a radiology suite. COVID-19 tends to affect the lung periphery, which is visualized well by POCUS. Numerous studies have described characteristic features for COVID-19 on lung ultrasound, including confluent B-lines (also known as "waterfall" B-line, "light beam artifact", or "white" lung). [12] [13] [14] [15] Multiple stratifying scoring systems to assess disease severity have been proposed. For example, an expert opinion from Soldati et al. proposed scanning 14 lung zones and assigning 0-3 points per zone; the total number of points would correlate with the severity of COVID-19 pneumonia. 16 The same group analyzed protocols with fewer lung zones (e.g., 4-12 zones), given that an abbreviated protocol could help reduce exposure time with infectious patients; however, the study concluded that a 12-zone protocol was optimal to assess COVID-19 severity. 17 To date, relatively few prospective research studies have evaluated the accuracy of POCUS to assist in the diagnosis of COVID-19. [18] [19] [20] While valuable, these studies have limitations which hinder their generalizability, including using a single operator, excluding patients with heart failure, or relying on the scanner's overall subjective gestalt for COVID-19 pneumonia. Our aim was threefold. First, we assessed the diagnostic accuracy of lung POCUS to detect either (1) a high-suspicion ("typical") pattern for COVID-19 on CT scan or (2) a positive COVID-19 NAT. We hypothesized that lung POCUS has superior accuracy compared to CXR for both All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. outcomes. Second, we assessed whether diagnostic accuracy would be affected by a simplified scanning protocol, which required fewer lung zones and less subtle findings than the protocol proposed by Soldati et al. 16 Third, we assessed whether diagnostic accuracy would differ in the subpopulation of patients with a diagnosis of congestive heart failure (CHF), given possible overlap in findings between COVID-19 and CHF, and therefore possible reduced accuracy in this population. This prospective cohort study took place in a 1,000-bed quaternary care hospital, from April to July 2020. We included a convenience sample of adult patients who were admitted to either the medical ward or intensive care unit (ICU). A query of the electronic medical record system identified patients who had either confirmed or suspected COVID-19 infection (i.e., "PUI" status and awaiting further testing) and who had completed or were planned to complete a chest CT within 24 hours of the POCUS scan. Patients with interstitial lung disease (ILD) were excluded, given expected abnormal lung POCUS scans. The protocol was approved in writing by the local Institutional Review Board. After obtaining assent from the treating team and verbal consent from the patient or proxy, a research physician scanned 12 lung zones (Figure 1 ), similar to prior protocols. 20 The patient wore a surgical mask, and the scanning physician wore hospital-recommended personal protective equipment (N-95 respirator, eye protection, gown, and gloves). The probe was held longitudinally All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted April 20, 2021. ; https://doi.org/10.1101/2021.04.19.21254974 doi: medRxiv preprint and perpendicular to the ribs to obtain the "bat sign" view. 21 Scanning physicians attempted to capture at least two intercostal spaces for each zone (and at least three intercostal spaces for each of the four posterior zones). For posterior lung zones, patients either sat upright or lay in lateral decubitus position. Scanning was completed using a handheld Butterfly iQ ultrasound machine (Butterfly Network, Inc., Guilford, CT) connected to an iPhone (Apple Inc., Cupertino, CA). The lung preset was used for all zones; a second clip was obtained using the abdominal preset for zones R4 and L4 (to investigate for pleural effusion). Single-use gel packets were used for ultrasound gel, to avoid cross-contamination. After scanning, machines were sanitized with hospital-approved Similarly, CXR and CT scans were reviewed by two board-certified radiologists specialized in thoracic imaging and blinded to clinical and POCUS information. CXRs and CTs were interpreted independently. Radiologists gave each CT and CXR a COVID-19 suspicion grade, following RSNA consensus categories: (1) typical (highest suspicion), (2) indeterminate, (3) atypical/low, or (4) negative. 22 A third radiologist provided an interpretation in the case of discordant All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted April 20, 2021. ; https://doi.org/10.1101/2021.04.19.21254974 doi: medRxiv preprint interpretations (as a tiebreaker). For both CXR and CT, the mode of the three interpretations was considered the consensus. For cases in which the third reader assigned a grade different from the primary two readers, the grade corresponding to the value closest to the median of the three observer grades was used as the consensus. For all imaging modalities, we calculated a Cohen's kappa to compare the interrater reliability between the first two readers (not including the "tiebreaker"). An ordinal scale was used for CT, CXR, and POCUS interpretations. For POCUS scans, we recorded the duration of time spent scanning (from the start of the first clip to the final clip). We assessed the accuracy of the 12-zone Soldati POCUS protocol to detect either (1) typical CT pattern or (2) positive NAT (by the time of discharge). The Soldati protocol assigned a score 0-3 for each of the 12 lung zones (thus, maximum score for a patient was 36). 16 Receiver operating characteristic (ROC) curves were created to compare the performance of Soldati score and CXR for our first outcome: a typical pattern for COVID-19 on CT. Additional ROC curves compared Soldati score, CXR, and CT against our second outcome: positive NAT. Areas under the curve (AUCs) were generated with the algorithm suggested by DeLong, DeLong, and Clark- All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. We next evaluated simplified modifications of the Soldati protocol. Machine learning analyses employing random forests and lasso regression were used to identify the lung zones and findings most correlative to our two outcomes (typical CT pattern and positive NAT). Using this information, we generated modified (simplified) versions of the Soldati protocol and analyzed the accuracy of each modification. Specifically, we sequentially reduced the number of lung zones that were incorporated into the POCUS score (starting at 12 and ending at 4 zones). In addition, we modified the point values that the Soldati score had assigned to different lung findings. For example, we varied whether 0-4 points would be scored for each lung zone that showed 3+ Blines. In total, over 150 scoring algorithms (modifications of the original Soldati score) were generated. ROC curves were generated for each of these modified scoring systems for the outcomes of (1) typical CT pattern and (2) positive NAT. The AUCs for each of these modified scores were compared to both the Soldati Score and the other imaging modalities (CXR for typical CT pattern, and CXR and CT for positive NAT). Finally, we performed the same analysis (ROC curves for our two outcomes) in the subpopulation of patients with a history of CHF. To calculate individual test characteristics (sensitivity, specificity, positive and negative predictive value) for our simplified POCUS score, we created 2 score cutoffs. These cutoffs allowed us to create low, intermediate, and high COVID-suspicion categories from the ordinal POCUS score (e.g., 0-12 points); these categories would mirror the RSNA CT and CXR categories. Data analysis was performed in Python 3.8.3 (Python Software Foundation, Beaverton, OR) 24 and STATA IC 14.2 (Stata Corp LLC, College Station, TX). All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Ninety patients were initially screened, and 79 were scanned and included in the final analysis. The remaining eleven were excluded based on lack of consent, presence of exclusion criteria (namely ILD), or CT scan not completed (see Appendix). Patient characteristics at the time of scan are given in Table 1 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. ROC curves were generated, and AUCs were calculated, to assess the accuracy of the 12zone Soldati protocol for detecting (1) typical CT pattern and (2) All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. To mirror the RNSA categories used for CT and CXR, we created 2 cutoffs for our MGH 4-Zone Protocol, to divide patients into low, intermediate, and high suspicion for COVID (see Appendix). We set low suspicion at <4 points; intermediate at ≥ 4 and < 8 points, and high suspicion at ≥8 points. Operating characteristics are summarized in Table 2 , which also includes the sub-analysis of patients with a prior medical history of CHF (n=17). A 12-zone lung POCUS protocol following Soldati et al. had reasonable diagnostic accuracy to detect either a typical CT pattern or a positive NAT for COVID-19. This accuracy did not decrease with a simplified four-zone POCUS protocol, which yielded a specificity of more than 90% at the high cut-off point for detecting either typical CT pattern or positive NAT. At the intermediate cut-off point, the sensitivity and negative predictive value for typical CT pattern and for positive NAT were both greater than 90%. When predicting the COVID-19 NAT result, POCUS performed better than chest X-ray. Otherwise, there was no statistically significant difference in the diagnostic accuracy among the 12-zone Soldati protocol, our four-zone protocol, CT, and CXR. We suspect that our simplified four-zone protocol performed well compared to the 12-zone Soldati protocol, because the latter was intended to assess the severity of COVID-19 (e.g., assigning higher points for consolidation), not to diagnose COVID-19. Extrapolating from our results, patients with a high four-zone POCUS score could be considered high risk for COVID-19, and further imaging with a CT could likely be avoided. Indeed, all patients above the high cut-off had either "typical" (14/18) or "indeterminate" (4/18) CT findings. Similarly, one could All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted April 20, 2021. ; https://doi.org/10.1101/2021.04.19.21254974 doi: medRxiv preprint extrapolate that patients below the low cut-off could be considered to have a low risk for COVID-19 and might be able to forgo a CT scan as well. The protocol performed equally well in CHF patients. In addition, our patients on average were overweight (mean BMI 27.0), suggesting that the protocol would perform well in the overall more obese U.S. population. The study results add to the literature that POCUS can assist in the diagnosis or risk stratification of PUIs for COVID-19. [18] [19] [20] Namely, lung POCUS may be a helpful adjunct to history and risk factors both to "rule in" or "rule out" COVID-19 infection. POCUS may be particularly helpful in resource-limited settings, where access to CT or NAT may be limited or results may be delayed. Benefits of lung POCUS include safety for the patient: no need for ionizing radiation (particularly important for younger or pregnant patients) and no need for transportation to a radiology suite (may be unsafe in sicker patients and a potential infectious control risk). One strength of our study is that we used an inexpensive, handheld, easily disinfected ultrasound machine. Such technology is increasingly available worldwide. Also, lung POCUS is rapid. Our 12-zone POCUS mean scanning time was 10 minutes; a protocol with only four zones would have been even faster. Indeed, strengths of our protocol include simplicity and potential applicability worldwide. The simplified protocol requires scanning only four posterior lung zones using a simple longitudinal technique, meaning scanning less time and standing behind the patient's facereducing exposure risk to the scanner. The interpretation is also simpler: it is easier to assess for B-lines than for subtle consolidations or pleural line breaks or irregularities (e.g., as recommended in prior studies). 16, 20 The protocol fits with findings from prior studies, as confluent B-lines (or "light beam artifacts") have been described as more specific for COVID-19, [12] [13] [14] [15] and as COVID-All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted April 20, 2021. ; https://doi.org/10.1101/2021.04.19.21254974 doi: medRxiv preprint 19 tends to affect the posterior lungs preferentially. 13, 17, 19, 20 The inter-rater reliability for readers for the simplified POCUS score was overall good, notably higher than CXR and similar to CT. Our study was subject to several limitations. First, this was a single-center study, using a convenience sample of inpatients based on scanning physician availability. However, there have been relatively few prospective studies to date, particularly multi-center studies. Validation studies performed at other centers are warranted and welcome. Second, our study examined only inpatients (mostly from the ward, with a small number of ICU patients); it is unknown whether this scanning protocol would yield similar results for outpatients or ED patients. Third, the relatively high incidence of COVID-19 (26.6%) could lead to a spectrum effect. However, this incidence is lower than some other studies, and our sample did include many patients with low-suspicion CXR and CT results. Fourth, we note that, although many of our patients were quite sick, none of them were proned at the time of exam. It is unknown whether our protocol would have the same accuracy in proned patients. However, it is rare to prone a ward patient without a confirmed diagnosis of COVID-19. Fifth, although scanning physicians were blinded to CT results, they were not blinded to NAT status (positive COVID status is displayed prominently in our hospital electronic record); it is possible that sampling bias of the lung zones was introduced. However, COVID-positive and PUI patients had similar POCUS scan times. Sixth, none of the patients with an initial negative NAT subsequently had a positive NAT. However, this was an inpatient study, and initial NAT is always performed in the Emergency Department in our institution. Incidence of positive NAT after initial negative NAT is very low. 2 Thus, it would be extremely challenging to capture an appreciable number of NAT-positive patients if we had limited our study to only those patients with an initial negative NAT. Finally, we acknowledge that many hospitals do not have inpatient providers who are sufficiently trained in lung POCUS. However, interest in POCUS among All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted April 20, 2021. ; https://doi.org/10.1101/2021.04.19.21254974 doi: medRxiv preprint inpatient clinicians is growing, and applications like these could bring impetus to further clinician training. In addition, most Emergency Department providers in the U.S. are trained in lung POCUS. In conclusion, lung POCUS could be a valuable tool in assessing patient risk of COVID-19. Both a 12-zone protocol and a simplified four-zone protocol performed well to detect patients with either a high-suspicion CT pattern for COVID-19 or a positive nasopharyngeal NAT. Indeed, the simplified four-zone lung POCUS out-performed CXR for identifying patients with positive NAT. In patients under investigation for COVID-19, a lung POCUS exam could potentially replace other chest imaging as a helpful adjunct for clinicians to decide whether a COVID-19 diagnosis should be further entertained or if patients can be taken off precautions. Further research in the form of confirmatory studies at other centers is warranted. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. B. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted April 20, 2021. ; https://doi.org/10.1101/2021.04.19.21254974 doi: medRxiv preprint Abbreviations: CT, computed tomography; CXR, chest X-ray; AUC, area under the curve; CI, confidence interval; NAT, nucleic acid test; MGH, Massachusetts General Hospital All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Low suspicion for COVID-19 ≥ 4 and < 8 Intermediate suspicion for COVID-19 ≥ 8 High suspicion for COVID-19 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted April 20, 2021. ; https://doi.org/10.1101/2021.04.19.21254974 doi: medRxiv preprint Evaluating the accuracy of different respiratory specimens in the laboratory diagnosis and monitoring the viral shedding of 2019-nCoV infections Clinical, laboratory, and radiologic characteristics of patients with initial false-negative SARS-CoV-2 nucleic acid amplification test results. Open Forum Infectious Diseases COVID-19 diagnosis in case of two negative nasopharyngeal swabs: association between chest CT and bronchoalveolar lavage results Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases Chest CT in the Emergency Department for Diagnosis of COVID-19 Pneumonia: Dutch Experience Implementation of the Radiological Society of North Determinants of chest x-ray sensitivity for covid-19: A multi-institutional study in the united states False-Negative Nasopharyngeal Swabs and Positive Bronchoalveolar Lavage: Implications for Chest CT in Diagnosis of COVID-19 Pneumonia Diagnostic Clinical Decision Support: A Pre-Post Implementation Study of CORAL (COvid Risk cALculator) Sonographic signs and patterns of COVID-19 pneumonia A preliminary study on the ultrasonic manifestations of peripulmonary lesions of non-critical novel coronavirus pneumonia (COVID-19) Point-of-care lung ultrasound in patients with COVID-19-a narrative review Chinese Critical Care Ultrasound Study G. Findings of lung ultrasonography of novel corona virus pneumonia during the 2019-2020 epidemic Proposal for International Standardization of the Use of Lung Ultrasound for Patients With COVID-19: A Simple, Quantitative, Reproducible Method On the Impact of Different Lung Ultrasound Imaging Protocols in the Evaluation of Patients Affected by Coronavirus Disease 2019: How Many Acquisitions Are Needed? Lung Ultrasonography for the Diagnosis of SARS-CoV-2 Pneumonia in the Emergency Department No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity We greatly appreciate assistance from Dr. Irene Ma and Dr. Emily Hyle for advice in planning this project and revising the manuscript. We are also grateful to Dr. Nicole Duggan and Dr. Daniel Restrepo for assistance in early stages of this project. Preliminary results from this study were presented at the Massachusetts General Hospital Clinical Research Day, a local research meeting held on October 1, 2020. These authors have no conflicts of interest to declare.All rights reserved. No reuse allowed without permission.(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.The copyright holder for this preprint this version posted April 20, 2021. ;