key: cord-1031272-9prewpof authors: Raiteri, Alberto; Muratori, Luca; Faggiano, Chiara; Alvisi, Margherita; Serio, Ilaria; Piscaglia, Fabio title: Efficacy of a short course of lung ultrasound for primary care physicians in the assessment of COVID-19-positive patients date: 2022-01-08 journal: Fam Pract DOI: 10.1093/fampra/cmab150 sha: b18addf8d4a167893904c6102c9c89919dba0e90 doc_id: 1031272 cord_uid: 9prewpof INTRODUCTION: Lung ultrasound (LUS) has become the first diagnostic imaging approach to assess lung involvement in COVID-19. While LUS proved to be safe, reliable, and accurate, not many primary care physicians (PCP) are capable to employ this instrument in the first evaluation of COVID-19 outpatients. The aim of this study was to determine the effectiveness of a brief training program in LUS for PCP. METHODS: Italian local authorities promoted a training program in LUS for PCP engaged in COVID-19 outpatients’ evaluation. The course took place in a COVID-19 unit and included a hands-on practice on real COVID-19 patients. We conducted a qualitative and quantitative analysis of the results of the training program. RESULTS: A total of 32 PCP completed the training. About 100% of participants reported an increase in competence and confidence in the use of LUS after the training. Self-reported confidence in detecting major COVID-19 LUS abnormalities was high (B-lines 8/10, pleural abnormalities 6.5/10). B-lines were accurately identified with a reliability of 81%, with a sensitivity of 96%, and a negative predictive value of 98%. Trainees were some less accurate in detecting pleural abnormalities (reliability 63%) but with a high specificity (99%). CONCLUSIONS: This study showed that a short training program, but comprising a hands-on practice, is capable to bring even almost novices to achieve a high overall accuracy and reliability in detecting lung involvement in COVID-19. This may result in a significant improvement of the performances of PCP involved in the first evaluation of COVID-19 cases in primary care facilities. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), cause of Coronavirus Disease 19 (COVID-19) pandemic, has been responsible for an unprecedented mortality toll for an infectious disease, making Italy one of the countries with the highest mortality rate (150.59 deaths per 100,000 population). 1 In this context finding strategies to optimize economical and human resources have become essential. Due to the unprecedented pressure on hospital care, it has become imperative to limit hospital admissions of patients without significative pulmonary involvement, in order not to subtract resources that could be devoted to patients with a higher risk of progression, who could take advantage of a timely hospital admission. To this end, it has become crucial to correctly identify and grade the lung involvement of SARS-CoV-2 infection in an outpatients' setting, especially in patients who do not still display signs of respiratory failure. Lung ultrasound (LUS) is not only a safe and repeatable tool for a first lung involvement assessment, but it is also potentially portable at patients' home. 2, 3 LUS has proven to produce a high sensitivity in early detecting lung involvement in SARS-CoV-2 infection, similar to chest computed tomography (CT), universally considered the gold standard, but with a quite lower specificity. 4, 5 However, a LUS pattern highly suspicious for COVID-19 pneumonia demonstrated to be also very specific, being a strong independent predictor of PCR nasopharyngeal swab positivity. An altered LUS demonstrated not only to predict chest CT abnormalities but also to be correlated with oxygen requirements in COVID-19 pneumonia. 6 In more general terms, LUS is highly capable of assessing pulmonary severity in patients with suspected or documented COVID-19 infection. 4, 7 These results are consistent with previous data showing a general high accuracy of LUS in diagnosing pneumonia and interstitial lung syndromes. 8 Kumar et al. demonstrated a high interrater reliability in detecting COVID-19 major LUS abnormalities, and in particular, the presence of B-lines more than 3 per field of view. The interrater agreement in detecting sub-pleural consolidations and pleural thickening resulted lower but substantial. 9 However, LUS was performed by trained medical doctors in an inpatient setting. In another study, the presence of B-lines more than 3 per field of view resulted the only independent LU findings associated with the appropriate referral to hospital for COVID-19suspected cases evaluated at home by an ultrasound expert primary care physician (PCP). 2 Hence, while LUS seems to be a reliable tool for detecting lung involvement in COVID-19 and accordingly to properly allocate patients to the inpatient or outpatient setting, not many PCPs are capable to perform ultrasound and even more LUS. Given these premises, it became vital to improve PCPs' competencies in LUS with appropriate training programs. At the time of the present study probably too many suspected or confirmed COVID-19 patients were referred to the emergency room almost uniquely to perform a medical imaging and arterial blood gas tests. The aim of this study was to evaluate the effectiveness of a brief training program in LUS for PCPs involved in COVID-19 outpatients' evaluation, and hence to evaluate their performances in detecting COVID-19 major LUS abnormalities in real patients. Between November 2020 and May 2021, IRCCS Sant'Orsola-Malpighi University Hospital of Bologna and Local Health Authority (AUSL) of Bologna promoted a training program in LUS for PCPs engaged in COVID-19 outpatients' evaluation. The aim of the program was to improve the use of LUS in the in-person home evaluation of suspected or confirmed COVID-19 cases. The course took place in our Unit of Internal Medicine, converted since several months to a COVID-19 unit. Every participant in the course was a PCP already employed in local Special Units for Outpatient Care (Unità Speciali di Continuità Assistenziele-USCA) for COVID-19 pandemic, but without previous specific knowledge about LUS and the almost no skills in ultrasound in general at all. Tutors were experts in ultrasound, in general, with experience in the lung assessment of COVID-19 patients. The training program was scheduled as follows: -At first, the trainees attended a pre-recorded video course, regarding the basics of ultrasound and specific ultrasound characteristics of COVID-19. The online course required a minimal dedication time of 120 min. A summary of the course content can be found at https://www.youtube.com/watch?v=rEeC9fN-He0&feature=emb_imp_woyt. -As second step, trainees attended a hands-on training on LUS, of the duration of about 90 min, aimed to teach them the ultrasound appearance of normal lungs in healthy subjects. Trainees performed a complete lung ultrasound (LUS) examination on each other's chest under the supervision of a tutor. -As third step, trainees attended a second practical session, of the duration of 3 h, consisting of the examination of a series of COVID-19 hospitalized patients, aimed to teach them major COVID-19 LUS abnormalities. -As conclusive step, each trainee was asked to perform a complete LUS examination on a patient hospitalized for COVID-19 pneumonia, under the supervision of a tutor. Patients to be examined were identified by another physician of the Unit in charge for inpatients. In those patients, LUS was already scheduled by the physician in charge the same day. Trainees were asked to fillin a form mimicking an official reporting of the patient they examined. Trainees and tutors were completely blind about the personal and clinical history of patients, which were instead known to the physician who made the selection. No clinical decisions were taken based on the results of these examinations. The training program as presented above had been independently approved by AUSL of Bologna and was run independently from the current study to satisfy local healthcare needs. The aim of this study was academic, namely to investigate the efficacy of this type of three steps course, and no clinical data were collected. The study was approved by the University of Bologna Bioethical Committee (22 Jun 2021, protocol No. 149962). The form used for reporting LUS was inspired by the European Federation of Societies for Ultrasound in Medicine and Biology (EFSUMB) Lung Ultrasound Protocol for COVID-19. 10 Trainees were asked to discriminate between 3 ultrasound patterns in each of the 12 LUS zones (right and left, upper and lower, anterior, lateral, and posterior areas). -A-pattern = regular A-lines, parallel to a thin, regular pleural line. -B-pattern = three or more bright lines (B-lines) visible in one intercostal space, arising from the pleura, regardless of whether normal or irregular. -C-pattern = fragmented pleura, described as irregular, interrupted, and thickened pleural line, with or without sub-pleural, hypo-echoic areas. Tutors filled in the same reporting form looking at trainees performing LU. We remind that "B"-and "C"-patterns have been described as major COVID-19 ultrasound alteration, and in particular, a B-line pattern has been correlated to an appropriate referral to a hospital. 2 • Lung ultrasound may improve primary care physicians' performances to COVID-19. • Primary care physicians can rapidly learn and practice lung ultrasound. • Lung ultrasound may change outpatients' management out of COVID-19 pandemic. Family Practice, XXXX, Vol. XX, No. XX Trainees were finally asked to fill-in an online questionnaire regarding their self-reported performances. Every question had answers graded either on a 5-options ordinal scale (strongly agree, agree, no opinion, disagree, strongly disagree), or on a digital scale from 1 to 10. Statistical analysis was performed using JASP, JASP Team (2020), JASP (Version 0.14.1) [Computer software], University of Amsterdam, Amsterdam, the Netherlands. Agreement between trainees and tutors in detecting patterns was evaluated with Cohen's kappa (Cohen's K). Trainees' accuracy was evaluated as sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) using tutors' results as gold standard. Every result was expressed as absolute value and 95% confidence interval (95% CI). Questionnaire results were reported as medians and interquartile range (IQR) for ordinal variables or as percentages for categorical variables. Difference between ordinal variables was evaluated with Wilcoxon signed-rank test. Statistical significance was considered achieved for P values less than 0.05. All 32 trainees completed the course and responded to the online survey. Each trainee assessed 12 chest areas in every patient for a total of 384 (32 * 12) LU areas analysed. Interrater reliability, measuring the agreement between trainees and tutors, and trainees' accuracy in performing LUS were evaluated on an area-to-area basis ( Table 1) . The interrater agreement in identifying a normal regular A-lines pattern ("A-pattern") was very high, with a Cohen's K of 0.82 (95% CI 0.76-0.88) ( Figure 1 ). Pupils had both a high PPV of 0.92 and a high NPV of 0.90 in confirming or excluding a normal A-line pattern. The agreement in identifying an abnormal B-lines pattern ("B-pattern") was more than substantial, with a Cohen's K of 0.81 (95% CI 0.75-0.87) (Figure 1) . B-lines were detected by participants with extremely high sensitivity and NPV of 0.96 and 0.98, respectively. Specificity (0.88) and PPV (0.81) lowered, reflecting the quite high rate of false-positive lung areas affected by B-pattern (30/161). Table 1 . A-, B-, and C-pattern contingency tables: number of areas classified as "A-pattern" , "B-pattern" , "C-pattern" (1) or not (0) by tutors and trainees. On the right, Cohen's K and accuracy parameters. The agreement in identifying what we called "C-pattern" was substantial, with a Cohen's K of 0.63 (95% CI 0.53-0.72) ( Figure 1 ). C-pattern was identified by participants with a high specificity (0.99) and PPV (0.94). The sensitivity resulted quite low (0.56), reflecting the not negligible proportion of false-negative lung areas affected by a "C-pattern" (45/322). Trainees were all Medical Doctors working in primary healthcare facilities. Median age was 29 years (IQR 27.5-30.5 years), and median length of professional experience was 1.9 years (IQR 1.0-3.4 years) ( Table 2) . About 100% of participants found the Faculty competent and effective during the course (74% strongly agreed, 26% agreed). About 100% of participants judged the course presented objectively and free of conflicts of interest. Trainees were asked to evaluate their competencies about LUS before and after the training. Median self-evaluated competence was 1/10 (IQR 1-3/10) before the training. After the training, 100% of participants declared that the course had been able to improve their competence (62.5% strongly agreed, 37.5% agreed). Median self-evaluated competence raised to 7/10 (IQR 6-8/10) after the training program ( Figure 2, Table 2) . A Wilcoxon signed-rank test showed that self-evaluated competence after the training was significantly higher compared to pre-training, W = 528, P < 0.001. The median difference between the two groups resulted 4.5 (95% CI 3.5-5.0) in Hodges-Lehmann estimate. The size effect resulted very high (rank-biserial correlation of 1.0). Table 2 . Trainees' age and years of professional experience. Self-evaluated confidence (out of 10) in detecting three main lung ultrasound patterns A, B, and C after the course. Self-reported competence (out of 10) before and after the training. Trainees were then asked to evaluate their confidence about the use of LU and the detection of major COVID-19 LU abnormalities. Median self-evaluated confidence about the use of LU was 7/10 (IQR 6-8/10) after the training program, and more than 70% of participants felt sufficiently confident with the tool (12.5% strongly agreed, 68.8% agreed, 15.6% no opinion, 3.1% disagreed). About 100% of participants reported an improvement in their confidence in using LU (56.3% strongly agreed, 43.7% agreed). After the training program, trainees felt very confident in detecting major COVID-19 LU abnormalities: median self-evaluated confidence in detecting normal A-pattern was 8/10 (IQR 7-9/10), in detecting B-pattern 8/10 (IQR 7-8/10), in detecting "C-pattern" 6.5/10 (IQR 6-7/10) ( Figure 1, Table 2) . A Wilcoxon signed-rank test showed that self-evaluated confidence in reporting major COVID-19 LU abnormalities/LU patterns, was significantly different: A-pattern vs. C-pattern W = 325, P < 0.0001, A-pattern vs. B-pattern W = 78, P < 0.001, B-pattern vs. C-pattern W = 269, P < 0.0001. The A-pattern was detected with the highest confidence, significantly superior to the B-pattern (+1/10) and to the C-pattern (+2/10) at Hodges-Lehmann estimate. The median difference between the confidence in detecting the B-pattern vs. the C-pattern was 1.5/10. The size effect resulted very high (rank-biserial correlation). Finally, we evaluated our trainees' future perspectives on LUS use in clinical practice in the first assessment of suspected or confirmed COVID-19 cases. More than 80% of participants expect a significant improvement in their performances in clinical practice after the course (28.1% strongly agreed, 59.4% agreed, 12.5% no opinion), whereas more than 50% of participants had no opinion about a significant improvement of their patients' outcome as a result of this training program (6.3% strongly agreed, 34.4% agreed, and 56.2% no opinion, 3.1% disagreed). About 75% of participants plan to change their diagnostic strategy for COVID-19-suspected or -confirmed patient evaluation after this course (6.3% strongly agreed, 68.7% agreed, 21.9% no opinion, 3.1% disagreed). It is worth to highlight that almost 50% of participants felt they would need a high level of commitment to make these changes (6.3% very high level of commitment, 46.9% high level, 15.6% no opinion, 31.2% low level). Possible barriers that may impact the use of LUS in primary care settings were found to be lack of time (25% agreed), lack of sufficient confidence in the use of LU (21.9% agreed), lack of evidence-based guidelines (34.4% agreed), lack of sufficient competence in the use of LU (21.9% agreed), lack of patient compliance (6.3% agreed), and lack of adequate equipment (6.3% agreed). The present study showed that even a relatively short training program, but comprising a hands-on practical training on real patients, is capable to bring even almost novices to achieve a high overall accuracy and reliability in detecting major COVID-19 LUS abnormalities and to increase physicians' self-confidence in using LUS. Trainees were highly performants in identifying normal regular A-pattern with confidence. Self-reported confidence was the highest (8/10 IQR 7-9/10). Participants tended to slightly overestimate the presence of a B-pattern. This can be at least partially explained by the occurrence of vertical ultrasound artefacts not cancelling the normal A-lines (sometimes called "Z-lines"). We remind that "real" B-lines must delate horizontal A-lines. Nevertheless, the accuracy in the detection of the B-pattern was very high, similar to that of the A-pattern. Trainees were extremely accurate not to overlook any LUS areas affected by a B-pattern (sensitivity 0.96). As a confirmation, trainees self-reported a high confidence in identifying the B-pattern (8/10 IQR 7-8/10). Differently from the detection of the A-and B-patterns, trainees were some less accurate in detecting irregular pleural thickening and/or small sub-pleural consolidations ("C-pattern"). This can be at least partially explained by the focality of these alterations and the fact that they can be surrounded by a normal A-lines pattern. Interrater agreement anyway remained good (0.63) with 95% CI lower limit above 0.50. In keeping with the greater difficulty in detecting these alterations, participants self-reported the lowest confidence in identifying the "C-pattern" between the three LU patterns considered, remaining anyway good (6.5/10 IQR 6-7/10). The very high effect size (rank-biserial correlation of 1 and 0.95) indicates that almost every participant felt significantly less confident in detecting the so-called C-pattern rather than the A-or B-pattern. It has been already shown in clinical practice that the presence of a B-pattern is the only independent LU finding associated with appropriate referral to a hospital for a COVID-19-suspected case evaluated by PCP. 2 Therefore, the fact that the B-pattern was detected very accurately by the trainees, with a very high sensitivity and with a very high selfreported confidence, appears of high potential clinical impact for outpatients' healthcare practice during COVID-19 pandemic. While the C-pattern is quite specific for COVID-19, a study demonstrated a not negligible incidence of this pattern in normal subjects. 11 The presence of multifocal peripheral consolidations, if associated with bilateral and multifocal B-lines, is highly suggestive of COVID-19, but their absence still gives COVID-19 lung ultrasound for primary care physicians at least an intermediate probability of the disease. 4 Thus, the low sensitivity of trainees in describing this pattern should not further preclude the correct identification of COVID-19 lung involvement, but rather points to the need of a more extensive focus in teaching the correct detection of the C-pattern. PCPs participating in this training program were young and with a limited working experience and almost no skills in LUS. On the one hand, this strengthens the fact that it is not necessary to be very experienced to quickly understand and practice LUS in a specific context and/or for a specific condition after an appropriate training. On the other hand, it cannot be speculated whether older and much experienced physicians would have done better or worse. In general, this training course has significantly increased self-evaluated competence in LU: 100% of participants felt their competencies increased and median increase in competence was +4.5/10 leading to a median self-evaluated competence of 7/10 after the training. The high effect size (rank-biserial correlation) highlights that almost all participants experienced a significant increase in their competencies. This short training program seemed capable to lead to a significant improvement in PCPs' performances and to a change in their diagnostic strategy about COVID-19-suspected or -confirmed cases. A limitation of the current study, devoted to the teaching methodology, was that no conclusion can be drawn whether such increase in trainees' competence will really translate into an improvement in patients' outcome. Moreover, the hands-on practice on real COVID-19 patients was indeed very beneficial for a rapid learning, but it has limited applicability out of a situation of pandemic. The first barrier to the application of LU in clinical practice was identified by the trainees in the lack of precise evidencebased guidelines (34%). We hope that additional studies will further strengthen the evidence about the consistency of LUS in COVID-19. The second major barrier resulted the lack of time (25%). This could be difficult to be modified in a context of pandemic. However, the duration of the examination can be shortened with an increasing experience of the examiner in the field. In conclusion, we demonstrated that the learning curve for correctly identifying COVID-19 major LU abnormalities can be short. Such a course can improve the performances of physicians involved in the first evaluation of suspected or confirmed COVID-19 cases in primary care facilities and thus could improve the correct referral of outpatients to the hospital. By extension, the acquisition of competencies in LUS could change the PCP approach to the evaluation of any outpatient with a pulmonary disease. Indeed, LUS is considered to be one of the most important new diagnostic tools in emergency medicine. 4 We hope that similar training programs might lead to make LUS become one of the most important diagnostic tools also in outpatient and home care in the near future, given the wide range of information provided by LUS, which largely extend beyond a COVID-19 pandemic context. We strongly encourage to continue sharing competencies between physicians of any speciality during COVID-19 pandemic. Usefulness of lung ultrasound examinations performed by primary care physicians in patients with suspected COVID-19 Benefits, open questions and challenges of the use of ultrasound in the COVID-19 pandemic era. The views of a panel of worldwide international experts Lung ultrasound for the early diagnosis of COVID-19 pneumonia: an international multicenter study Comparison of admission chest computed tomography and lung ultrasound performance for diagnosis of COVID-19 pneumonia in populations with different disease prevalence Lung ultrasound is a reliable diagnostic technique to predict abnormal CT chest scan and to detect oxygen requirements in COVID-19 pneumonia Our Italian experience using lung ultrasound for identification, grading and serial follow-up of severity of lung involvement for management of patients with COVID-19 Thoracic ultrasonography: a narrative review Interobserver agreement of lung ultrasound findings of COVID-19 Lung ultrasound is often, but not always, normal in healthy subjects: considerations for COVID-19 pandemic Supplementary material is available at Family Practice online. The authors received no financial support for the research. F.P. declares to have received honoraria from Samsung and ESAOTE, for speaker bureau activities and from GE and Bracco as a consultant. All other authors declare no conflict of interest.