key: cord-0788690-ytu0sudk authors: Bidner, Amber; Bezak, Eva; Parange, Nayana title: Evaluation of antenatal Point-of-Care Ultrasound (PoCUS) training: a systematic review date: 2022-04-05 journal: Medical education online DOI: 10.1080/10872981.2022.2041366 sha: 7150e0c401474627881837b922c83872ae4aac5b doc_id: 788690 cord_uid: ytu0sudk INTRODUCTION: There is limited access to life-saving antenatal ultrasound in rural and low-resource settings largely due to shortages in skilled staff. Studies have shown healthcare practitioners can be upskilled in PoCUS through focused training, offering a viable solution to this deficit. However, standards for training and competency assessment are unclear and regulation surrounding practice is lacking. We aimed to review published literature examining antenatal PoCUS training programs, comparing teaching approaches and study methodologies. METHODS: A search of electronic databases EMBASE, MEDLINE and Google Scholar was conducted. Original research articles evaluating antenatal PoCUS training of healthcare professionals worldwide were identified for analysis. Articles with limited detail on the PoCUS training intervention and those describing comprehensive diagnostic training programs were excluded. Evaluations were compared against the Kirkpatrick Evaluation Framework (KEF). RESULTS: Twenty-seven studies were included from an initial search result of 484 articles. There was considerable heterogeneity between the PoCUS training programs described. Course duration ranged from 3 hours to 2 years, with 11 of the 27 studies delivering obstetric-exclusive content. 44% trained multidisciplinary groups of health professionals. Long-term follow-up training and skills assessments were lacking in over half of the reviewed studies. Study quality and reporting detail varied, but overall beneficial outcomes were reported with 3/4s of the studies reaching upper KEF levels 3 and 4. CONCLUSION: PoCUS performed by upskilled healthcare professionals offers an attractive solution to the problem of inequitable access to antenatal ultrasound. A review of available literature highlighted a paucity of comparable high-quality studies needed to establish a stronger evidence base for antenatal PoCUS, and a need to standardise training and competency assessment. This review may inform educators, researchers and policy-makers on existing training formats and methodologies to assist in establishing best practice antenatal PoCUS training methods for safe service delivery by remote healthcare professionals. Antenatal ultrasound is the primary imaging modality in pregnancy [1, 2] , routinely used to estimate due dates, monitor fetal growth and well-being, detect anomalies, and guide specialist referral [3, 4] . It can also facilitate the early detection of life-threatening complications such as ectopic pregnancy, fetal malpresentation, multiple pregnancies, placenta praevia and placental abruption [5] [6] [7] [8] . The International Society of Ultrasound in Obstetrics and Gynaecology (ISUOG) have published guidelines recommending women receive two antenatal US examinations during a normal low-risk pregnancy [9] . However, studies into service accessibility in rural, remote and low-resource settings around the world indicate women are not receiving this care [10] [11] [12] . The WHO estimates most maternal deaths are preventable, with over 90% occurring in low-resource settings. Approximately 86% of estimated global maternal deaths in 2017 were attributed to the developing nations of sub-Saharan Africa and Southern Asia [13] . The majority of neonatal mortality also occurred in these regions [14] . A 2015 survey of healthcare providers in South America, Africa and Asia reported the primary reason for not making ultrasound available to pregnant women was a lack of suitable education [15] . Skill shortages are also evident in developed nations like the USA, Canada and Australia, where many remote medical centres have no onsite sonographer and rely on visiting professionals available as infrequently as one day per month [10, 16] . The recent COVID-19 pandemic has increased uncertainties in travel and logistics, impacting locum staffing in rural areas and highlighting the importance of trained remote healthcare workers. Accurate estimation of due dates and early detection of potentially life-threatening complications are crucial for remotely located women, who may need days of travel to access specialist obstetric care [7, 10, 17] . It is in these low-resource settings that antenatal Pointof-Care ultrasound (PoCUS) can offer substantial benefits. Modern portable ultrasound machines capable of producing high-quality images are affordable and have helped establish PoCUS in many medical fields [1, 3] . Performed and interpreted at the bedside by the healthcare provider, PoCUS allows for focused studies to assist procedures or direct care and referal [18] . As a highly skilled, operator-dependent modality, PoCUS requires appropriate and ongoing training of experienced healthcare professionals [3] . It takes years of study and training to produce qualified sonographers, and once trained it is challenging to entice these professionals to relocate and remain in rural locations [16, 19, 20] . There is growing evidence indicating PoCUS training programs can effectively teach the skills necessary to allow for task-shifting of focussed ultrasound examinations from sonographers to doctors, nurses and midwives [21] [22] [23] [24] [25] [26] . Ultrasound training is being increasingly incorporated into undergraduate medical curricula and on the job training [26] [27] [28] , but it is less well established in nonphysician (nursing and midwifery) programs [29, 30] . In most developing countries and low-resource settings, antenatal care is provided primarily by midwives and nursing staff, which presents an opportunity to task-shift and upskill these essential workers [21] [22] [23] [24] [25] [26] . Table 1 lists general PoCUS workshop requirements and methods for assessing competency [31] [32] [33] . The WHO recommends a standardised curriculum and competency assessment be adopted by all countries for antenatal PoCUS training [5] . However, training guidelines and standards to ensure a minimum level of competency for safe practice vary between countries, with PoCUS remaining largely unregulated globally [34] [35] [36] . In many countries, health practitioners may perform PoCUS with little or no training, and without formal accreditation, leading authorities to call for reform and regulation of its use [16, 37] . Ultrasound performed by untrained clinicians may represent a higher risk of misdiagnoses. Overlooked health conditions may lead to delayed diagnoses and treatment ('false negatives'), while misinterpretations and incidental findings ('false positives') can cause considerable patient anxiety and unnecessary follow-up investigations, increasing the economic burden on the healthcare system [38] [39] [40] . Endorsement for outreach training programs have been provided by the Australasian Society for Ultrasound in Medicine (ASUM), ISUOG, World Federation for Ultrasound in Medicine and Biology and RAD-AID, indicating a global effort to address the skill shortage [35] . This literature review examines the training and evaluation methods being employed to teach antenatal PoCUS to medical and allied health professionals. This review investigated international literature on antenatal PoCUS education from 2000 to January 2021, focusing on publications that evaluate the efficacy of training models. It has adopted the 'Preferred Reporting Items for Systematic Reviews and Meta-analysis' (PRISMA) guidelines and was formerly registered with the international prospective register of systematic reviews (PROSPERO), registration number CRD42021230267. A team of three researchers from the University of South Australia with experience in research, tertiary education and clinical practice (including ultrasound and PoCUS training in low-resource settings) conducted the review and reached consensus on the eligibility criteria, search strategy and terms, final article inclusion, data extraction and quality assessment. A systematic search of electronic databases EMBASE and MEDLINE was conducted through OVID for original research literature, performed on the 7 th of January 2021. An experienced librarian was consulted to assist in the design of search terms and strings, which were then reviewed by all members of the research team. The search was limited to 'Human' and 'English' language only. No restrictions were set regarding the publication year. Search terms used for both databases were grouped into main four areas and combined using terms synonymous with pregnancy, ultrasound, point-of-care, and training. A grey literature search was conducted through Google Scholar (5 pages, 50 results) using the key search terms (pregnancy/antenatal, point-of-care ultrasound/ultrasound, and training/education). Connected papers (https://www.connectedpapers. com/) was searched to canvas for additional relevant articles (see Appendix Table A1 : Search strategy). For inclusion, an original research study must have described and evaluated an ultrasound training program intended for point-of-care or bedside application on antenatal patients. Articles involving PoCUS obstetric training as part of a broader training curriculum were eligible. All medical and allied health specialties were included as the training participant population, and pre-graduate students from all health disciplines. No restrictions were placed on study/ training setting. Studies evaluating advanced training (complex and interventional scanning) or formal diagnostic ultrasound programs leading to qualification as a sonographer were excluded. Articles with limited descriptions of the training program provided were excluded, as were conference abstracts/reviews, editorials, commentaries and letters (see Table 2 : Eligibility criteria). The database search was performed by the primary author. Duplicates were removed prior to the initial title and abstract screening conducted independently by two reviewers, who then performed full-text reviews. Non-consensus at both initial title/abstract screening and later full-text screening was decided by a third independent reviewer. The citations of all identified articles included in the review were searched, with title/abstract then full-text screening performed by two independent reviewers. All three reviewers discussed themes and agreed on data points to be extracted, and reviewed these predetermined categories in a Microsoft Excel spreadsheet prior to data extraction. Data was extracted to the spreadsheet and collated by the primary author and examined by a second reviewer. Meta-analysis was not possible due to the heterogeneous teaching and assessment methodologies used, thus an integrative approach to data synthesis was employed. Emerging themes were discussed amongst all three reviewers and a narrative response was composed. Comprehensive tables summarising the reviewed studies' training and evaluation methods, and key investigated outcomes are included to facilitate comparison. A modified Medical Education Research Quality Index (MERSQI) tool (see Appendix Table A2 ) was used to assess the quality of the included articles. The MERSQI is a validated assessment instrument used in medical education research to measure the quality of experimental, quasi-experimental and observational studies [41, 42] . The tool was modified to include two categories: 1) 'Number of trainees' (score of 0.5 to 1.5), and 2) 'Follow-up of training' (score of 0 to 3). These domains were considered valuable to this review given the variation of recruited participant numbers between studies (impacting study power), and the importance of follow-up training, assessment and ongoing support of trainees for learning PoCUS and ensuring safe practice. The percentage response rate was omitted as this measure was not applicable to the vast majority of the studies. Each domain was scored out of 3. The primary author performed the quality assessment, grading all articles within a potential score range of 5 to 21. A risk of bias assessment was also conducted relating to five key areas of educational development: Underpinning bias, Resource bias, Setting bias, Content bias, Educational/ Development bias (see Appendix Table A3 : Risk of bias assessment tool). However, assessment of quality and risk of bias did not restrict article General workshop requirements defined by *ASUM [31, 32] Methods for assessing competency in PoCUS [33] Faculty-must include a medical specialist with appropriate and extensive clinical experience/qualifications. Instructors must have significant practical experience in the application being taught. Registered sonographers can assist with teaching skills. Technical competency assessment-Probe selection, image mode selection (e.g., cardiac, obstetric), proper image orientation, probe positioning, depth, gain, centering of target structure, demonstrates advanced functions (M-mode, Doppler, image capture), troubleshooting. Teaching (including practical) hours should at least meet those published in credentialing syllabus for the application taught. inclusion in this review's final synthesis, ensuring the inclusion of a broad cross-section of literature representative of the range of study quality and methodologies in published circulation. Each study reviewed was compared against The Kirkpatrick Evaluation Framework (KEF), a validated four level model designed to evaluate and classify training and development programs [43, 44] . The four levels described (Level 1-Reaction; Level 2-Learning; Level 3-Behaviour; Level 4-Results) by the Kirkpatrick model represent a continuum of complexity and value in evaluation measures. Level 4, representing the highest evaluation measure (healthcare or patient-related outcomes), assesses impact and aligns with the MERSQI 'Outcomes' domain. Twenty-seven studies were included, from an initial 484 articles retrieved in the OVID database search during the identification stage (see Figure 1 : PRISMA framework). Of the 27 articles, 16 were identified through EMBASE and MEDLINE, with a further 11 obtained through citation searches. Google Scholar and Connected papers searches yield no additional eligible articles. The identified studies spanned two decades; the oldest study published in the year 2000. Three-quarters were published after 2014 reflecting the growing interest in PoCUS research. Most were single arm interventional studies using convenience samples of health professionals and patients. Four of the studies were conducted by the same research team, Shah et al. [45] [46] [47] [48] . Twenty studies were conducted in developing countries, of which only five were conducted in urban settings. Seven studies included developed countries (five in the USA, all urban areas with one intended for rural deployment; one in Australia and one in Denmark). Two of these studies used training sites in both developed and developing countries; one [49] employed a clustered randomised control trial design that was carried out over five clinical sites located in Africa, Asia and America, and the other [50] compared six training sites and formats conducted in rural and urban settings in Australia, Timor Leste and Indonesia. Key outcomes investigated and the main findings of the studies are summarised in Table 3 . A summary of the included studies' teaching methods, including duration and location, trainee and instructor demographics, curriculum and practical skills taught, and follow-up training/support, is provided in Appendix Table A4 : Training methods and delivery. Course duration ranged from 3 hours to 2 years. Obstetric-exclusive curricula were delivered in 11 of the 27 studies (see Figure 2a ). Of the studies that covered multiple-organ systems, several delivered their content in as few as 1-2 days [21, 22, 51, 52] , with either refresher sessions or additional online learning modules provided to support the intensive practical training. When designing intensive courses with limited delivery time, other resources to supplement face-to-face training, such as digital and written learning modules/resources, and online/distance teaching and feedback are important. Many of the studies employed these supportive measures (see Appendix Table A4 ). Maintaining a focussed curriculum for specific predefined clinical indications and remote technical assistance was recommended [51] . The course topics and practical skills taught by each study are listed in Table A4 of the Appendix; these ranged from basic assessments like fetal lie/presentation to more complex fetal biometry, heart assessment and the detection of anomalies such as ventriculomegaly, anencephaly, hydronephrosis, and spina bifida. A training needs assessment was conducted by five of the studies [24, 26, 46, 47, 53] which informed the choice of ultrasound applications that were most relevant for program inclusion. Specific skills were taught if relevant to endemic needs; for example, three studies [51, 54, 55] conducted in Tuberculosis prevalent regions included ultrasound assessment of HIV/Tuberculosis (FASH) in their multisystem curriculum. Over two-thirds of the studies detailed ultrasound physics and instrumentation in their curriculum, and four specifically mentioned 'safety' [30, 45, 49, 56] , consistent with the WHO's recommendations that trainees are able to monitor mechanical and thermal indices on equipment and understand safety concepts underpinning ultrasound exposure of the patient and fetus [57] . Trainees should understand the limitations of their focussed training and seek assistance with image interpretation and patient management decisions from an experienced sonographer and/or clinician where necessary. Appropriate documentation of the PoCUS exams findings and patient referral is also important. Three of the studies listed 'documentation' in their curriculum [29, 30, 58] , one specifically including 'liability and risk reduction strategies' [29] . Lee [55] 2017 Pre & post course trainee knowledge assessment. Post training practical skills assessment and course evaluation survey (scanning confidence & intent-to-use US). Average pre-course exam score was 35.2% (2.4% pass rate). The average post-course exam score was 82.0% (92.7% pass rate). Average practical score on completion of the course was 83.2% (SD = 0.145) with 82.9% of the class passing (pass mark above 75.0%). Post-course survey-overall increased level of comfort performing all scans. Cardiac followed by OB US were anticipated to be most frequent indication. Lindgaard [52] 2017 Expert review of trainee images. Expert to trainee agreement for intrauterine pregnancy-100%, GA-93%. Low-to-moderate complexity PoCUS exams performed by GPs with sufficient prior training have a very high level of inter-rater agreement when compared to exams conducted by radiologists & gynaecologists. (Continued ) Participant trainee numbers in any single study ranged from 3 to 162, with a total of 903 trainees across all included studies. Over half of the studies trained fewer than 20 trainees, limiting study power (see Figure 2b ). Of the studies that reported trainees' prior ultrasound experience, nine trained participants with mixed ultrasound experience, six enrolled trainees with limited but similar ultrasound exposure and seven restricted eligibility to trainees with no prior ultrasound training or experience (see Figure 2c ). Delivering a program to suit trainees with varied ultrasound education and proficiency was an acknowledged challenge [22, 51] . Shokoohi et al. [51] raised concerns that more experienced trainees may not have received the same benefit from their curriculum, particularly in the introductory session. Grouping trainees and designing specific curricula and objectives catered to each groups' experience and skill level could mitigate this problem. The future of healthcare and its education is moving towards a more cooperative interdisciplinary culture [59] , demonstrated in the multidisciplinary participant groups trained together in almost half of the reviewed studies. Ten (33%) studies exclusively trained medical physicians and only 5 (19%) were dedicated to nurses and midwives (see Figure 2d ). Several authors [29, 50] described unique challenges to interprofessional teaching, where individual needs and preferences can vary significantly between trainees. This scenario, however, provides the opportunity to use trainees' unique experiences and individual strengths to enhance course design and foster collaborative practice. Shaw-Battista et.al [29] . reported enhanced inter-professional collaboration as a benefit of their multidisciplinary PoCUS training initiative, stating it fostered better communication, coordination of care and understanding of other professionals overlapping and unique scopes of practice. Providing clear objectives, varying curricula in breakout groups tailored to participant's experience level, and the opportunity for peer-led teaching in mixed skills/experience groups would benefit a multidisciplinary PoCUS training cohort. Instructor-to-trainee ratios were reported in over half of the studies, mostly for practical training sessions, and ranged from 1:1 to 1:6; guidelines have recommended a maximum ratio of 1:5 [32, 60] . The instructors' experience ranged from second year medical students (aimed at providing low-cost outreach training in developing countries [54, 55] ) to trained sonographers and medical specialists. Some courses benefited from a multidisciplinary teaching team with specialised content delivered by experts in their field (cardiac, paediatric and obstetric specialists). Several articles included principles of teaching in their curriculum [45, 61] , and reported scenarios where trainees went on to teach ultrasound to colleagues on return to work [51, 61, 62] . Training the trainer initiatives, where trainees are taught teaching methods to pass on learning to colleagues, is a useful and potentially cost-saving option in settings with limited resources and access to training. Care should be taken using this teaching model with PoCUS courses of short duration, given the complexities of learning ultrasound and the often unsupervised work environments trainees return to. Instructor qualifications and teaching ratios are provided in Appendix Table A4 . Few papers mentioned the ethics surrounding the use of pregnant patients for ultrasound training purposes. Ideally, practical training would utilise both simulated and real-life patients with strictly limited times placed on scanning pregnant volunteers and a heavier reliance on phantom models and virtual/simulation technologies in early training [50, 63] . In most studies, only healthy models were used during training, which precludes the demonstration of pathology [29, 55] . Simulation can be beneficial in this respect and offers the advantage of learning in a safe, patient-free environment. They have a particular utility in obstetric ultrasound training, but the cost of implementing high-fidelity simulated systems would be prohibitive in the majority of settings reviewed [64, 65] . Westerway [50] reported using commercial and handmade phantoms and Vinayak et al. [63] used 'scanning phantoms' in the initial week of training, but no high-fidelity simulation systems were used in any of the reviewed studies. Shaw-Battista et al. [29] discussed the ethics of using pregnant models, stating their intention to introduce equipment to simulate first-trimester ultrasound in their next course iteration. This would reduce reliance on pregnant volunteers and provide the opportunity to scan simulated first trimester pregnancies, commonly lacking in training courses due to the early gestation of the fetus and risk of identifying an unexpected abnormality in pregnant volunteers who are yet to receive formal scanning. Follow-up training is beneficial to reinforce learning and provides the opportunity to assess knowledge retention, which is important for continued safe practice. Follow-up training sessions, in either face-to-face or online format, were offered by half the studies, with periods varying from 3 months to 2 years. Appendix Table A4 summarises the followup training and support provided by the reviewed studies. Of those who did not report/provide additional follow-up training sessions, five offered assistance through personal telecommunications feedback and image review. Telemedicine was investigated by Kolbe et al. [26] who provided remote real-time scanning and image review. Vinayak et al. [63] used an asynchronous method, where trainee images and interim report was sent for specialist review while the patient waited. Other studies used telecommunications via email and various messaging platforms to remotely assist trainees, provide feedback and review images, but only Kolbe et al. [26] used remote real-time scanning supervision. The approach and method used to evaluate the courses and trainees varied widely across studies. A summary of the evaluation methods reported including knowledge and practical assessment, expert image review, frequency and application of scanning, patient outcomes and trainee feedback/survey is provided in Table 4 : Trainee & course evaluation. Fourteen studies performed knowledge assessment of the trainees. Nine of these conducted pre-and post-training tests, and of these, five administered identical exams before and after training. Utilising the same exam allows for a quantifiable measure of trainee improvement but can introduce bias. This risk may be mitigated by randomising question order, not informing trainees the test would be re-administered or discussing test results. Trainees from five studies were required to pass a written test following online self-directed learning before proceeding to practical training. Pre-course learning and testing saves face-to -face time for hands-on learning and can ensure trainees have similar base knowledge on course entry. Six studies reported performing follow-up (knowledge retention) assessment in addition to any immediate post-course testing. Practical assessments or Objective Structured Clinical Examinations (OSCE) were conducted in 16 of the studies, of which half performed consecutive testing allowing for improvement measures. Overall, half the studies performed expert image review (remotely or during training) for quality assurance and to assess competence, and for some, to guide feedback and the necessity for refresher training. This is a useful competence measure where direct supervision is not possible, as it may be performed asynchronously and remotely. Written or practical assessments were not described by seven of the studies. Of these, expert image review was performed by four studies as an indirect evaluation method. Almost half the studies investigated patient outcomes, several going further to ascertain if PoCUS changed the patient diagnosis and if this impacted their management/treatment. The clinical application was evaluated in nine studies, and in some cases, the frequency of scanning following training was used as an evaluation measure [21, 24, 30, 51, 53, 62, 66, 67] . Other measures useful for quality control and course improvement are post-course evaluation surveys, conducted by seven [21, 29, 45, 50, 51, 54, 55] of the reviewed studies, and self-reported post-course scanning confidence that was evaluated by three [45, 54, 56] . Several studies [48, 63] used 'time to complete scans' as an evaluation measure. In Vinayak et al. [63] scan times halved after 30 completed examinations with consistent image quality. While not the best measure of competence, speed is an important consideration in time poor, resource-limited settings where extensive scanning times could be prohibitive to PoCUS examination during antenatal consultation. Shah's et al. [48] study demonstrated that an entire focussed ultrasound assessment (fetal heart rate, head position and estimated gestational age) could be completed by trainees in under 5 minutes, important when facing timesensitive decisions in an emergency caesarean delivery. With the aim of revising policy and training to align with the credentialing requirements of their site/country, Lathrop et al. [58] investigated the introduction of a learner portfolio (documented evidence of didactic learning, teaching resources, logged cases and images) and evaluation rubric to demonstrate competence. This approach is consistent with other accrediting authorities' requirements of scanning logs and evidence of completing a pre-set number of studies. Lathrop et al. [58] was the only study whose participants all progressed to formal accreditation in the use of antenatal PoCUS. It should be noted many of the reviewed studies were outreach projects that aimed to maximise training opportunities in low-resource settings. While minimal trainee assessment may have been undertaken by some, it is possible that the methods reported were not the only means of assuring trainee competence. Ideally, some form of competence assessment should be performed and support options provided before trainees perform unsupervised clinical scanning. All studies concluded positively regarding the PoCUS training intervention investigated. The outcomes investigated by each study and their main findings are summarised in Table 3 . Longer-term follow-up training and skills assessments, essential to building confidence and ensuring competence and retention of learning [36] , was lacking in over half of the reviewed studies. Insufficient onsite supervising experts on return to clinical practice was also a recurrent theme [21, 29, 51, 56, 58, 66, 67] . This problem is further compounded in some locations by poor telecommunications access, which impedes off-site assistance [22, 51, 66] . In such cases, telehealth, which is emerging strongly in the wake of the COVID pandemic, would be ineffective as a tool for real-time support of trainees. Several articles in this review offered off-site asynchronous expert image review for quality assurance and feedback [26, [45] [46] [47] 51] . This solution is inadequate in cases where technical hands-on correction is required or immediate image review is needed to guide patient management in an emergency. Another common barrier included access to quality ultrasound equipment following training. Henwood et al. [53] reported some trainees did not have routine access to ultrasound and the ability to save images from completed examinations, and in Westerway [50] , not all trainees scanned patients on return to work due to no or poor/faulty equipment. Busy departments allowing little time for scanning or supervision was another reported barrier. Half of the midwives surveyed in Kimberly et al. [62] reported difficulty finding time to perform ultrasound due to heavy clinical workloads and raised concerns over neglecting other clinical obligations. Almost half of the participants surveyed by Shokoohi et al. [51] listed 'lack of time to scan' as the main perceived challenge integrating PoCUS into patient care. There was a distinct lack of pedagogy described by the reviewed studies. Only Westerway [50] provided a detailed discussion describing the New World Kirkpatrick training evaluation model. Elements of pedagogy were present in other studies' designs. For example, a constructivist, flipped classroom approach was utilised by a number of training programs [22, 51, 52, 63] , but limited description of the principles and foundations of this model were provided. Generally, the studies suffered from limitations and biases inherent in research conducted in remote settings, including small study designs-low participant numbers, convenience samples, and loss to follow-up. A lack of longer-term follow-up of trainee outcomes was a reported limitation of many of the reviewed studies [46, 49, 62, 66, 67] . This is likely the result of geographic isolation, finite funding and an overburdened and transient health workforce [68, 69] . Of those that did follow trainee progress, comparison of participants was confounded for some by inconsistent ultrasound exposure on their return to different workplaces, with varying onsite assistance and supervision between sites. Eleven of the studies performed expert review of trainee images, most asynchronously with the expert unable to perform concurrent scanning to verify trainees' findings [49, 58, 63] . Blinded image review was specified by several authors [45, 70] to mitigate this shortcoming. Poor and worsening participant compliance with patient data recording and image logs was reported by several studies [62, 66] , some basing image review and quality control on scans selected for uploading by the trainees, potentially biasing results [49] . The overall quality of studies and their evidence varied, with the MERSQI assessment scores ranging from 7.5 to 18 out of 21 (mean score of 12.9). The review lacked randomised, controlled studies necessary to achieve a top MERSQI score. The upper KEF levels 3 and 4 were reached by 3/4s of the included studies. Studies reaching KEF levels 1-2 had an average MERSQI score of 11.2 and KEF Levels 3-4 averaged 13.5. Figure 3 illustrates the KEF level reached by the included studies and corresponding average MERSQI score. Table A5 of the Appendix presents the MERSQI evaluation of each reviewed study inclusive of corresponding KEF level and colour coded bias assessment ranking. The limited use or reporting of conceptual frameworks and models underpinning the development of the educational programs reviewed is reflected in the generally low MERSQI validity scores and underpinning bias ratings; categories commonly underreported in medical education research [71, 72] . However, it should be noted that 'limited reporting' increases the risk of bias but does not necessarily mean the educational development is of poor quality [72] . This review identified 27 observational studies of moderate to low quality within the specified eligibility criteria despite an inclusive search strategy (obstetric or obstetric inclusive PoCUS training of professionals from any healthcare discipline globally). Description of theoretical concepts or pedagogy underpinning the training programs was generally lacking. Substantial heterogeneity in training formats was found between the reviewed studies, and half lacked follow-up support, training and assessment important to the safe ongoing practice of PoCUS (see Table A4 -Training methods and delivery). The studies' findings were generally positive, reporting improved knowledge (where pre-and post-assessments were conducted), and competence being attained despite the substantial variation in course durations (3 hours to several years). Variation in competence assessment and levels of evidence was also observed (see Table 4 -Trainee and course evaluation), with 11 of the 27 studies not surpassing KEF levels 1 or 2 of assessing immediate Post training practical assessment, repeated at 6 & 11 month follow-up. Scanning on returning to clinic/work (confidence and application). PoCUS-Point-of-Care UltrasoundUS-Ultrasound reaction to training and knowledge gained. This is a recognised trend in which medical education researchers commonly cease evaluation at the lower KEF levels, finding longer-term investigations required for evaluations at levels 3 and 4 difficult to accomplish [73, 74] . This is especially the case for PoCUS training research, which is predominantly conducted in low-resource settings where additional logistical challenges exist. We identified several recently published systematic reviews of PoCUS training [36, [75] [76] [77] . All were limited to the instruction of medical physicians, excluding allied health professionals. Other differences included setting, with Rajamani et al. [36] (2020) and Andersen et al. [76] (2019) focusing on general practice and critical care, respectively. Dickson et al. [77] (2017) included studies of trainees with no prior formal ultrasound training only. Consistent with this review's findings, variation in teaching curriculum and assessment methods, and an overall low standard of study quality were unanimously reported. Rajamani et al. [36] reported a distinct lack of high-quality evidence on PoCUS competence, with two-thirds of the reviewed studies failing to describe important details on how assessments were conducted, and very few utilising bias minimisation strategies important to observational study designs [78] . None of the 42 studies identified by Rajamani et al. performed follow-up repeat assessments essential to assessing learning retention and safety to practice, and most of the educational programs reviewed failed to follow recommended processes for assessing PoCUS competence. Anderson et al. [76] found a 'great variety of pedagogic approaches' and substantial disparity in training durations (2 to 320 hours) between their included studies, which were reported to be of 'low quality. . . mainly because of issues with design and reporting'. Assessment methods also varied but focussed PoCUS scans were found to require less training. Higher diagnostic accuracy and frequency of clinical use was also reported for obstetric indications, consistent with the findings in this review. These outcomes, and the need to use non-ionising imaging modalities in pregnancy highlight the utility of antenatal PoCUS in the hands of appropriately trained healthcare providers. This literature review has relevance to medical educators, researchers, clinicians and policymakers interested in developing curriculum and translating PoCUS safely into clinical practice. To the authors' knowledge, this is the first published systematic review of worldwide antenatal PoCUS training of multidisciplinary healthcare clinicians. It provides comprehensive tables summarising the reviewed studies PoCUS teaching and evaluation methods. The variable training and evaluation methods described and limitation in reporting made direct comparison of study results for metanalysis unfeasible, meaning effective and ineffective training approaches could not be confidently discerned. Whether one approach to training and assessment was superior to another could not be reliably established, limiting this review. Such heterogeneity, while complicating synthesis of evidence, can offer the advantage of examining the consistency of findings and generalizability of interventions across studies, assessing the relative feasibility of different educational approaches [71, 79] . While the level of evidence and detail in reporting was lacking in many of the reviewed studies, a strength of this review is evident in the identification of this gap and the onus for more comprehensive and comparable longer-term studies with which to establish a stronger evidence base for antenatal PoCUS. An inclusive search string with broad criteria ensured a wide cross-section of antenatal PoCUS studies could be scrutinised. However, only two peer reviewed medical databases (MEDLINE and EMBASE) were searched and 'English only' articles included. Published studies from non-English-speaking countries in particular would have been missed. Robust clinical studies demonstrating the efficacy of training models, and the clinical impact of trainee operated antenatal PoCUS on quality of care and maternal/fetal outcomes are needed. Economic analyses investigating the cost-effectiveness of PoCUS training and implementation would be valuable to justify and inform future programs. The use of Telehealth with antenatal PoCUS for real-time scanning assistance (Teleultrasound) in the clinical setting also merits further investigation, along with Artificial Intelligence systems that have the capacity to assist minimally trained operators in unsupervised clinical environments [80] . Advancements in Teleultrasound systems now make it possible for the remote clinician to view the ultrasound monitor, images and probe position, communicate via live video and text message with the operator, and even take control of the ultrasound machines functions and demonstrate findings to the patient, all in real time [81] . Many of the reviewed studies are unlikely to have the resources and infrastructure (quality internet/broadband) to support such advanced systems, but they do offer considerable advantages for remote supervision of ultrasound trainees and may see greater utilisation in the future as the technology becomes more affordable. Only one article in this review reported on the patient experience. Vinayak et al. [63] surveyed patients, reporting they felt trainee performed PoCUS during antenatal care was safe, convenient & reassuring, providing a better antenatal visit experience, increased confidence in care delivery and increased spouse attendance. Considering the propensity for cultural minorities to live in remote communities and the underutilisation of Antenatal care services in these regions [82-84], investigating the patient and partners' perspective on trainee provided antenatal ultrasound would be beneficial to inform curriculum development with an aim to providing culturally sensitive patient centred care. PoCUS is an increasingly utilised diagnostic tool that can enhance the physical exam and guide clinical decision making. It has particular utility in rural clinical practice and developing countries, where advancements in ultrasound equipment and telemedicine are opening new avenues for its establishment. Its lack of regulation and rapid expansion into most specialties underscores the need to establish standards in PoCUS training, competency and on-the-job scanning, ensuring providers are safe to practice. Acknowledging the urgent need for these skills and the difficulty accessing training in remote areas, regulation must be implemented carefully to preserve the time and financial advantages offered by PoCUS training. Quality education programs require careful and informed consideration in their initial course design, with ongoing review of curricula, training resources, knowledge/practical skills assessment, availability of expert trainers, follow-up support and evaluation. Overall, findings from this review support intensive PoCUS training courses for task-shifting and upskilling of the medical and allied health workforce. However, significant heterogeneity in training, evaluation and research methodologies in the included studies was observed. Quality longitudinal studies with comparable evidence are needed to help frame policy guidelines and inform validated antenatal PoCUS training programs, ensuring the safe implementation of this valuable healthcare resource. This work was part of a larger project, The Healthy Newborn Project, funded by The Hospital Research Foundation. The authorship listing conforms with the journal's authorship policy. All authors (AB, EB, NP) contributed to the review's design, data acquisition, inclusion and interpretation, and were involved in drafting, revision and final approval of the manuscript. No potential conflict of interest was reported by the author(s). The views expressed in the submitted article are the authors own and not an official position of the institution or funder. Additional data referred to but not included in the main manuscript is provided in the Appendix as supplementary material ('Additional Tables'). This systematic review is registered with the International Prospective Register of Systematic Reviews (PROSPERO registration number CRD42021230267). Learning portfolio-(a) documentation of didactic learning (certificates from USrelated training or workshops attended); (b) readings (journal articles, clinical bulletins, & practice statements); (c) US cases. No follow-up of trainees reported but an additional future workshop planned to include more advanced skills, such as fetal biometry, firsttrimester US, & cervical length measurement. (Continued ) (2) Behaviours (2) 14 Levels 1, 2, 3 Adapted from the Medical Education Research Study Quality Instrument (MERSQI) [41, 42] . *Categories added-Number of trainees (score of 0.5 to 1.5); **Follow-up of training (score of 0 to 3). Categories omitted for this assessment-Response rate (%). Possible score range 5 to 21 (original MERSQI score range 5 to 18). Note: Study design-No points were assigned where pre/post training testing was described as part of a course but not included in the analysis and results of the study. Studies were categorised as single cohort where the same training program was delivered to multiple groups, where these groups were not specifically defined as separate cohorts or compared in analysis. Groups trained together then stratified during analysis (e.g. by prior ultrasound experience or clinical role) were categorised as single cohort. KEF level-Image review and interpretation during or immediately after training recorded as KEF level 2-Learning. Image review and interpretation performed as longer-term follow-up recorded as KEF level 3-Behaviour. Where relevant information was not reported or unclear the lowest category score was assigned. Low scores represent limited reporting which increases the 'risk' of bias, but does not necessarily mean the study or educational development is of poor quality [72] . A short history of sonography in obstetrics and gynaecology. Facts Views Vis Obgyn Clinical Practice Guidelines: pregnancy care 2019 edition Point-of-care ultrasonography Imaging and imagining the fetus: the development of obstetric ultrasound WHO recommendations on antenatal care for a positive pregnancy Ultrasound for fetal assessment in early pregnancy The Role of Obstetric Ultrasound in Reducing Maternal and Perinatal Mortality Role of ultrasound in the evaluation of first-trimester pregnancies in the acute setting ISUOG practice guidelines: ultrasound assessment of fetal biometry and growth accessibility of diagnostic imaging equipment around Australia. Australian Parliament House, Canberra Obstetric care in low-resource settings: what, who, and how to overcome challenges to scale up? First look: a cluster-randomized trial of ultrasound to improve pregnancy outcomes in low income country settings Maternal mortality. 2020, World Health Organization Newborns: improving survival and well-being. 2020, World Health Organization Perceived barriers in the use of ultrasound in developing countries Minimum education & training requirements for ultrasound practitioners Point-of-care ultrasound in resource-limited settings: common applications Discussion paper: definition of Point of Care Ultrasound (POCUS) Sonographer accreditation. 2020, Australian Sonographer Accreditation Registry (ASAR) The Australasian sonographers association 2019-20 Australian government pre-budget submission Assessing a novel point-of-care ultrasound training program for rural healthcare providers in Kenya A pilot training program for point-of-care ultrasound in Kenya Collaborative task-sharing to enhance the Point-Of-Care Ultrasound (POCUS) access among expectant women in Kenya: the role of midwife sonographers The implementation of a longitudinal POCUS curriculum for physicians working at rural outpatient clinics in Chiapas, Mexico Handheld ultrasound to avert maternal and neonatal deaths in 2 regions of the Philippines: an iBuntis intervention study Point of care ultrasound (POCUS) telemedicine project in rural Nicaragua and its impact on patient management A mixed-methods evaluation of a multidisciplinary point of care ultrasound program An integrated ultrasound curriculum (iUSC) for medical students: 4-year experience Interprofessional obstetric ultrasound education: successful development of online learning modules case-based seminars; and skills labs for registered and advanced practice nurses, midwives, physicians, and trainees. J Midwifery Women's Health Point-of-care ultrasound education for non-physician clinicians in a resource-limited emergency department A guide to providing an ultrasound workshop Guidelines for course providers seeking CCPU course accreditation Evaluation of trainee competency with Point-of-Care Ultrasonography (POCUS): a conceptual framework and review of existing assessments Ultrasound curricula in obstetrics and gynecology training programs The application of hand-held ultrasound scanner in teaching of telemedicine and rural medicine Longitudinal Competence Programs for Basic Point-of-Care Ultrasound in Critical Care: a Systematic Review Point-of-care ultrasound in obstetrics Safety of ultrasonography in pregnancy: WHO systematic review of the literature and meta-analysis Benefits and risks of ultrasound in pregnancy Safety assurance in obstetrical ultrasound Association between funding and quality of published medical education research Predictive validity evidence for medical education research study quality instrument scores: quality of submissions to jgim's medical education special issue Techniques for evaluating training programes Evaluating training programs the four levels Efficacy of an ultrasound training program for nurse midwives to assess high-risk conditions at labor triage in rural Uganda Development of an ultrasound training curriculum in a limited resource international setting: successes and challenges of ultrasound training in rural Rwanda Impact of the introduction of ultrasound services in a limited resource setting: rural Rwanda BE-SAFE: bedside sonography for assessment of the fetus in emergencies: educational intervention for late-pregnancy obstetric ultrasound Evaluation of focused obstetric ultrasound examinations by health care personnel in the democratic Republic Of Congo Comparing the effectiveness of training course formats for point-of-care ultrasound in the third trimester of pregnancy Assessment of point-of-care ultrasound training for clinical Validation of ultrasound examinations performed by general practitioners. Scand J Prim Health Care Point-ofcare ultrasound use, accuracy, and impact on clinical decision making in Rwanda Hospitals Evaluation of a point-of-care ultrasound curriculum taught by medical students for physicians, nurses, and midwives in rural Indonesia Evaluation of a point of care ultrasound curriculum for Indonesian physicians taught by first-year medical students Impact of point-of-care ultrasound training on surgical residents' confidence WHO recommendations on antenatal care for a positive pregnancy experience: ultrasound examination: highlights and key messages from the World Health Organization's 2016 global recommendations Learner portfolios and hands-on workshop to facilitate and evaluate nurses' learning in obstetric ultrasound Interdisciplinary collisions: bringing healthcare professionals together Ultrasound guidelines: emergency, point-of-care and clinical ultrasound guidelines in medicine Effectiveness of 'teaching the teachers' initiative for ultrasound training in Africa Focused maternal ultrasound by midwives in rural Zambia Training midwives to perform basic obstetric point-of-care ultrasound in rural areas using a tablet platform and mobile phone transmission technology-A WFUMB COE project Ultrasound simulators in obstetrics and gynecology: state of the art Assessment and learning of ultrasound skills in obstetrics & gynecology Introduction of a portable ultrasound unit into the health services of the Lugufu refugee camp Ultrasound training for emergency physicians-a prospective study Motivation and retention of health workers in developing countries: a systematic review Promotion and reporting of research from resource-limited settings Feasibility study of minimally trained medical students using the Rural Obstetrical Ultrasound Triage Exam (ROUTE) in rural Panama Challenges in systematic reviews of educational intervention studies Developments in medical education in response to the COVID-19 pandemic: a rapid BEME systematic review: BEME Guide No. 63 Kirkpatrick's levels and education evidence Has the new Kirkpatrick generation built a better hammer for our evaluation toolbox? Tteaching point-of-care ultrasound in medicine: a scoping review Point-ofcare ultrasound in general practice: a systematic review The path to ultrasound proficiency: a systematic review of ultrasound education and training programmes for junior medical practitioners Commentary: can a Quasi-experimental Design Be a Better Idea than an Experimental One? Integrating heterogeneous pieces of evidence in systematic reviews Obstetric ultrasound: where are we and where are we going? Ultrasonography We thank the Hospital Research Foundation for their funding and support of our research-The Healthy Newborn Project, which has provided outreach antenatal PoCUS training for remote Australian clinicians.