key: cord-0854776-wqcjoxjq
authors: Arieta-Miranda, Jessica; Alcaychahua, Abad Salcedo; Santos, Gary Pereda; Sevillano, Gustavo Chávez; Verástegui, Rosa Lara; Victorio, Daniel Blanco; Ramos, Gilmer Torres
title: Quality assessment of Clinical Practice Guidelines for the management of paediatric dental emergencies applicable to the COVID-19 pandemic, using the AGREE II instrument. A Systematic Review
date: 2020-12-09
journal: Heliyon
DOI: 10.1016/j.heliyon.2020.e05612
sha: 8d6e2615ea439ef1b90a31d96d1c70e6b41f9443
doc_id: 854776
cord_uid: wqcjoxjq

OBJECTIVE: To assess the quality of Clinical Practice Guidelines (CPG) related to the management of paediatric dental emergencies applicable to the COVID-19 pandemic, through the use of the measuring instrument AGREE II (Appraisal of Guidelines for Research and Evaluation in Europe). SOURCES AND DATA COLLECTION: A rigurous online search of CPG was accomplished among the main CPG compilers: National Institute for Health and Care Excellence (NICE), National Guideline Clearinghouse, Agency for Healthcare, Research and Quality (AHRQ), Andalusian Health Technology Assessment Department (AETSA), American Academy of Family Physicians, Tripdatabase. Furthermore, because of the need to identify CPG that meet the inclusion criteria, a manual search, among the main national and international dental organizations as well as recognized web sites, was also accomplished. SELECTION OF RESEARCH STUDIES: All of the guides focused on paediatric dental emergencies, available in the database and “gray” literature, and published between 2000 and 2020 (applicable to COVID-19 pandemic) were included without any language restrictions. The CPG that did not contain the full paper or were addressed to adults or children with special needs, were excluded from the selection. The evaluation of the CPG, independently included, were achieved by four (04) experts by using AGREE II. RESULTS: Five (05) out of twenty-three (23) selected CPG, were classified as “acceptable” according to AGREE II. These five guides were evaluated to determine their “Recommendation degree”. Only one (01) CPG “Guía Clínica AUGE de Urgencias Odontológicas Ambulatorias-Chile, 2011” reached a score of 75%, the highest among the other guides (5 domains with a score >= 60%, including the domain III “Rigour of Development”) to be considered as a “highly recommended” CPG. CONCLUSIONS: According to the quality assessment and recommendation degrees criteria from AGREE II, high, middle and low quality CPG were identified. Only one CPG reached a score of 75%, to be classified as “highly recommended”. Therefore, it is suggested that the existing CPG updates and future CPG use the available tools and methodologies during their elaboration, in order to guarantee their quality. CLINICAL SIGNIFICANCE: High quality CPG for the management of dental emergencies are designed to support dental health professionals in decision-making to adopt specific dental procedures in the current COVID-19 pandemic. As a matter of fact, these CPG might contribute to reduce the risk of transmission, in case of fresh outbreak of the illness. Likewise, they might help to determine which cases warrant medical attention in centres with special facilities for COVID-19.

Global public health is currently undergoing a significant crisis due to the outbreak and spread of the new SARS-Cov-2 (Severe Acute Respiratory Syndrome), originally reported in the city of Wuhan, China in December 2019 (1). Renamed as COVID-19 by the World Health Organization (WHO) (2) and categorized as Pandemic on March 11 th , 2020 (3) is characterized by showing symptoms such as fever, cough, fatigue, myalgia, dyspnoea and in some cases diarrhoea. Patients with co-morbidities (hypertension, diabetes, obesity) and the elderly constitute the main population at risk. On the other hand, the majority of COVID-19 paediatric patients exhibit mild symptoms, no fever nor pneumoniae. During the first phase of the pandemic, there were not severe cases or deceases reported among paediatric patients (4). As a matter of fact, a study that analysed 44,672 confirmed cases in China since February 2020, reported that only 416 cases (0.9%) were patients under 10 years old (5). By June 2020, only two (02) deaths in children testing positive for COVID-19 were reported in China and no deaths, in Italy (the two countries with more confirmed cases). Nevertheless, with the progressive increase of confirmed cases in the adult population, the number of paediatric infections also increased concomitantly.

In general, any patient (either adult or paediatric) ought to be considered as potential COVID-19 carrier (7). A large percentage of COVID-19 confirmed cases are asymptomatic or have mild symptoms (7-9). Wang et. al. identified some risk factors associated with the virus transmission during dental treatments in paediatric patients i.e. the droplets emitted during sneezing and the aerosols generated by the high-speed piece (10). The American Dental Association (ADA) and the Centres of MediCare and MedicAid Services (CMS), recommend that during the pandemic, dental procedures should be restricted only to emergencies so as to reduce the risk of virus spread among patients and dental staff (11, 12).

Life-threatening dental emergencies demand immediate treatment to stop continuous tissue bleeding, relieve pain or treat a severe infection. On the other hand, urgent dental care is focused on the management of conditions that require immediate attention to relieve moderate-severe pain, reduce the risk of infection and alleviate the patient burden in emergency centres (12).

The most frequent emergencies in children are: the reversible pulpitis, irreversible pulpitis (13), acute apical periodontitis, facial cellulitis, facial abscess and dental trauma (14) . Half of them are characterised for presenting sequelae related to dental caries (15).

The management of dental emergencies has become increasingly important due to the COVID-19 pandemic. The constant search for reliable scientific evidence, that allows solving clinical doubts and identifying suitable treatments, is more frequent in this context. As a result, it is necessary for the clinical dentists to have access to high quality CPG, which enable them to promote and recommend practical solutions to clinical doubts regarding efficient treatments in their daily routines. CPG can represent one part or the determining pillar in the elaboration of health policies. Therefore the preparation of CPG requires rigorous methodologies to ensure its quality. However, we need to take into consideration that not all CPG meet the basic requirements.

AGREE II is a reliable tool which assesses the methodological rigour and transparency used in the CPG preparation (16). After having used this practical tool, it was shown that some CPG did not present an adequate structure, either due to a poor quality elaboration or a lack of updated scientific evidence (17). The quality of a CPG is defined as the confidence that potential biases (in the development of the guide) have been adequately pointed out and that the recommendations are valid, both internally and externally (18).

The purpose of this systematic review is to evaluate the quality of the CPG for the management of paediatric dental emergencies, published in the period 2000-2020 and applicable to the context of pandemic, by using the AGREE II tool. Additionally, to provide relevant information to those researchers and/or institutions responsible for the development of CPG worldwide.

The present systematic review was registered in PROSPERO (registry number: CRD42020195678) and detailed methods are available in the published protocol (19). The systematic review is reported according to PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) and to a checklist available as supplementary material (20). The protocol was focused on the strategic search for published and available CPG. The questions, asked for the present review, were:

• How many CPG for the management of paediatric dental urgencies and emergencies, are available and applicable to the COVID-19 pandemic?

• Which high quality CPG could be recommended?

Details regarding the search are visualised in Fig.1 

Inclusion criteria:

• CPG published in databases and gray literature, aimed at dental emergencies in children, and applicable to the current context of the COVID-19 pandemic.

• CPG written in any language.* • CPG published between 2000 and 2020.

* The native language of the evaluators is Spanish, with basic knowledge of English, Portuguese and Italian. For other languages, translations tools were used (https://www.enago.com/ar/ y https://oxfordediting.com/ ). The CPG, included in this study, and the AGREE II instrument were translated into Spanish. Table 1 Exclusion criteria:

• GPC that do not contain full text.

• Previous versions of GPC • GPC aimed at children with special abilities.

• CPG aimed at adults.

Ethical approval and informed consent were not necessary since no human beings were involved.

An online search was carried out among the main CPG compilers: National Institute for Health and Clinical Excellence (NICE), National Guidelines Clearinghouse, Agency for Health Research and Quality (AHRQ), Andalusian Health Technology Assessment Department (AETSA), American Academy of Family Physicians and Tripdatabase. The key terms used for this search were: (Guide practice dental emergency children), (guidelines emergency dental), "urgency dental", (guidelines dental urgency emergency children); associated with the boolean operators: "AND" and "OR". This search was carried out from 30 th of April to 30 th of July, 2020.

Additionally, a manual search was carried out for CPG that met the inclusion criteria and were available on the websites of various national and international dental organizations.

Initially, 5070 articles and CPG were collected. After the first filter, carried out by the reviewer "ASA", and the subsequent examination carried out by the reviewers "JAM" and "GPS", 5026 guides were excluded as they did not contain eligible aspects in the title and/or abstract. As a result, 44 CPG were selected for further content evaluation. Subsequently, a videoconference with all the reviewers ("JAM", "ASA", "GPS", "GCHS", "RLV" and "GTR") was held to support the inclusion or exclusion of the assigned documents. Any disagreement among the reviewers was solved with further discussion and in cases where consensus was not reached, the judgement of an expert reviewer ("GTR") was decisive. Eventually, only 23 papers met all the selection criteria. These were processed for data extraction and quality evaluation. Table 1 2

The 23 selected CPG were assigned to the reviewer "JAM" to sort them according to their characteristics (year of publication, origin, type of guide -Expert opinion, Consensus or Based on evidence) and to classify them according to their specialty (dental emergencies). Table 2 The evaluation of the quality and recommendations was carried out by using AGREE II.

AGREE II, an instrument for evaluating research guidelines, is commonly used to evaluate the quality of the information of the studies (components of the preparation and documentation of the process) and the recommendation degrees (18).

The 23 selected guides were independently reviewed and five (05) CPG were classified as "acceptable". After this result, all of the reviewers were calibrated (trained) in the use of this tool by an expert reviewer ("GTR") via online (Cisco Webex and Zoom).

An instructive guide was used for this training. Subsequently, the evaluation of all of the five CPG was completed by each reviewer, who presented their data independently. The data were assessed for statistical analysis by using Cohen Kappa coefficient. The objective: to determine the degree of concordance among the reviewers.

During the calibration/training period, any discrepancy among the reviewers was discussed until consensus was reached and the Cohen Kappa coefficient (0.8) was obtained. All the data were compiled in a single table, in alphabetical order according to title, country of origin, organization that prepared it, year of publication etc. In addition, the evaluation scores achieved by all the CPG, according to the 6 domains of AGREE II, were included.

The Recommendation Degrees (RD) of the selected CPG were determined by using the following strategy: The guidelines could be classified as "Recommended" (R) (when at least 3 domains are >= 60%) "Recommended with Modification" (RM) (>30% to <60%) and "Not Recommended" (NR) (when at least 3 domains are =< 30%) (22). Table 4 The domain scores were calculated by adding the scores for each item in that domain and then scaling the total, as a percentage of the "Maximum Possible Score" for that domain. This was carried out by using the following mathematical operation: J o u r n a l P r e -p r o o f

The statistical analysis was carried out by using Stata V.15 software (Stata Corporation, College Station, Texas, USA).

The concordance degree among the reviewers for the eligibility of the guideline was calculated by using Cohen Kappa Coefficient, a qualitative assessment. (Cerda, L et al. 2008) (23). The Kappa concordance coefficient among the 4 reviewers was k= 0.82. In addition to this, "Generalization of weighted Kappa coefficient" for more than two observers was necessary. After downloading the command Kappa2 (through the syntaxes "findit kappa2") the expression "kappa2 OB01-OB04, wgt(w2)" was executed. All this, in order to obtain the global result.

The characteristics of the CPG were summarised by using descriptive statistics. The general scores of the included CPG are presented for each AGREE II domain through summary measures (mean, median and standard deviation) and Shapiro-Wilk p.

From the analysis of the 23 CPG selected according to the inclusion criteria, the following results were obtained:

A gradual increase in the number of CPG publications was observed over the years. Of the total selected CPG, 13% of them were published between 2000 and 2010, 26.1% were published between 2011 and 2015; and 60.9%, between 2016 and 2020.

21.7% of the selected CPG came from Europe, 4.3% from Asia and 73.8% from America. On the other hand, five (05) were considered specific for paediatric dental emergencies, and applicable to this current context of pandemic. In addition, regarding the method of elaboration, 34.8% (08 CPG) of the selected guides were based on expert opinion, 47.8% (11 CPG) were created with consensus, and 17.4% (04 CPG), based on evidence. Table 2 The global evaluation of the 23 selected CPG, revealed that 78.3% (18 CPG) were "Not recommended" (NR) due to the lack of methodological rigour. Moreover, 21.7% (05 CPG) were identified to have acceptable quality and were categorised as "Recommended" (R) or "Recommended with Modification" (RM). On the other hand, regarding the Evaluation of the domains" in all the 23 CPG, it was shown that Domain I "Scope and Purpose" was the only one obtaining the highest average score (39.3%) and the Domains III and V (Rigour of Development and Applicability, respectively) obtained the lowest scores. The Shapiro-Wilk statistical test showed that Domains III, IV and VI (in all the 23 selected CPG) presented a statistically significant difference (p<0.01) Table 3 After evaluating all the domains in the 5 CPG classified as "acceptable" for this review, the following results were observed: The domain that achieved the highest average score was Domain I "Scope and Purpose" (76%) and the one with lowest score was Domain V "Applicability" (24%). On the other hand, Domain III, corresponding to "Rigour of development", ranged from 38% to 65% with an average score of 44.4%. Furthermore, according to Shapiro-Wilk, it is observed that Domain III, presented p<0.01, indicating that there is a statistically significant difference among the 5 CPG with respect to this domain. The summary measures (Mean, Median and Standard Deviation) were also obtained in each of the domains. Table 4 The quality evaluation of the 5 CPG using the AGREE II domains assessment, revealed that there was no specific CPG for the management of paediatric dental emergencies. However, when the objective of this topic was rigorously evaluated in these 5 CPG, it was observed that only one (AUGE Clinical Guide for Ambulatory dental emergencies -Chile, 2011) (24) reached the highest score (75%). This document exhibited 5 domains with a score >= 60%, including Domain III, and it was considered as "Recommended", while the other four guides reached an average score of 43.5%. The results of the evaluation of these 4 guidelines were: Scotland, 2013 (25) that obtained 50%; Brazil, 2013 (26) that obtained 45%, Sweden, 2012 (27) that obtained 44% and Italy, 2012 (28), with 35%. All of them presented 1 to 2 domains with a score >=60%. Furthermore, they all presented 1 to 2 domains with a score <= 30%. As a result, these four CPG were categorised as "Recommended with Modification". Table 4 4

These CPG for the management of paediatric dental emergencies have gained big importance during this COVID-19 pandemic since the American Dental Association (ADA) and the Centres for Medicare and Medicaid Services (CMS) (11, 12) recommended prioritising dental emergencies to avoid the spread of SARS COV-2 among patients and oral health professionals.

Considering the current global situation we are undergoing, we planned to carry out this systematic review in order to find CPG, based on scientific evidence, with high methodological quality and applicable to this COVID-19 context.

Using the Agree II tool, we accomplished quality evaluations on all the CPG available online. The results of this review indicated that the general quality of the CPG for paediatric dental emergencies is mainly medium or high. These guides may be recommended with modification since the general scores are less than 50% for 3 out of the 6 AGREE II domains. As a matter of fact, we consider that it is still necessary to improve the presentations of the CPG, especially on the "Rigour of development", "Applicability" and "Editorial independence".

In addition, it is worth mentioning that although we used a search strategy for CPG, we did not find CPG, based of scientific evidence, including the title Paediatric Dental Emergency Management during COVID-19 pandemic. We strongly believe this is due to the recent SARS COV-2 outbreak. Consequently, we decided to include all the CPG that were related to the management of Dental Emergencies and with this, we were able to find provisional or preliminary CPG with special focus on the current context. One of these, was elaborated by the ADA though it was not prepared with the methodological rigour required for high quality CPG. Table 3 . Likewise, due to the little information regarding our objectives in this study, we decided to include the evaluation of protocols as they play an important role on CPG elaboration.

On the other hand, it is also relevant to clarify that, only one (01) of the five (05) selected CPG was exclusively made for children ("Reference Manual for Clinical Procedures in Paediatric Dentistry" ALOP, Brazil, 2013), while two (02) of these five, were aimed at primary dentition in relation to dentoalveolar trauma ("Linee Guida Nazionali per la Prevenzione e la Gestione Clinica dei Traumi Dentali negli individui in eta' evolutiva", Italy 2012) and ("International Association of Dental Traumatology Guidelines for the Management of Traumatic dental injuries: injuries in the primary dentitition", Sweden 2012). Likewise, "Scotland 2013" guide is aimed at management of acute dental problems. Finally, the "Guideline for healthcare professionals" and the "Chilean CPG 2011" addressed to children and adolescents.

The CPG classification in this review showed that 17.4% of CPG are "Evidence-based", while 34.8% are based on "Expert opinion", and 47.8% are "Based on Consensus". It should be noticed that a CPG prepared by consensus, represents the collective opinion or suggestions of a group (14). In contrast, a CPG, made with scientific evidence, provides recommendations from a systematic review on a specific health issue and the possible benefits or disadvantages about the different treatment options (18). Although both type of guidance documents contain suggestions for improving patients care and they both show their potential risks of bias (18, 19) . The CPG prepared by expert opinion were excluded since their methodological quality was poor (they did not have scientific rigour), the risk of bias was high and also the conflict of interest was considerable (18).

The online training, taken by the reviewers and directed by an expert (GTR), has also been reported by other authors (30, 31) and it has gained big importance for being an optimal way to guide, analyse and investigate through virtual platforms, during this pandemic.

The specific criteria for stablishing the limits of evaluation for general quality in CPG, vary widely among the different studies (32). In our study, it was considered to assess the CPG quality in a domain-specific way, with a limit of 60% to discern whether the CPG present high, medium or low quality. This strategy was adopted, based on previous studies.

In this regard, Hoffmann-Eber (2018) (22), reported that global or general evaluations of CPG, using AGREE II, are not frequently performed by CPG evaluators. This study recommends making more objective evaluations by weighing individual domains of AGREE II and considering Domains III and V as key factors on the results. Based on these studies, we established 60% as the cut-off point to discern high, medium or low quality guidelines (in the global evaluation and specific evaluation by domain). In addition, it is worth mentioning that the average score for domain III "Rigour of development" should be greater than or equal to 60%, to be considered as high quality.

In a parallel analysis, if our study only adopted the global evaluation strategy, also used by O'Donnell et al. (2020) (29) , the result would show that 2 CPGs (Chile, 2011 (75%) and Scotland, 2013 (50%)) would be classified as high quality CPG. In contrast to this strategy, after the specific evaluation (domains) and assessment for recommendation degrees with the established limit of 60% including domain III, a more rigorous result was observed (only 1 CPG (Chile, 2011) exhibited high quality and therefore, it was classified as recommended (R). Nevertheless, one of the main drawbacks in the application of this recommended CPG around the world, is the original language. As a matter of fact, its content is written in Spanish.

The evaluation of the global quality of the CPG is basically represented by the average of the 6 domains. In our review, a global average score of 49.8% was found, revealing a low global average quality in all the studied CPG (Table 4) In our review, we found that the only recommended guideline that met the criteria of the AGREE II tool, was the 2011 Chilean CPG, with an overall score of 75%; the other four CPG did not exceed 60% and therefore were considered recommended with modifications (RM), due to their low rigour of development.

Some authors question the global way of evaluating the CPG, without considering the weight of each domain. In fact, they mention that this global method is not "scientific" enough (30,33), and as a consequence, we decided to carry out the independent evaluation by domains in parallel, to obtain a more rigorous result. Domain IV "Clarity of Presentation": In our study, this domain obtained an average score of 54.8%, Chile presented 85% and Brazil 67% independently. The clarity of presentation, through key recommendations, algorithms and therapeutic options that facilitate decision-making, in most of the CPG were not explicit. Clarity in the presentation was only found in the CPG of Chile and partially in the CPG of Brazil and Scotland. Other authors (30, 31, 37) , reported a global score greater than 60% in this domain. In other aspects, Burgers JS (38) conducted a comparison study between North American and European CPG and found that European CPG exhibited a better quality. In our systematic review, the best quality CPG is from South America, Chile, 2011. This is due to the scientific progress this country has had in recent years and the large investment in public health development. Although Burgers JS mentioned that most of the high-quality guides have been developed by organizations in countries with more resources and funds for research (e.g. United Kingdom, United States, Canada etc.); in our review, we were able to verify that there are good quality CPG in developing countries, such as Chile, Mexico and Brazil. These countries present attractive proposals that could be modified and translated into the universal language for their application worldwide.

The present lack of rigour in the development of CPG on dental emergencies, encourages us to develop new CPG based on high quality scientific evidence, to generate grades of recommendation aimed at the paediatric population.

The institutions in charge of elaborating CPG require a team of experts, internal and external, for the development of guides, complying with the methodological rigour.

A short-term measure is to update the high-quality CPG available and associated with the research topic of the guide to be developed. For this purpose, it is necessary to strengthen the cooperation of methodological experts, seek patients (opinions from the public) to improve the applicability of the CPG, solve financing problems and define conflicts of interest in a clear way.

The strategy used for searching CPG constitutes one of the strengths of the present study. A meticulous investigation in the different guideline compilers and governmental entities from different countries, was carried out. The manual search of CPG applicable to the current context of the COVID 19 pandemic and the gray literature, provided additional value for obtaining eligible guidelines.

The world is undergoing a dreadful pandemic and this current context forces us to seek and provide quick solutions. The development of new knowledge on CPG is necessary. The management of emergencies in this context is relevant and so are high quality guidelines.

Nevertheless, this COVID-19 pandemic also represents one of the limitations, since in this context, the administrative processes that favour the adequate preparation of high-quality CPG for the management of paediatric dental emergencies are slowed down. In addition, specific CPG regarding this topic and written in the international language (English) are not available. As a matter of fact, this aspect represents a great limitation for our review.

For future research, it would be interesting to study the relationship between the quality of the guidelines and the effectiveness of the guidelines' recommendations in different countries because the economic, social and cultural realities of each country are different.

High, medium and low quality CPG for the management of paediatric dental emergencies were found. It is necessary to pay especial attention to the AGREE II domains so as to improve the CPG quality and apply them during the COVID19 pandemic.

According to the quality evaluation criteria and recommendation degree of the AGREEII instrument, only one CPG (AUGE clinical guide for ambulatory dental emergencies-Chile, 2011) was considered a "Recommended" CPG, but applicable only among Spanish-speaking countries. It would be advisable to work on this guide, using English as an international language. J o u r n a l P r e -p r o o f 

A 2-year retrospective study of pediatric dental emergency visits at a hospital emergency center in Taiwan

A ten-year retrospective study of paediatric emergency department visits for dental conditions in Montreal, Canada

Paediatric dental emergencies: a retrospective study and a proposal for

1. AGREE II scoring system: For each domain, scores are rated out on a 7-point scale (1 = strongly disagree, 7 = strongly agree) by individual appraisers. Individual appraiser scores are summed for an overall domain score, which is then scaled to a percentage of the maximum possible score for the domain, with higher scores indicating higher quality. The six domain scores are independent and are not aggregated into a single quality score. 2. Overall evaluation of the guidelines according to domain score: High quality, when at least 3 domains are ≥60% (including domain III), it will be considered as Recommended (R).When