key: cord-0693658-ty0fgant authors: Edmond, M.; Wong, C.; Chuang, S.K. title: Evaluation of sentinel surveillance system for monitoring hand, foot and mouth disease in Hong Kong date: 2011-10-28 journal: Public Health DOI: 10.1016/j.puhe.2011.09.002 sha: 63623d3ca9054b31d5a336d6f5c421b1b8503de1 doc_id: 693658 cord_uid: ty0fgant OBJECTIVES: A sentinel surveillance system (SSS) was set up in Hong Kong to monitor hand, foot and mouth disease (HFMD) trends. This evaluation assessed the performance of the SSS from 2001 to 2009, and aimed to identify areas for improvement. STUDY DESIGN: A retrospective review using structured guidelines for evaluating public health surveillance systems published by the US Centers for Disease Control and Prevention. METHODS: The effectiveness of the SSS was evaluated using routine service statistics, laboratory surveillance data, and results of an acceptability survey conducted among the sentinel doctors. This information was used to assess various attributes of the SSS including simplicity, flexibility, data quality, usefulness, sensitivity, specificity, positive predictive value (PPV), representativeness, timeliness and acceptability. RESULTS: The SSS was simple and flexible with high-quality data. It correlated well with the laboratory surveillance data (P < 0.001) and facilitated early detection of community epidemics. It helped to identify seasonal trends and high-risk groups. Specificity was high (83.4–88.5%), while sensitivity and PPV were borderline satisfactory (38.4–56.8%). The sentinel clinics were representative of the population distribution. The SSS was acceptable to the sentinel doctors, but 17.9–28.2% of them had delays in reporting. CONCLUSIONS: The SSS is effective for monitoring HFMD trends in Hong Kong, and is useful for initiating preventive measures. Hand, foot and mouth disease (HFMD) is a common condition, especially in children <5 years of age. 1 A seroprevalence study in Singapore showed that half of children aged 5 years had antibody to enterovirus (EV) 71, which is a commonly identified virus causing HFMD. 1 Although most patients present with mild illness including fever, skin vesicles or rash, some children may suffer from severe complications such as meningitis, encephalitis, acute flaccid paralysis, myocariditis, pulmonary oedema or even death. 2e7 Since the 1970s, HFMD epidemics have been reported in Bulgaria, Hungary, the UK, the USA, Australia and India, with significant numbers of cases of aseptic meningitis, brain stem encephalitis and death. 8e11 From 1997 to 2000, areas in South East Asia including Malaysia, Taiwan, Singapore and Japan also experienced HFMD epidemics. 1,12e15 The largest reported outbreak occurred in Taiwan in 1998, accounting for >100,000 cases of HFMD, 400 severe illnesses and 78 deaths. 13 The potential public health impact of HFMD in Hong Kong has captured much attention. In 2008, another epidemic affected South East Asia, including Mainland China, Taiwan, Singapore and Hong Kong. 16e19 Apart from mortality and morbidity, outbreaks occurring in school settings caused significant social disturbance to children, their families and teachers. In large school outbreaks of HFMD or outbreaks associated with EV71 in Hong Kong and Singapore, the Health Authority may have to suspend classes or close schools in order to prevent disease transmission. 18, 20 Unfortunately, there is currently no effective vaccination or chemoprophylaxis for the prevention of HFMD. When large community outbreaks occur, public health actions including increasing public awareness, vigilance of personal and environmental hygiene, and social distancing may be necessary to stop disease transmission. A sensitive surveillance system is therefore essential for early detection of community outbreaks in order to guide public health actions. In Hong Kong, HFMD was added to the sentinel surveillance system (SSS) in 1998. The main objectives were to monitor HFMD trends in Hong Kong and to enable early detection of community epidemics, which in turn helps to initiate timely preventive measures. The SSS includes over 100 sentinel clinics, including approximately 40 general practitioners (GPs) and 60 general outpatient clinics (GOPCs). GPs are doctors who practice in the private sector while GOPCs are public outpatient clinics. They are all distributed throughout the territory. Each week, the sentinel doctors record the number of patients clinically diagnosed with HFMD, with age group breakdown, and the total number of patients seen. Patients with clinical features of HFMD are characterized by having fever, small discrete papulovesicular lesions in the oral cavity, and sparse, grayish vesicles surrounded by an erythematous base on the hands or feet. These data, in aggregated format, are sent to the Department of Health (DH) by facsimile or e-mail using a standardized surveillance record form. A weekly summary report is disseminated to all participating doctors. The authors evaluated the performance of the SSS in order to identify areas for improvement. The SSS was assessed using a structured framework, 'Updated Guidelines for Evaluating Public Health Surveillance Systems' published by the US Centers for Disease Control and Prevention. 21 The main quality attributes included usefulness, sensitivity, specificity, positive predictive value (PPV), data quality, timeliness, representativeness, acceptability, simplicity and flexibility. The SSS data from 2001 to 2009 were used to assess its performance. The data were analysed using Excel 2007 (Microsoft, Redmond, USA) and Statistical Package for the Social Sciences Version 14.0 (SPSS Inc., Chicago, IL, USA). To assess the usefulness of the SSS, the HFMD consultation rate from the SSS was examined to determine whether it correlated with the laboratory surveillance data of the Public Health Laboratory Centre (PHLC) of the DH. The HFMD consultation rate was calculated by dividing the number of patients diagnosed with HFMD by the total number of patients seen. The positive viral culture rate was calculated by dividing the weekly number of specimens positive for enterovirus by the weekly total number of specimens performed. The enteroviruses included Coxsackie A and B, EV71, EV68, echovirus and some untyped enterovirus. Coxsackie A24 was excluded because it was associated with acute conjunctivitis instead of HFMD. The Pearson correlation coefficient of the weekly consultation rate with the positive viral culture rate was calculated. The HFMD consultation rate from the SSS was plotted on a monthly basis to document seasonal trends. HFMD cases were broken down by age group to determine the most susceptible group. Sensitivity, specificity and PPV were calculated by comparing the weekly HFMD consultation rate from the SSS with the weekly viral culture rate from the PHLC. A key step in these calculations was determining the epidemic and non-epidemic weeks by setting a threshold level. When the consultation rate or viral culture rate was above the threshold in a particular week, it was considered to be an epidemic week. Conversely, when the consultation rate or viral culture rate was below the threshold in a particular week, it was considered to be a nonepidemic week. Using the control chart method, the threshold level was calculated as two standard deviations (SD) above the 'baseline level'. The 'baseline level' was the mean consultation rate or viral culture rate of the study period, after excluding extreme values above the 80th percentile. The respective epidemic and non-epidemic weeks for the consultation rate and viral culture rate were then determined, and sensitivity, specificity and PPV were calculated. Sensitivity analysis was conducted by varying the 'baseline level' and threshold level. The surveillance record forms submitted by the sentinel doctors were assessed for data quality. The items 'Total number of patients diagnosed with HFMD each day' and 'Total number of patients seen each day' were checked for data completeness. The record forms were also examined for calculation error and illegible hand writing. Five weeks from each year between 2003 and 2009 were selected at random, and all record forms selected were reviewed individually for data quality. The delay in reporting (timeliness) was examined by calculating the percentage of sentinel doctors who submitted the report later than the prescribed time frame (every Tuesday). The representativeness of the HFMD SSS was assessed by comparing the geographical distribution of the sentinel sites with the population distribution obtained from the Census and Statistics Department. 22 The expected numbers of p u b l i c h e a l t h 1 2 5 ( 2 0 1 1 ) 7 7 7 e7 8 3 sentinel GPs/GOPCs in the 18 districts were compared with the actual numbers. Acceptability was reflected by the willingness of the sentinel doctors to participate in the SSS by reporting the requested information to the DH each week. An acceptability survey was also conducted in June 2006. A questionnaire was sent to all sentinel GPs and GOPCs to collect their views on the SSS. The survey also examined the opinions of sentinel doctors regarding the simplicity of the SSS. The flexibility of the SSS was assessed in terms of its adaptability to change in data provider, organization structure, application of new technology, and advances in data analysis. The time series plots of the HFMD consultation rate and the positive viral culture rate of enterovirus are shown in Figs. 1 and 2. The consultation rates of both GPs and GOPCs were found to be significantly correlated with the viral culture rate from the PHLC, with Pearson correlation coefficients of 0.60 and 0.48, respectively (both P < 0.001). The correlation was higher for GPs than GOPCs. Figs. 1 and 2 show a consistent summer peak of HFMD from May to July, except in 2003 and 2009. A severe acute respiratory syndrome epidemic and an influenza pandemic H1N1 occurred in 2003 and 2009, respectively, so the decrease in HFMD disease activity in 2003 and 2009 is likely to be associated with the territory-wide hygiene measures implemented at these times. 23, 24 The seasonal epidemic was more consistent among GPs than GOPCs. Children aged 1e5 years were found to be most susceptible to HFMD, and 94.8% and 79.3% of cases occurred among children aged <12 years in the GP and GOPC groups, respectively (Fig. 3) . Using the control chart method, the epidemic threshold was 1.41 per 1000 consultations for GPs and 0.12 per 1000 consultations for GOPCs ( Figs. 1 and 2) . The epidemic threshold for the viral culture rate was 1.69%. Sensitivity, specificity and PPV for GPs were 48.2, 88.5 and 56.8, respectively, while the corresponding values for GOPCs were 38.4, 83.4 and 42.2 (Table 1) . Overall performance of GPs and GOPCs was fairly comparable. Both groups had high specificity, but sensitivity and PPV were borderline satisfactory. Sensitivity analysis revealed that varying the percentile to exclude the extreme values and SDs would affect the sensitivity, specificity and PPV calculations ( Table 1) . For example, using 3 SD instead of 2 SD would result in lower sensitivity and PPV, but slightly higher specificity for both GPs and GOPCs. Using the present parameters (80th percentile and 2 SD) yielded fairly balanced results for both groups. In total, 3303 reporting forms were examined. The data required to calculate the consultation rate were largely complete. The completion rate for 'Total no. of HFMD patients' was 100% for both GPs and GOPCs, while completion rates for 'Total no. of patients seen' were 97.7% and 99.7% for GPs and GOPCs, respectively. Only a minority of providers (1.0% among GPs and 1.1% among GOPCs) had illegible hand writing and required clarification. A small percentage of the figures (1.3% among GPs and 1.2% among GOPCs) were calculated erroneously. The geographical distributions of the sentinel GPs and GOPCs are shown in Table 2 . The actual distribution of sentinel GP and GOPC sites matched the expected sites fairly well, apart from an over-representation of GPs in Yau Tsim Mong District, an over-representation of GOPCs in the Island District, and an under-representation of GPs in the Shatin District. Only 80.5% and 91.0% of GPs and GOPCs, respectively, met the target of submitting the surveillance record form every Tuesday. The remaining sentinel doctors took a further 1e2 days after they received the reminder from the DH staff. The overall participation rate was 83.3% and 93.3% among GPs and GOPCs, respectively. GOPCs had a consistently higher participation rate throughout the study period. In addition, the high degree of data completeness also reflected that the SSS was acceptable to the sentinel doctors. The overall response rate of the acceptability survey was 87.3% (89/102), with a slightly higher rate among GOPCs (89.1%, 57/64) than GPs (81.6%, 31/38). On the whole, 65.9% of the participating doctors were satisfied or highly satisfied with the SSS (Table 3) . Both GPs and GOPCs were satisfied with the case definition, surveillance record form and method of returning the surveillance record. Compared with GPs, GOPCs showed consistent lower satisfaction with all aspects, especially the timeliness and content of the feedback. The case definition was clinical without any need to take a specimen. Only aggregated data were required without patients' personal information. Sentinel doctors were required to send the surveillance record form once every week by facsimile or e-mail. In the survey, 93.6% of the sentinel doctors agreed or strongly agreed that the case definition was simple. Only 17.7% regarded the notification form as tedious to complete. Approximately 88% of the doctors agreed or strongly agreed that the method for returning the form was simple. Overall, the system was regarded as relatively simple. During the past few years, there have been significant changes related to the HFMD SSS. SSSs have been used to monitor various infectious diseases in the community in different parts of the world. 25e28 GPs are often the first point of entry of the patients into the medical system. The collection of simple data by GPs enables disease activity to be monitored at an early phase, compared with other data such as laboratory tests or hospital discharge diagnoses. In Taiwan, Japan and Malaysia, SSSs have been implemented to monitor HFMD in the community in order to allow the health authorities to issue early warning of seasonal activity, detect abnormal outbreaks and assess the impact of public health control measures. Surveillance is an ongoing activity and programme evaluation is prudent for health authorities to demonstrate accountability and identify room for improvement. This is the first comprehensive evaluation of the HFMD SSS in Hong Kong since its establishment. The overall performance of this system was considered to be reasonably good. It has fulfilled its main objectives to monitor HFMD trends and to assist with the detection of community epidemics. Few evaluation reports from other countries were found in the literature for comparison. However, one published report from Japan 29 reported sensitivity, specificity and PVP of the Japanese SSS of 59.7%, 98.9% and 40.1%, respectively. Compared with the Japanese counterpart, the Hong Kong HFMD SSS had lower sensitivity and specificity but a higher PPV. In calculating sensitivity, specificity and PPV, the laboratory data were considered to be the gold standard to reflect the true viral activity in the community. The PHLC is the only public health laboratory in Hong Kong and receives hundreds of specimens from major public hospitals, GOPCs and GPs every week. From 2001 to 2009, an average of >37,000 specimens were tested for enterovirus each year by the PHLC. Given the representativeness and large scope of service, the GPs, general practitioners; GOPCs, general outpatient clinics. p u b l i c h e a l t h 1 2 5 ( 2 0 1 1 ) 7 7 7 e7 8 3 laboratory data were considered to be a good reflection of viral activity in the community. There is no universally agreed method for setting the epidemic threshold level for HFMD surveillance. In this evaluation, the control chart method was used. 30, 31 Such a method has been used both locally and internationally for public health surveillance, including SSS for influenza. 32, 33 The control chart is a relatively simple and widely used method 30, 31 ; one advantage of this method is that it gives persistent epidemic signals after the epidemic starts, continuing until the outbreak ends. The sensitivity analysis performed showed that this evaluation has obtained relatively balanced values for sensitivity, specificity and PPV. The participation rate among GOPCs was consistently higher than that among GPs. However, in the acceptability survey, GOPCs were less satisfied with all aspects than GPs. This could be partly explained by voluntary participation by GPs as those GPs who are not satisfied with the SSS are likely to stop participating. Regarding the study results, prompt actions have been taken to enhance the representativeness of the GP and GOPC sites by recruiting more sentinel partners in the underrepresented areas. To ensure high data completeness and validity, and improve the timeliness of reporting, an online electronic system is being developed to replace the existing reporting format. Online reporting would also enhance the efficiency of data processing and dissemination of information. It is important to note that the DH should not depend solely on the SSS to monitor HFMD trends in Hong Kong. The SSS could be supplemented by other surveillance systems, such as hospital admission data and syndromic surveillance in childcare centres. Further intelligence is needed to integrate the findings detected from these various systems. For accurate assessment of data quality, case record review or practice visits would be necessary to ensure that clinical presentations meet the case definition. However, this would require additional resources and could compromise the rapport between the sentinel doctors and the Health Authority, since review of case records or practice visits could be perceived as auditing their personal performance. In assessing timeliness, the time lapse between disease onset and date of reporting to the SSS should be assessed. 34 This would help to assess the time lag between the occurrence of the outbreak and the time when public health actions were initiated. However, no such information was available from the existing data collection system. In assessing representativeness, the demographics of patients attending the sentinel clinics could be compared with the general population. However, as these data were not available, such analysis could not be undertaken. The overall performance of the HFMD SSS in Hong Kong was considered to be reasonably good. The HFMD consultation rate was a good reflection of viral activities in the community, as evidenced by its significant correlation with the laboratory surveillance data. The SSS has provided useful information for understanding local epidemiology in terms of seasonal trends and high-risk groups. It allows early detection of increased disease activity in the community, and enabling early health warnings to be made to the public and health professionals. The results indicate that the SSS is simple, flexible, representative and acceptable by the participating doctors. However, its performance in terms of sensitivity and PPV needs further improvement. Seroepidemiology of human enterovirus 71 Deaths among children during an outbreak of hand, foot, and mouth disease e Taiwan, Republic of China Hand, foot and mouth disease in Singapore: a comparison of fatal and non-fatal cases Clinical characteristics of an outbreak of hand, foot and mouth disease in Singapore Abu Bakar S. Deaths in children during an outbreak of hand, foot and mouth disease in Peninsular Malaysia e clinical and pathological characteristics Cardiac manifestations of fatal enterovirus infection during the 1998 outbreak in Taiwan Enterovirus 71: the virus, its infections and outbreaks Epidemiological, virological and clinical features of an epidemic of hand, foot, and mouth disease in England and Wales Hand-foot-and-mouth disease in Calicut Enterovirus surveillance e United States Outbreak of enterovirus 71 infection in Victoria Identification of enterovirus 71 isolates from an outbreak of hand, foot and mouth disease (HFMD) with fatal cases of encephalomyelitis in Malaysia An epidemic of enterovirus 71 infection in Taiwan. Taiwan Enterovirus Epidemic Working Group Epidemic hand, foot and mouth disease caused by human enterovirus 71 Outbreak of central nervous system disease associated with hand, foot, and mouth disease in Japan during the summer of 2000: detection and molecular epidemiology of enterovirus 71 Appearance of mosaic enterovirus 71 in the 2008 outbreak of China Reemergence of enterovirus 71 in 2008 in Taiwan: dynamics of genetic and antigenic evolution from 1998 to EV71-associated epidemic hand, foot and mouth disease in Singapore Epidemic of enterovirus 71 in 2008 e its public health implication to Hong Kong Press release on public urged to prevent hand, foot and mouth disease. Hong Kong: Centre for Health Protection, Department of Health Updated guidelines for evaluating public health surveillance systems Population and household statistics analyzed by district council district. Hong Kong: Census and Statistics Department Respiratory infections during SARS outbreak Effects of public health interventions in reducing transmission of hand, foot, and mouth disease Sentinel surveillance for human enterovirus 71 in Sarawak, Malaysia: lessons from the first 7 years Evaluation of a sentinel surveillance system for influenza An evaluation of the Australian Sentinel Practice Research Network (ASPREN) surveillance for influenza-like illness Using sentinel surveillance to monitor effectiveness of influenza vaccine is feasible: a pilot study in Denmark Detection of epidemics in their early stage through infectious disease surveillance Part II: basic methods of statistical process control and capability analysis Statistical quality control methods in infection control and hospital epidemiology, Part I: introduction and basic theory A review of the Hong Kong influenza surveillance system The control chart: an epidemiological tool for public health monitoring Evaluation of reporting timeliness of public health surveillance systems for infectious diseases The authors would like to thank all the sentinel doctors, and medical, nursing and research staff in the Epidemiology Section, Surveillance and Epidemiology Branch, Centre for Health Protection, Department of Health, HKSAR, for contributing the epidemiological HFMD information.r e f e r e n c e s None sought. None declared. None declared.