Inter- en intraobserver agreement bij wondbeoordeling TITLE PAGE 1. Title: ‘Validity of diagnosis of superficial infection of laparotomy wounds using digital photography: inter and intra observer agreement amongst surgeons’ 2. Running head: Digital photography for assessment of wound infection 3. Authors: Gabriëlle H. van Ramshorst1, MD, Wietske W. Vrijland2, MD, PhD, Erwin van der Harst3, MD, PhD, Wim C.J. Hop4, PhD, Dennis den Hartog1, MD, PhD, Johan F. Lange1, MD, PhD, Professor of Surgery 4. Name of departments and institutions: 1 Erasmus University Medical Center, Department of Surgery, Rotterdam, Netherlands. 2 Sint Franciscus Gasthuis, Department of Surgery, Rotterdam, Netherlands 3 Maasstad Hospital, Department of Surgery, Rotterdam 4 Erasmus University Medical Center, Department of Biostatistics, Rotterdam, Netherlands. 5. Disclaimers: none 6. Corresponding author: Gabriëlle H. van Ramshorst, Erasmus MC, Department of Surgery, Room Z-836, P.O. Box 2040, 3000 CA Rotterdam, Netherlands, email: g.vanramshorst@erasmusmc.nl, telephone: + 31 10 7034631, fax: + 31 10 7044752 7. Request for reprints: see corresponding author 8. Source(s) of support: none Digital photography for assessment of wound infection 1 mailto:g.vanramshorst@erasmusmc.nl ABSTRACT Background: The validity of digital photography for the diagnosis of wound infection is unknown. We intended to measure its validity by measuring inter and intra observer agreement. Methods: Laparotomy wounds were photographed daily in a prospective study. Four surgeons independently assessed photographs of 50 wounds opened for infection within hours after photography and of 50 normally healed wounds (n=50). Surgeons recorded presence of infection and treatment for each wound. Paired kappa values were calculated. Intra observer agreement was measured after 4-6 weeks. Results: Mean specificity with regard to infection was 97% (94-100%) and mean sensitivity was 42% (32-48%). Paired kappa-values ranged between 0.54-0.68 for infection and 0.15- 0.72 for treatment. Kappa values for intra observer agreement on infection ranged between 0.43-0.76. Conclusions: Inter and intra observer agreement on the diagnosis of superficial infection with digital photography are moderate, but specificity is high. Findings at physical examination should also be reported. KEY WORDS: wound, digital photography, wound infection, surgical site infection, inter observer agreement, intra observer agreement, abdominal, surgery. Digital photography for assessment of wound infection 2 TEXT INTRODUCTION In recent years, the use of digital photography has become increasingly popular and highlighted in literature for documentation and evaluation of wound healing progression and for its usefulness in telemedicine for diagnosis in dermatology and vascular surgery. [1-3] For chronic and burn wounds in particular, photography can be used to assess treatment results and support continuation or alteration of treatment strategy. [4] One of the most common complications of surgery is wound infection, affecting approx. 10% of all abdominal wounds. [5,6] Although the validity of digital photography has been reported for a number of indications, its validity of diagnosing infection in surgical wounds remains unclear. [3] The diagnosis infection has been based on the symptoms ‘rubor’, ‘dolor’, ‘calor’, ‘tumor’ and ‘functio laesa’ ever since the time of Hippocrates. Wound infection is diagnosed by doctors based on subjective and objective criteria and experience. The international gold standard for diagnosis of surgical wound infection is represented by the criteria for surgical site infection (SSI) as defined by the Centers for Disease Control and Prevention (CDC). [7] The surgeon’s judgment is, according to the CDC, very important for diagnosis of superficial SSI. In several studies in which wound photography was used for assessment of healed lacerations and incisions, moderate to good inter and intra observer agreement were found on wound appearance scales. [8-10] Few reports exist on agreement amongst surgeons with regard to wound assessment. In literature, kappa values for inter observer agreement on infection vary between 0.08 to 1.00 [1, 11, 12] Unfortunately, nor the absolute numbers of infected wounds, nor the levels of intra observer agreement were reported in these studies [1, 11, 12] Wirthlin). The goal of our study was to measure the degree of inter and intra observer Digital photography for assessment of wound infection 3 agreement on the diagnosis superficial infection of laparotomy wounds using digital photography, thereby assessing its validity. MATERIALS AND METHODS Between May 2007 and January 2009, a total number of 1000 patients were included in a prospective observational clinical study on surgical wound healing. After informed consent was obtained, the abdominal wound was photographed on a daily basis (including weekends and holidays) until discharge or until the 21st postoperative day using a Fujifilm (Tokyo, Japan) model Finepix S5700 digital camera (7.1 megapixels, 10x optical zoom) with standardized multi-auto focus and macrosettings. Each day, two photographs (resolution 3072 x 2304 pixels) were taken according to a standardized protocol: one of the entire abdomen from sternum to the pubic bone at a distance of approx. 40 cm, and one close-up photograph of the wound at a distance of approx. 20 cm. Photographs were downloaded onto a personal computer and saved in JPEG (Joint Photographic Expert Group) format, coded for patient, postoperative day and number of sequence. Presence of signs of infection were documented using a standard procedure and relevant data on wound infection were retrieved prospectively from hospital and nursing charts. Four gastrointestinal surgeons (A, B, C, D) with clinical experience ranging between 10-30 years independently assessed 100 randomly ordered sets of abdominal wound photographs, consisting of one overview photograph and one close-up. Fifty of these consisted of photographs of wounds that had been opened within hours on suspicion or presence of infection ánd had met the criteria for superficial SSI of the CDC. According to the CDC, the following criteria have to be met for diagnosis of a superficial wound infection (SSI) [7]: Digital photography for assessment of wound infection 4 ‘Infection occurs within 30 days after the operation and infection involves only skin or subcutaneous tissue of the incision and at least one of the following: 1. Purulent drainage, with or without laboratory confirmation, from the superficial incision. 2. Organisms isolated from an aseptically obtained culture or fluid or tissue from the superficial incision. 3. At least one of the following signs or symptoms of infection: pain or tenderness, localized swelling, redness, or heat, and superficial incision is deliberately opened by surgeon, unless incision is culture-negative. 4. Diagnosis of superficial incisional SSI by the surgeon or attending physician.’ Photographs of infected wounds were matched by postoperative day, type of incision and skin color with fifty sets of photographs of wounds that had healed without complications, verified by surveillance by means of out patient clinic visits after discharge and review of hospital charts, discharge letters and complication registration systems. Surgeons were requested to read the CDC criteria for superficial SSI and to apply these criteria if possible before all sessions. Wound pain scores (visual analogue scale ranging from 0 – no pain to 100 – worst imaginable pain) of the current and previous day, morning temperature and postoperative day were notified for each wound. All photographs were viewed on one laptop using standardized settings with the possibility to adjust the viewing screen (Toshiba A100 portable personal computer, 17 inch screen). Surgeons were requested to record for all wounds whether or not superficial infections were present and whether the wounds should be treated conservatively (i.e., remain closed) or be opened (either partially or fully). Four to six weeks after the initial sessions, all photographs were placed in a different, Digital photography for assessment of wound infection 5 random order and were reevaluated in order to measure intra observer agreement for all surgeons. Reevaluations took place in the same room at approx. the same time of day as the first evaluations. Statistical analysis was performed by calculating paired kappa values with 95% confidence intervals (CI, calculated as 1.96 ± SE) between all observers (A-B, A-C, A-D, B-C, B-D, C- D) for inter observer agreement. For each observer intra observer agreement was measured by calculating kappa values (including 95% CI). In general, kappa values of 0.80 or over are considered to represent a good level of agreement. [13] Sensitivity and specificity were calculated for each observer. RESULTS On average, abdominal wounds had been opened on the seventh postoperative day (range 3- 15). Mean specificity with regard to wound infection was 97% (94-100%) and mean sensitivity was 42% (32-48%) (Table 1). Paired kappa values with regard to wound infection varied between 0.54 and 0.68 (Table 2). Agreement on treatment (conservative or opening of the wound) was present in 76 of 100 wounds (kappa values: 0.15, 0.17, 0.20, 0.72, 0.63, 0.68). The diagnosis of wound infection was unanimous in 12 of 50 cases. In some cases, surgeons preferred not to open wounds in presence of infection. In 13 patients symptoms of infection were considered minimal by one or more surgeons, and in five cases spontaneous drainage of pus was present and further opening of wound was therefore not considered compulsory. Surgeon A was the only surgeon to report low morning temperature Digital photography for assessment of wound infection 6 as a reason for not opening infected wounds. None of the additional information given on morning temperature or wound pain scores were significantly associated with wound infection in this group of patients (all p>0.05). Kappa values for intra observer agreement varied between 0.43-0.76 for wound infection and 0.52-0.87 for wound treatment (Table 3). DISCUSSION Assessment of wounds is normally based on a combination of subjective and objective information, visual and physical information and experience. This study has demonstrated that the inter observer agreement on wound infection of laparotomy wounds is moderate amongst surgeons when using digital photography. The inter observer agreement on the treatment of wound infection is moderate, and shows high variability amongst different surgeons. Moreover, the intra observer agreement on wound infection and treatment differs amongst surgeons. This implies that wounds are possibly assessed and treated differently depending on which individual is supervising care for operated patients. Infection rates as collected in several national surveillance programmes might therefore vary between hospitals partly as a result of differences in judgment amongst doctors. [14-18] Standard protocols for the assessment of acute wounds such as ASEPSIS and the Southampton Wound Assessment Scale are time-consuming and have not been implemented widely. [14, 19, 20] Therefore, wounds are still subjected to individual surgeons or attending doctors and their experience. The predictive value of the criteria for wound infection used in the aforementioned protocols is unclear. The European Society for Wound Management reported the results of a Delphi approach in order to identify criteria for SSI in various types Digital photography for assessment of wound infection 7 of wounds. [21] In the Delphi approach of the acute wound, 8-10 panel members were asked to list relevant clinical indicators of infection. Panel members were offered the opportunity to review scores for the most important criteria in view of the position of the group as a whole (the ‘group score’). Cellulitis and pus/abscess were considered the most important factors, followed by delayed healing, erythema with or without induration, haemo- or seropurulent exudate, malodour and wound breakdown/enlargement. Assumed early signs of infection included increase of local skin temperature, oedema, serous exudates with erythema, swelling with increase in exudate volume and unexpected pain/tenderness. [21] The predictive value of these signs are yet unknown for acute wounds. Gardner et al found positive predictive values of 1.00 for increasing pain and wound breakdown in chronic wounds. [22]. Sensitivity of classic signs of infection in chronic wounds showed large variability among different items: heat and purulent exudate 0.18, increasing pain 0.36, erythema 0.55 and edema 0.64 [22]. Moreover, from the few studies that exist on inter observer agreement in wound assessment, it appears that kappa values for many of the important variables in the Delphi approach were not high. Hollander et al reported inter observer concordances (kappa values) of 0.51 for erythema, 0.39 for warmth, 0.38 for tenderness and 1.00 for infection of 100 wounds registered in the emergency department by two independent doctors. [11] Allami et al reported inter observer variations in the evaluation of 50 lower limb arthroplasty wounds between four observers. [12] In this study, poor inter observer agreement (kappa values <0.40) was reported for tenderness, localised swelling, redness, heat, moderate agreement for pain (kappa values 0.60-0.80) and good agreement (kappa values 0.80-1.00) for clinical diagnosis of superficial SSI, purulent drainage, dehiscence and fever. In a study by Wirthlin et al, agreement amongst surgeons in the ‘remote’ assessment of digital photographs of 38 Digital photography for assessment of wound infection 8 vascular surgery wounds, similar to our study, proved lowest on the aspects cellulitis/infection and erythema (kappa values of 0.08 and 0.28, respectively). [1] The mean kappa value for inter observer agreement on wound infection of 0.62 found in our study may be fair considering the results from previous studies, in which presumably fewer infected wounds were included. In our study, digital wound photographs were assessed with additional information available on wound pain -expressed as visual analogue scale scores-, postoperative day and morning temperature, which was thought to have been of additional value for the diagnosis of infection and to better simulate the clinical setting. The two-dimensional aspect of digital photographs hampered assessment of swelling of the wound edges. Palpation of the wound was an aspect which was considered an omission from the regular physical examination of wounds by the surgeons participating in this study. Palpation can provide valuable information in view of expression of wound pain and pus production during pressure exertion and elicit increased capillary refill. Digital photography, even with the provided additional information, seems adequate for diagnosis of ‘normal wound healing’ (i.e., no infection) in wounds based on a high specificity of 97%, but at a mean sensitivity of 42% not be sensitive enough to diagnose infections in all wounds. Besides the discussion on the use and validity of digital photography in wound assessment, it appears important that more objective criteria for wound infection should be defined to create more uniformity in the diagnosis of wound infection. For this reason, more research is needed to evaluate the predictive value of wound characteristics for wound infection such as wound temperature and production of exudate, to be incorporated in a standardized wound appraisal tool. Structural assessment of wounds, combined with on-site registration of SSI and plenary Digital photography for assessment of wound infection 9 discussion will undoubtedly result in more uniformity amongst surgeons and higher reliability of reported infections and infection rates. CONCLUSIONS Inter and intra observer agreement on the diagnosis of wound infection when using digital photography were both moderate, but specificity was very high. Physical examination, including palpation, appeared of high additional value to digital photography in the assessment of laparotomy wounds and should therefore be documented in detail. We recommend that digital photographs come with this information in future studies on ‘electronic’ wound assessment. Furthermore, we feel that more objective criteria for wound assessment and knowledge on the predictive value of wound characteristics for infection are needed. Digital photography for assessment of wound infection 10 ACKNOWLEDGEMENTS We thank Dr. W.E. Tuinebreijer for his thoughtful review of the manuscript. Digital photography for assessment of wound infection 11 REFERENCES 1. Wirthlin DJ, Buradagunta S, Edwards RA, Brewster DC, Cambria RP, Gertler JP, et al. Telemedicine in vascular surgery: feasibility of digital imaging for remote management of wounds. J Vasc Surg. 1998;27(6):1089-99; discussion 1099-1100. 2. Kvedar JC, Edwards RA, Menn ER, Maofid M, Gonzalez E, Dover J, et al. The substitution of digital images for dermatologic physical examination. Arch Dermatol. 1997;133(2):161-7. 3. Baumgarten M, Margolis DJ, Selekof JL, Moye N, Jones PS, Shardell M. Validity of pressure ulcer diagnosis using digital photography. Wound Repair Regen. 2009;17(2):287-90. 4. Molnar JA, Lew WK, Rapp DA, Gordon ES, Voignier D, Rushing S, et al. Use of standardized, quantitative digital photography in a multicenter web-based study. Eplasty. 2009;9:e4. 5. Reid R, Simcock JW, Chisholm L, Dobbs B, Frizelle AA. Postdischarge clean wound infections: incidence underestimated and risk factors overemphasized. ANZ J Surg. 2002; 72:339-43. 6. Coello R, Charlett A, Wilson J, Ward V, Pearson A, Borriello P. Adverse impact of surgical site infections in English hospitals. J Hosp Infection 2005;60:93-103. 7. Mangram AJ, Horan TC, Pearson ML, Silver LC, Jarvis WR. Guideline for prevention of surgical site infection. Hospital Infection Control Practices Advisory Committee. Infect Control Hosp Epidemiol. 1999;20(4):250-78. 8. Quinn JV, Drzewiecki AE, Stiell G, Elmslie TJ. Appearance scales to measure cosmetic outcomes of healed lacerations. Am J Emerg Med 1995;13(2):229-31. 9. Singer AJ, Arora B, Dagum A, Valentine S, Hollander JE. Development and validation of a novel scar evaluation scale. Plast Reconstr Surg 2007;120(7):1892-7. 10. Quinn JV, Wells GA. An assessment of clinical wound evaluation scales. Acad Emerg Med. 1998;5(6):583-6. Digital photography for assessment of wound infection 12 11. Hollander JE, Singer AJ, Valentine S, Henry MC. Wound registry: development and validation. Ann Emerg Med 1995;25(5):675-85. 12. Allami MK, Jamil W, Fourie B, Ashton V, Gregg PJ. Superficial incisional infection in arthroplasty of the lower limb. Interobserver reliability of the current diagnostic criteria. J Bone Joint Surgery 2005;87(9):1267-71. 13. Sacket DL, Haynes RB, Guyatt GH, Tugwell P (1991). Second Edition. Clinical epidemiology. A basic science for clinical medicine. Boston/Toronto/London: Little, Brown and Company, 1991. 14. Bruce J, Russell EM, Mollison J, Krukowski ZH. The measurement and monitoring of surgical adverse events. Health Technol Assess. 2001;5(22):1-94. 15. Centers for Disease Control and Prevention: National Nosocomial Infections Surveillance System (NNIS) USA. http://www.cdc.gov/ncidod/dhqp/nnis.html. 16. Health Protection Agency (HPA): Nosocomial Infection National Surveillance Service (NINSS), England. http://www.hpa.org.uk. 17. Scientific Institute of Public Health: National Programme for Surveillance of Hospital Infections (NSIH), Belgium. http://www.iph.fgov.be. 18. Dutch Institute for Healthcare Improvement (CBO), Netherlands. http://www.cbo.nl. 19. Wilson AP, Treasure T, Sturridge MF, Gruneberg RN. A scoring method (ASEPSIS) for postoperative wound infections for use in clinical trials of antibiotic prophylaxis. Lancet 1986;1:311-3. 20. Bailey IS, Karran SE, Toyn K, Brough P, Ranaboldo C, Karran SJ. Community surveillance of complications after hernia repair. BMJ 1992;304:469-71. 21. Cutting KF, White RJ, Mahoney P, Harding KG. Clinical identification of wound infection: a Delphi approach. In: EWMA Position Document. Identifying criteria for wound infection. London: MEP Ltd, 2005. Digital photography for assessment of wound infection 13 22. Gardner SE, Frantz RA, Doebbeling BN. The validity of the clinical signs and symptoms used to identify localized chronic wound infection. Wound Repair Regeneration 2001;9(3):178-86. Digital photography for assessment of wound infection 14 TABLES AND CAPTIONS TABLE 1 Surgeon Infection present (N) Sensitivity Infection absent (N) Specificity A 24 48 % 48 96 % B 16 32 % 48 96 % C 22 44 % 50 100 % D 21 42 % 47 94 % Total 50 42 % (mean) 50 97 % (mean) CAPTION 1: Sensitivity and specificity for infected and non-infected wounds Digital photography for assessment of wound infection 15 TABLE 2 Surgeons Wound infection 95% CI lower - upper Treatment 95% CI lower - upper A – B 0.54 0.34 - 0.73 0.15 0.00 - 0.39 A – C 0.67 0.50 - 0.84 0.17 0.00 - 0.39 A – D 0.68 0.51 - 0.85 0.20 0.00 - 0.43 B – C 0.63 0.43 - 0.82 0.72 0.54 - 0.90 B – D 0.58 0.39 - 0.78 0.63 0.42 - 0.84 C – D 0.61 0.42 - 0.79 0.68 0.49 - 0.86 CAPTION 2: Paired kappa values for inter observer agreement on wound infection and treatment Digital photography for assessment of wound infection 16 TABLE 3 Surgeon Wound infection 95% CI lower -upper Treatment 95% CI lower-upper A 0.66 0.49 – 0.84 0.52 0.15 – 0.89 B 0.43 0.26 – 0.61 0.53 0.31 – 0.75 C 0.74 0.57 – 0.91 0.76 0.59 – 0.93 D 0.76 0.62 – 0.91 0.87 0.75 – 0.99 CAPTION 3: Kappa values for intra observer agreement on wound infection and treatment Digital photography for assessment of wound infection 17 TITLE PAGE 1. Title: ‘Validity of diagnosis of superficial infection of laparotomy wounds using digital photography: inter and intra observer agreement amongst surgeons’ 3. Authors: Gabriëlle H. van Ramshorst1, MD, Wietske W. Vrijland2, MD, PhD, Erwin van der Harst3, MD, PhD, Wim C.J. Hop4, PhD, Dennis den Hartog1, MD, PhD, Johan F. Lange1, MD, PhD, Professor of Surgery 8. Source(s) of support: none TEXT INTRODUCTION ACKNOWLEDGEMENTS 15. Centers for Disease Control and Prevention: National Nosocomial Infections Surveillance System (NNIS) USA. http://www.cdc.gov/ncidod/dhqp/nnis.html.