DC111317 482..484 Imager Evaluation of Diabetic Retinopathy at the Time of Imaging in a Telemedicine Program The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters Citation Cavallerano, Jerry D., Paolo S. Silva, Ann M. Tolson, Taniya Francis, Dorothy Tolls, Bina Patel, Sharon Eagan, Lloyd M. Aiello, and Lloyd P. Aiello. 2012. Imager evaluation of diabetic retinopathy at the time of imaging in a telemedicine program. Diabetes Care 35(3): 482-484. Published Version doi:10.2337/dc11-1317 Citable link http://nrs.harvard.edu/urn-3:HUL.InstRepos:10611819 Terms of Use This article was downloaded from Harvard University’s DASH repository, and is made available under the terms and conditions applicable to Other Posted Material, as set forth at http:// nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of- use#LAA http://osc.hul.harvard.edu/dash/open-access-feedback?handle=&title=Imager%20Evaluation%20of%20Diabetic%20Retinopathy%20at%20the%20Time%20of%20Imaging%20in%20a%20Telemedicine%20Program&community=1/4454685&collection=1/4454686&owningCollection1/4454686&harvardAuthors=ec12615d347079a50c5d1489946fa14f&department http://nrs.harvard.edu/urn-3:HUL.InstRepos:10611819 http://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#LAA http://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#LAA http://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#LAA Imager Evaluation of Diabetic Retinopathy at the Time of Imaging in a Telemedicine Program JERRY D. CAVALLERANO, OD, PHD1,2 PAOLO S. SILVA, MD1,2 ANN M. TOLSON, BA1 TANIYA FRANCIS, BA1 DOROTHY TOLLS, OD1 BINA PATEL, OD1,3 SHARON EAGAN, OD1 LLOYD M. AIELLO, MD1,2 LLOYD P. AIELLO, MD, PHD1,2 OBJECTIVEdTo evaluate the ability of certified retinal imagers to identify presence versus absence of sight-threatening diabetic retinopathy (stDR) (moderate nonproliferative diabetic retinopathy or worse or diabetic macular edema) at the time of retinal imaging in a telemedicine program. RESEARCH DESIGN AND METHODSdDiabetic patients in a primary care setting or specialty diabetes clinic received Joslin Vision Network protocol retinal imaging as part of their care. Trained nonphysician imagers graded the presence versus absence of stDR at the time of imaging. These gradings were compared with masked gradings of certified readers. RESULTSdOf 158 patients (316 eyes) imaged, all cases of stDR (42 eyes [13%]) were iden- tified by the imagers at the time of imaging. Six eyes with mild nonproliferative diabetic reti- nopathy were graded by the imagers to have stDR (sensitivity 1.00, 95% CI 0.90–1.00; specificity 0.97, 0.94–0.99). CONCLUSIONSdAppropriately trained imagers can accurately identify stDR at the time of imaging. Diabetes Care 35:482–484, 2012 T he American Telemedicine Associa- tion Telehealth Practice Recommen- dations for Diabetic Retinopathy identifies four categories of telemedi- cine care for diabetic retinopathy (1). Cat- egory 1 programs identify patients with no or minimal diabetic retinopathy (Early Treatment Diabetic Retinopathy Study [ETDRS] level 20 or below) versus those with diabetic retinopathy more severe than ETDRS level 20. Category 2 pro- grams accurately determine whether sight-threatening diabetic retinopathy (stDR), as evidenced by any level of dia- betic macular edema (DME), severe or worse levels of nonproliferative diabetic retinopathy (NPDR) (ETDRS level 53 or worse), or proliferative diabetic retinopa- thy (ETDRS level 61 or worse), is present or not present. Category 3 programs ac- curately identify ETDRS-defined levels of diabetic retinopathy and DME to deter- mine appropriate follow-up and treat- ment. Category 4 programs can replace ETDRS 7-standard field 35-mm stereo- scopic color fundus photographs in any clinical or research program. The Joslin Vision Network (JVN) is a validated category 3 program (2–5). Im- agers undergo an intensive 3-day program that includes fundus camera operation and imaging software navigation; struc- tured courses on diabetes, ocular anat- omy, diabetic retinopathy, and common ocular disorders; and a guided review demonstrating retinal images of nondi- seased and diseased eyes. As part of the certification, imagers learn to recognize lesions of diabetic retinopathy, including hemorrhages, microaneurysms, venous caliber abnormalities, intraretinal micro- vascular abnormalities, retinal neovascula- rization, cotton wool spots, hard exudates, and laser scars. Salient retinal abnormali- ties not related to diabetes are also dem- onstrated, including choroidal nevi, retinal emboli, and large or asymmetrical optic cup-to-disc ratios. After the 3-day program, imagers serve a probationary period with senior imager supervision and ongoing quality improvement and assurance. This prospective study assessed the ability of two certified imagers to conduct American Telemedicine Association Cat- egory 2 (presence vs. absence of stDR) grading at the time of retinal imaging. RESEARCH DESIGN AND METHODSdPatients with diagnosed diabetes had nonmydriatic JVN imaging as part of their routine physical examina- tions in a primary care setting (HealthCare Associates, Beth-Israel Deaconess Medical Center) or a specialty diabetes clinic (Adult Diabetes, Joslin Diabetes Center). At the time of imaging, certified imagers (A.M.T., 71 patients [45%]; T.F., 87 patients [55%]) identified patients with potential stDR, defined for this program as ETDRS levels of 43 or worse (6) or DME, and un- gradable images. Imagers were not able to manipulate the color, brightness, contrast, or other features of the images and could not view images stereoscopically. To grade retinal thickening without stereoscopic viewing, imagers relied on identifying hard exudates or microaneurysms within 3,000 microns from the center of the mac- ula as surrogate markers for DME. The two certified imagers were Bachelor of Arts col- lege graduates with no prior health care experience in evaluating retinal images and had not provided direct patient care before working as retinal imagers. Certified readers graded images according to the previously described JVN protocol (2,3) in a central reading center with calibra- ted monitors and stereoscopic viewing capability. All readers in the JVN program are Massachusetts-licensed optometrists. c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c From the 1Beetham Eye Institute, Joslin Diabetes Center, Boston, Massachusetts; the 2Department of Oph- thalmology, Harvard Medical School, Boston, Massachusetts; and the 3New England College of Optometry, Boston, Massachusetts. Corresponding author: Paolo S. Silva, paoloantonio.silva@joslin.havard.edu. Received 13 July 2011 and accepted 21 November 2011. DOI: 10.2337/dc11-1317 © 2012 by the American Diabetes Association. Readers may use this article as long as the work is properly cited, the use is educational and not for profit, and the work is not altered. See http://creativecommons.org/ licenses/by-nc-nd/3.0/ for details. 482 DIABETES CARE, VOLUME 35, MARCH 2012 care.diabetesjournals.org C l i n i c a l C a r e / E d u c a t i o n / N u t r i t i o n / P s y c h o s o c i a l R e s e a r c h B R I E F R E P O R T mailto:silva@joslin.havard.edu http://creativecommons.org/licenses/by-nc-nd/3.0/ http://creativecommons.org/licenses/by-nc-nd/3.0/ All readers were masked to the grading performed by the imagers. All findings were recorded on a specifically designed template. RESULTSdA total of 158 consecutive patients were imaged. Mean age was 56.5 years (range 22–86), 54% female, and the mean diabetes duration was 7.0 years (range 0.1–42). A total of 316 eyes were evaluated, and 195 (61.7%) had no dia- betic retinopathy, 62 (19.6%) had mild NPDR, 24 (7.6%) had moderate NPDR, 3 (1%) had severe or very severe NPDR, 2 (0.6%) had proliferative diabetic reti- nopathy, and 30 (9.5%) were ungradable for diabetic retinopathy. DME was absent in 266 (84.2%) eyes, present in 13 (4.1%), and 37 (11.7%) were ungradable for DME. Of the 316 eyes assessed, imagers identified 48 (15%) eyes with potential stDR at the time of imaging. Subsequent grading by certified readers classified 6 (12.5%) of these eyes as mild NPDR. The imagers accurately identified all cases of stDR as graded by the readers. Although limited by the moderate sample size and the use of only two independent imagers, the agreement for determining stDR be- tween imagers and readers was 0.95 6 0.02. The sensitivity and specificity in identifying stDR at the time of imaging by a certified imager is 1.00 (95% CI 0.90–1.00) and 0.97 (95% CI 0.94– 0.99), respectively (positive predictive value 0.88 [95% CI 0.74–0.95]; negative predictive value 1.00 [0.98–1.00]). There was complete agreement between im- agers and readers regarding ungradable eyes (37 [12%]). Table 1 presents a cross-tabulation of imager and reader evaluations for the presence of stDR and ungradable images. CONCLUSIONSdFilm or digital ret- inal imaging is a sensitive method to identify the presence and level of diabetic retinopathy (7–10). Despite efforts to au- tomate retinal image evaluation (11–13), currently no system can perform such analyses in real time, and present meth- ods of retinal imaging require trained im- agers to acquire retinal images. This study shows that appropriately educated and certified imagers following a clearly defined imaging and grading pro- tocol can accurately evaluate retinal im- ages with a high degree of sensitivity and specificity for the presence of stDR and inadequate image quality at the time of imaging. The ability to identify ungradable images and detect potential stDR facili- tates reacquisition of retinal images during a single imaging encounter and allows prompt referral to appropriate eye care. Although this study involved a moderate number of eyes (n = 316), 42 (13%) eyes with stDR and 37 (12%) eyes with ungradable images were identified, representing all cases that would have re- quired further ophthalmic evaluation and care. Additional studies with a vari- ety of imagers and patient populations will be required to determine whether similar results can be obtained across di- verse health care scenarios. However, the fact that the two certified imagers in- volved in this study had no prior health care experience in evaluating retinal images suggests that similar results are possible. In this study, retinal imagers had received a validated standardized method of cer- tification and training, which is an impor- tant consideration when extrapolating these results to other retinal imaging programs. AcknowledgmentsdNo potential conflicts of interest relevant to this article were re- ported. J.D.C. and P.S.S. researched data and wrote the manuscript. A.M.T., T.F., D.T., B.P., and S.E. researched data and reviewed and edited the manuscript. L.M.A. and L.P.A. reviewed and edited the manuscript and contributed to discussion. J.D.C. and P.S.S. are the guarantors of this work and, as such, had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. References 1. Li HK, Horton M, Bursell SE, Cavallerano J, et al. Telehealth practice recommendations for diabetic retinopathy, second edi- tion. Telemed J E Health 2011;17:814– 837 2. Ahmed J, Ward TP, Bursell SE, Aiello LM, Cavallerano JD, Vigersky RA. The sensi- tivity and specificity of nonmydriatic digital stereoscopic retinal imaging in de- tecting diabetic retinopathy. Diabetes Care 2006;29:2205–2209 3. Bursell SE, Cavallerano JD, Cavallerano AA, et al.; Joslin Vision Network Research Team. Stereo nonmydriatic digital-video color retinal imaging compared with Early Treatment Diabetic Retinopathy Study seven standard field 35-mm stereo color photos for determining level of diabetic retinopathy. Ophthalmology 2001;108: 572–585 4. Cavallerano JD, Aiello LP, Cavallerano AA, et al.; Joslin Vision Network Clinical Team. Nonmydriatic digital imaging alter- native for annual retinal examination in persons with previously documented no or mild diabetic retinopathy. Am J Ophthalmol 2005;140:667–673 5. Chow SP, Aiello LM, Cavallerano JD, et al. Comparison of nonmydriatic digital reti- nal imaging versus dilated ophthalmic examination for nondiabetic eye disease in persons with diabetes. Ophthalmology 2006;113:833–840 6. Early Treatment Diabetic Retinopathy Study Research Group. Grading dia- betic retinopathy from stereoscopic color fundus photographs: an exten- sion of the modified Airlie House clas- sification. ETDRS report number 10. Ophthalmology 1991;98(Suppl.):786– 806 7. Klein R, Klein BE, Neider MW, Hubbard LD, Meuer SM, Brothers RJ. Diabetic retinopathy as detected using ophthal- moscopy, a nonmydriatic camera and a standard fundus camera. Ophthalmol- ogy 1985;92:485–491 8. Li HK, Danis RP, Hubbard LD, Florez- Arango JF, Esquivel A, Krupinski EA. Comparability of digital photography with the ETDRS film protocol for evaluation Table 1dCross-tabulation of grading for the presence of stDR by JVN imagers and JVN readers Grading by JVN reader No stDR stDR Cannot grade n (%) Grading by JVN imager No stDR 231 0 0 231 (73) stDR 6 42 0 48 (15) Cannot grade 0 0 37 37 (12) n (%) 237 (75) 42 (13) 37 (12) Total 316 (100) Numbers represent number of eyes. Exact matches between JVN imagers and JVN readers are in bold. Simple k statistic: 0.95 6 0.02 (95% CI 0.92–0.99). Agreement for absence of stDR: 97% (0.94–0.99). Agreement for presence of stDR: 88% (0.74–0.95). Agreement for cannot grade: 100% (0.88–1). care.diabetesjournals.org DIABETES CARE, VOLUME 35, MARCH 2012 483 Cavallerano and Associates of diabetic retinopathy severity. Invest Ophthalmol Vis Sci 2011;52:4717– 4725 9. Hubbard LD, Sun W, Cleary PA, et al.; Diabetes Control and Complications Trial/ Epidemiology of Diabetes Interventions and Complications Study Research Group. Comparison of digital and film grading of diabetic retinopathy severity in the Di- abetes Control and Complications Trial/ Epidemiology of Diabetes Interventions and Complications Study. Arch Ophthalmol 2011;129:718–726 10. Gangaputra S, Almukhtar T, Glassman AR, et al. Comparison of film and dig- ital fundus photographs in eyes of in- dividuals with diabetes mellitus. Invest Ophthalmol Vis Sci 2011;52:6198– 6173 11. Chaum E, Karnowski TP, Govindasamy VP, Abdelrahman M, Tobin KW. Au- tomated diagnosis of retinopathy by content-based image retrieval. Retina 2008; 28:1463–1477 12. Abràmoff MD, Reinhardt JM, Russell SR, et al. Automated early detection of dia- betic retinopathy. Ophthalmology 2010; 117:1147–1154 13. Fleming AD, Philip S, Goatman KA, Pre- scott GJ, Sharp PF, Olson JA. The evi- dence for automated grading in diabetic retinopathy screening. Curr Diabetes Rev 2011;7:246–252 484 DIABETES CARE, VOLUME 35, MARCH 2012 care.diabetesjournals.org Grading diabetic retinopathy at time of imaging