key: cord-0979272-o8545vs2
authors: Silverstein, Evan; Williams, Jonathan S.; Brown, Jeffrey R.; Bylykbashi, Enjana; Stinnett, Sandra S.
title: Teleophthalmology: Evaluation of Phone-based Visual Acuity in a Pediatric Population
date: 2020-08-11
journal: Am J Ophthalmol
DOI: 10.1016/j.ajo.2020.08.007
sha: 7b6cb0ad47b150d92e1e77091254492de9ee6e8a
doc_id: 979272
cord_uid: o8545vs2

PURPOSE: With the recent rise of teleophthalmology due to coronavirus disease, we are in need of accurate and reliable methods of checking visual acuity remotely. We compare the visual acuity as measured by the GoCheck Kids application, HOTV with the amblyopia treatment study (ATS), and our clinic protocol. DESIGN: This is a prospective, comparison of visual acuity assessment methods. METHODS: Established patients (aged 3-18) in the practice of a single pediatric ophthalmologist were eligible. Visual acuity was measured by: 1) GoCheck Kids mobile application (by patient’s family member), 2) HOTV-ATS (by study personnel), and 3) regular clinic protocol (by ophthalmic technician). To assess agreement between measures of acuity, intraclass correlations with 95% confidence intervals were computed. RESULTS: Fifty-three children participated. The mean difference between GoCheck Kids and HOTV-ATS acuities (0.094) was significantly different (p<0.001); the intraclass correlation (ICC) was 0.55 (95% CI: 0.40, 0.68). The mean difference between GoCheck Kids and chart acuities (0.010) was not significantly different (p=0.319); ICC: 0.59 (95% CI: 0.45, 0.71). The mean difference between HOTV-ATS and chart acuities (0.084) was significantly different (p<0.001); ICC: 0.66 (95% CI: 0.53, 0.76). The percent of eyes with visual acuity as measured by GoCheck Kids that was within 1 line of the HOTV-ATS and chart acuity was 65.3% and 86.7%, respectively. CONCLUSIONS: GoCheck Kids as checked by a family member provides a modest correlation of visual acuity compared to the chart screen and a fair correlation of visual acuity compared to HOTV-Amblyopia Treatment Study protocol, though a vast majority are within 1 line.

Telemedicine has been thrust into our ophthalmology practices by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and resulting coronavirus disease . The American Academy of Ophthalmology has recommended significant reduction of the number of clinic visits and recommends only seeing patients for urgent and emergent reasons. 1 These guidelines are reinforced by social distancing instructions and Stay at Home orders in many states like Virginia. 2 With this new landscape of medicine, we are in need of accurate ways of measuring vision at our patient's homes.

There are multiple methods of checking visual acuity in children in clinic: Lea symbols, Kay pictures, HOTV, Snellen letters, Sloan letters, Early Treatment of Diabetic Retinopathy Study (ETDRS), tumbling "E"; all with and without crowding bars. [3] [4] [5] [6] [7] [8] [9] [10] Lea symbols, HOTV, and letter acuity have been recommended for use in children. 8, 11 Visual Acuity is also mass-tested in vision screening programs with the goal of detecting children with visual impairment as recommended by the American Academy of Pediatrics and the American Association for Pediatric Ophthalmology and Strabismus. 12, 13 In these screening programs, many different eye charts are employed and visual acuity is measured by different people: teachers, nurses, certified medical assistants, public health care workers, students, and other lay-people. [14] [15] [16] [17] [18] [19] [20] [21] Mobile and computer applications have been developed to improve and increase the consistency of checking visual acuity in children's screening programs. Some of these applications have been investigated through peer-reviewed publications. [22] [23] [24] [25] [26] [27] [28] [29] The GoCheck Kids application is used for vision screening through photo-screening and checking visual acuity. This application has been studied for photoscreening and detecting amblyopia risk factors, but not for checking visual acuity. [30] [31] [32] This study was approved by the Institutional Review Board in December 2018 with the goal of exploring a family member checking a child's visual acuity.

This prospective study was approved by the Virginia Commonwealth University Institutional Review Board. Informed consent was obtained by parents of the subjects and assent to study was obtained by subjects > 7 years of age.

From April 2019 -February 2020, established patients in the practice of a single pediatric ophthalmologist (ES) between the ages of 3 and 18 were eligible for the study. This population was selected since the goal of this study was to evaluate the visual acuity measurement of children with known and treated refractive errors (if required).

Visual acuity was measured in three ways: 1) GoCheck Kids mobile application (Scottsdale, Arizona), 2) HOTV-Amblyopia Treatment Study protocol (HOTV-ATS) on M&S System (Niles, Illinois), and 3) regular clinic protocol. To mimic visual acuity tested at home, visual acuity was checked by the parent, guardian, or sibling ("tester") of the patient for GoCheck Kids. A medical student or the pediatric ophthalmologist checked the vision for HOTV-ATS, and an ophthalmic technician checked vision for the regular clinic protocol. The Apple iPhone 7 plus (Cupertino, California) used for the study was loaned to the study by GoCheck Kids.

Visual acuity from the GoCheck Kids application was measured with the following protocol. A trained medical student or the ophthalmologist taught the parent or sibling of the patient how to use the application. GoCheck Kids checks threshold visual acuity by displaying a test letter in crowding bars (HOTV for children 6 years and younger, and ETDRS letters for children 7 years and older) below four randomized letters in crowding bars ( Figure 1 ). The child indicates which direction the tester should tilt the device until the test letter matches the letter in the randomized row. The child then notifies the tester of a correct placement of the letter and the tester tilts in the phone downwards to lock in the answer (Video 1). The protocol is based on the EyeSpy video game software. 28 The test is performed at 5 feet using a previously measured piece of rope held between the child and the tester to ensure proper distance. The right eye was tested before the left eye and the non-tested eye was occluded. Testers were instructed not to look at the screen to blind them to the test. No feedback was given to the parents during the test to mimic testing environment at home. The GoCheck Kids application can report vision from 20/20 (0.0 logMAR) -20/63 (0.5 logMAR).

HOTV-ATS visual acuity was measured at 16 feet (the length of the examination room from chair to visual acuity system) using the M&S System running the Amblyopia Treatment Study protocol. HOTV with crowding bars were used for all participants. The right eye was tested before the left eye. The non-tested eye was occluded. Visual acuity was also measured at 16 feet with the M&S system using the clinic's protocol, testing with the most challenging acuity that the child could reliably perform: Sloan > HOTV line > HOTV crowding bar ("chart acuity").

Also recorded was the relationship of the tester to the child, the child's gender and race, the glasses prescription, and their ophthalmic diagnoses. The presence of vision disease was defined as a diagnosis that affected the child's visual acuity (for example, a child with juvenile idiopathic arthritis with a normal examination was classified as not having a vision disease and a child with amblyopia is classified as having vision disease).

The data analysis for this paper was generated using SAS/STAT software, Version 9.4 of the SAS System for Windows (Copyright 2002-2012 SAS Institute Inc). Descriptive statistics were computed for demographic variables and for GoCheck Kids acuity, HOTV-ATS acuity and the paired difference between them in LogMAR units. To assess the significance between acuities, generalized estimating equations (GEE) were used to test whether the mean difference was equal to zero. To assess agreement between measures of acuity, intraclass correlations with 95% confidence intervals were computed. In addition, the percentage of children with visual acuity as measured by GoCheck Kids that were within 1 and 2 lines of HOTV-ATS and of regular clinic protocol were calculated.

Study data were managed using REDCap electronic data capture tools hosted at Virginia Commonwealth University. 33, 34 REDCap (Research Electronic Data Capture) is a secure, web-based software platform designed to support data capture for research studies, providing 1) an intuitive interface for validated data capture; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for data integration and interoperability with external sources.

Fifty-three children were consented to participate in the study. The application prematurely stopped and recorded a vision much worse than the child's true vision in two children (96% testability). Observation from the study personnel suggested that the tester was tilting the phone downwards while tilting the phone to the side, thus prematurely locking in an answer. These children were not used in the statistical evaluation. Demographics presented in Table 1 . There were 15 children <7 years of age. Four eyes could not be measured by the GoCheck Kids application because they could not identify the largest (20/63 -0.5 logMAR) optotype on the device. At least 2 testers physically dropped the phone during testing.

The mean logMAR (imperial) acuity assessed by GoCheck Kids was 0.106 (20/25.5), by HOTV-ATS was 0.012 (20/20.5) , and by chart was 0.096 (20/22). The mean difference and confidence intervals between GoCheck Kids and HOTV-ATS acuities (0.094, 0.074-0.114) was significantly different (p<0.001). The mean difference and 95% confidence intervals between GoCheck Kids and chart acuities (0.010, -0.010-0.030) was not significantly different (p=0.319). The mean difference and confidence interval between HOTV-ATS and chart acuities (0.084, 0.014-0.063) was significantly different (p<0.001). The intraclass correlation between GoCheck Kids and HOTV-ATS acuities was 0.55 (95% CI: 0.40, 0.68) and between HOTV-ATS and chart acuities was 0.59 (95% CI: 0.45, 0.71), indicating fair agreement between each of the two sets measures of acuity. The intraclass correlation between GoCheck Kids and chart acuities was 0.66 (95% CI: 0.53, 0.76), indicating a modest agreement between the measures. Bland-Altman plots are presented in Figure 2 .

The percent of eyes with visual acuity as measured by GoCheck Kids that was within 1 line of the HOTV-ATS and chart acuity was 65.3% and 86.7%, respectively. The percent of eyes with visual acuity as measured by GoCheck Kids that was within 2 lines of the HOTV-ATS and chart acuity were both 96.9%.

There were 14 children with a history of unilateral amblyopia. The differences between the acuity between the amblyopic eye and the non-amblyopic eye as measured by each method are in Table 2 .

Telemedicine has arrived. COVID-19 has pushed us to adapt so we can continue to safely serve our patients. Before COVID-19, telemedicine was rapidly developing in the J o u r n a l P r e -p r o o f field of ophthalmology, mostly for diseases like diabetic retinopathy and glaucoma, by obtaining images and sending them to an ophthalmologist for review [35] [36] [37] [38] [39] or for interpretation by artificial intelligence 40 and in retinopathy of prematurity using deep neural networks. 41 In the COVID era of teleophthalmology, we do not have the benefit of these sophisticated methods of image acquisition and interpretation. We also need a way of accurately measuring visual acuity, especially in children, in order help diagnose and treat amblyopia. The ideal method of checking visual acuity at home would be easy to use, accurate and precise, inexpensive (or free), ensure monocular testing, and be available to every single patient and family.

ABCD-Vision (Anchorage, Alaska) developed a paper-based HOTV crowding bar visual acuity test to be used for remote use in Alaska. 42 This 8.5x11" paper acuity chart can be emailed for at-home printing or mailed to patients who do not have access to a printer.

Another method of checking vision at home is using a smartphone or computer. Some of these programs have peer-reviewed studies (Peek Acuity (Peek Vision, London, United Kingdom), EyeHandbook (Cloud Nine Development LLC, Overland Park, Kansas), Jaeb Visual Acuity Screener (Jaeb Center for Health Research, Tampla, Florida). [22] [23] [24] [25] [26] [27] [28] [29] 43 and some are not (Farsight (Farsight.care), Eye Chart Pro (Dok LLC), Visual Acuity Charts (Fonlo), Snellen Acuity (João Meneses)). Table 3 .

There are several issues with using electronic applications to check visual acuity. The first being access-some applications are only available on Android (Google, Mountain View, California) and others are iOS exclusive (Apple, Cupertino, California). Other applications are only available on Windows-based computers (Microsoft, Redmond, Washington). While smartphones are considered ubiquitous in today's society, consideration should also be taken into account for those without access to this technology due to their financial situation. This limits the ability to provide uniform directions to patients and have consistent measurements across all patients. But, if a patient uses the same method for checking visual acuity over two visits, the acuities are theoretically comparable. Second, we have to trust that the developer has modified the size of the optotypes for different screen sizes. Some applications provide configuration screens to ensure the correct size of the optotypes. Third, the choice of optotypes can be important. Tumbling "E" optotypes (used by Peek Acuity) can be difficult for very young children who have not developed the ability to express the orientation of the optotypes, 12 though more recent studies report excellent testability. 44 Fourth, some orientation may be required for a family to check vision at home-via in app directions, a video guide, or testing while on a teleconference with an ophthalmic technician.

This study is unique in that we explore the use of GoCheck Kids to check visual acuity by the patient's relative with minimal training. There have been many studies evaluating the assessment of visual acuity by lay screeners in vision screening programs. [14] [15] [16] [17] [18] [19] [20] [21] The goal of those studies is to refer children with decreased visual acuity below an agedbased critical line for further examination by an eye care provider. Now, we are shifting the narrative to explore the use of the visual acuity as measured by these methods as a visual acuity that an ophthalmologist will use for their evaluation and decision-making process.

This study shows that visual acuity using GoCheck Kids at 5 feet has fair agreement with HOTV-ATS. On average, the visual acuity as measured by GoCheck Kids is one line worse than HOTV-ATS and very close to the chart acuity. The visual acuity measured by HOTV-ATS averages one line better than the chart acuity. Therefore, it is unclear if both GoCheck Kids and chart acuity underestimate vision, or the HOTV-ATS overestimates vision. The average age of children in our study was 10 years old; these children may have understood that there were only 4 choices in the HOTV-ATS and guessed on the letters based on similar recognition rather than true identification, thus measuring vision as better than the other methods. In addition, prior studies have showed that HOTV-ATS overestimates vision compared to E-ETDRS by 0.68 lines for amblyopic eyes and 0.25 lines for fellow eyes. 45 This is the largest weakness of this study.

A similar study was performed for the Peek Acuity application. 22 This study tested 111 children, ages 3-17 and compared visual acuity from Peek Acuity to clinic protocol visual acuity and showed a good ICC for first (0.88) and second (0.84) eyes tested; though the ICC dropped for second eyes (0.45) in children aged 3-5. We did not test for this variable.

When assessing and treating amblyopia, one might argue that the exact visual acuity does not matter as much as the difference of visual acuity between the two eyes. In children with amblyopia, there was no consistency in the difference between the two eyes among all the methods of checking visual acuity. One out of the 14 children had the same difference of acuity between all methods.

A strength of this study is the method that visual acuity was checked using the GoCheck Kids application. We purposefully had the family members of the child perform the measurement to simulate checking vision at home. The study personnel also limited communication and instruction to the tester once the test had started to further mimic home testing environment. Children were able to learn the visual acuity game quickly. Parents had mild difficulty learning the motions needed to manipulate the device, as indicated by the dropped phones and the premature stopping of the test. Further studies could be performed by having family members check vision at home with the application prior to presenting to the clinic for their examination. We recommend further studies use HOTV-ATS for children < 7 years of age and E-ETDRS for children > 7 years of age.

A weakness of this study is the children were not patched. Occlusion with patches over glasses, occlusion glasses (that have occluders than flip down over the non-tested eye), or a handheld occluder. Children are more consistent with visual acuity when an occlusive patch is applied directly to the skin. In addition, since most children who would be treated for amblyopia may be < 7 years of age, our study is limited by our small sample size in this age group.

In its current form, GoCheck Kids is limited by the size of an iPhone Plus screen size. The lowest acuity that can be measured at 5 feet is 20/63 (0.5 logMAR). Some testing methods allow children to move closer to the chart and changing the numerator of the visual acuity, but this is not possible with the already short testing distance of 5 feet for GoCheck Kids. This significantly limits the use of this application when testing children with amblyopia, as many of them will have an acuity of < 0.5 logMAR. When testing vision at home, it may be important to have a backup test if clinical suspicion arises that the acuity is not consistent with prior tests of vision or if the visual acuity is too low to be tested by the application. It is unknown how GoCheck Kids will perform for measuring visual acuity in children with uncorrected refractive errors. It may over-estimate distance acuity in children with myopia due to the 5-foot testing distance. A program that still allows for matching when needed and the ability to test at 10-feet would be beneficial for a pediatric testing device; a second device that is paired may be helpful for the child to hold when matching the letter at distance (though this would require the family to have two devices).

In June 2020, GoCheck released an app with an updated user interface that is dedicated to checking visual acuity. It is available outside of their GoCheck Kids ecosystem.

In conclusion, GoCheck Kids has a high rate of testability and provides a modest correlation of visual acuity compared to the chart screen and a fair correlation of visual acuity compared to HOTV-Amblyopia Treatment Study protocol, though a vast majority are within 1 line. More studies are needed to evaluate this method of checking visual acuity at home for teleophthalmology. J o u r n a l P r e -p r o o f 43 , d EyeChart Pro has two apps for iPhone, EyeChart -Vision Screening and EyeChart HD-Vision Screening. Both work at 4 feet, but do not recognize the size of iPhone 11 pro, states it is using an iPhone 6. There are other mobile applications for checking visual acuity that are not included on this list. 

• Visual acuity can be checked at home with paper charts or devices • GoCheck Kids has a modest correlation of visual acuity compared to the chart screen • GoCheck Kids has a fair correlation of visual acuity compared to HOTV-ATS

Governor Northam Issues Statewide Stay at Home Order. Office of the Governor

The repeatability of best corrected acuity in normal and amblyopic children 4 to 12 years of age

The amblyopia treatment study visual acuity testing protocol

Crowded letter and crowded picture logMAR acuity in children with amblyopia: a quantitative comparison

Examination of young children with Lea symbols

Do picture-based charts overestimate visual acuity? Comparison of Kay Pictures, Lea Symbols, HOTV and Keeler logMAR charts with Sloan letters in adults and children

Assessment of near visual acuity in 0-13 year olds with normal and low vision: a systematic review

Comparison of visual acuity measurements via three different methods in preschool children: Lea symbols, crowded Lea symbols, Snellen E chart

Comparison of the HOTV and Lea symbols visual acuity tests in patients with amblyopia

Computerized method of visual acuity testing: adaptation of the amblyopia treatment study visual acuity testing protocol

Procedures for the Evaluation of the Visual System by Pediatricians

Guidelines for automated preschool vision screening: a 10-year, evidence-based update

Comparison of preschool vision screening tests as administered by licensed eye care professionals in the Vision In Preschoolers Study

Assessing accuracy of non-eye care professionals as trainee vision screeners for children

Paediatric vision screening by non-healthcare volunteers: evidence based practices

Sensitivity and Specificity of Preschool Vision Screening in Iran

Strategies to improve the accuracy of vision measurement by teachers in rural Chinese secondary schoolchildren: Xichang Pediatric Refractive Error Study (X-PRES) report no

Agreement and diagnostic accuracy of vision screening in children by teachers, community eye-health workers and vision technicians

Validation of performance of certified medical assistants in preschool vision screening examination

Accuracy of visual assessment by school teachers in school eye screening program in delhi

Visual Acuity Assessment and Vision Screening Using a Novel Smartphone Application

Computerized method of visual acuity testing: adaptation of the amblyopia treatment study visual acuity testing protocol

Development and Validation of a Smartphone-Based Visual Acuity Test (Peek Acuity) for Clinical Practice and Community-Based Fieldwork

Validation of Peek Acuity application in pediatric screening programs in Paraguay

A new computer-based pediatric visionscreening test

A pilot trial of the iPad tablet computer as a portable device for visual acuity testing

A pilot study evaluating the use of EyeSpy video game software to perform vision screening in school-aged children

Development and Validation of a Smartphone-Based Visual Acuity Test (Peek Acuity) for Clinical Practice and Community-Based Fieldwork

Effectiveness of the GoCheck Kids Vision Screener in Detecting Amblyopia Risk Factors

Positive predictive value and screening performance of GoCheck Kids in a primary care university clinic

Effectiveness of the iPhone GoCheck Kids smartphone vision screener in detecting amblyopia risk factors

Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support

The REDCap consortium: Building an international community of software platform partners

Evaluation of telemedicine for screening of diabetic retinopathy in the Veterans Health Administration

Diabetic retinopathy screening and the use of telemedicine

Ocular telemedicine for diabetic retinopathy and the Joslin Vision Network

Prospective evaluation of teleophthalmology in screening and recurrence monitoring of neovascular agerelated macular degeneration: a randomized clinical trial

Tele-transmission of stereoscopic images of the optic nerve head in glaucoma via Internet

Artificial intelligence in diabetic retinopathy: A natural step to the future

Automated retinopathy of prematurity screening using deep neural networks. EBioMedicine

Portable acuity screening for any school: validation of patched HOTV with amblyopic patients and Bangerter normals

Effectiveness of a smartphone application for testing near visual acuity

Should tumbling E go out of date in amblyopia screening? Evidence from a population-based sample normative in children aged 3-4 years

Comparison of the amblyopia treatment study HOTV and the electronic-early treatment of diabetic retinopathy study visual acuity protocols in amblyopic children aged 5 to 11 years