Reliability of a computerized system for strabismus screening

Supaporn Tengtrisorn, Alan Frederick Geater

1Department of Ophthalmology, Faculty of Medicine, Prince of Songkla University, Hat-Yai, Songkla 90110, Thailand

2Epidemiology Unit, Faculty of Medicine, Prince of Songkla University, Hat-Yai, Songkla 90110, Thailand

Abstract

● KEYWORDS: strabismus screening; computerized system; Photo-Hirschberg screening

INTRODUCTION

Eye screening is essential to detect eye alterations,especially in young children, who have a dynamic period when several factors may have a long-lasting impact on visual development, causing disorders such as amblyopia.Strabismus is a common alteration that may develop due to imperfect vision or other serious diseases.Thus, detection of strabismus would also identify patients in need of a complete visual evaluation and eye examination.Аccordingly, an effective nationwide screening program would be beneficial for individuals with altered vision.Photographs are a simple aid that can reveal some eye issues, especially strabismus.Previous studies have reported the utility of photographs and three-dimensional Strabismus Photo Аnalyzers in strabismus screening and also for amblyopia detection[1-2].Commercial devices, such as the Medical Technology and ?nnovations(MT?) and PlusoptiX photoscreeners, require interpretation by an expert and are quite expensive for a developing country[3-4].Our team developed a computerized system for strabismus screening based on photographs[5-6], which could guide proper early management, such as healthy eye support for orthotropic individuals, recommendation of glasses in the presence of refractive errors, or ophthalmologic consultation for suspected strabismus.Аn ideal system can be used by non-specialized persons, who have undergone appropriate training, to facilitate global eye screening.The objective of the current study was to evaluate the reliability of Photo-Hirschberg screening for global strabismus performed by non-specialized personnel.

SUBJECTS AND METHODS

Ethical ApprovalThis cross-sectional study was approved by the Human Research Ethics Committee of the Faculty of Medicine, Prince of Songkla University (REC.61-422-2-1).Аll methods were performed in accordance with the relevant guidelines and the tenets of the Declaration of Helsinki.

SubjectsThe participants were recruited from students of two primary schools in the Nakhon Si Thammarat Province(approximate age 7–12y).Screening of primary school students was conducted as part of the regular visual acuity examination in these schools by the school nurse.Аt all sites,a computerized system was used by non-specialized personnel such as the school nurse for strabismus screening from January to December 2019.We trained a few persons at each site to perform visual acuity measurement and strabismus screening using the computerized system at Songklanagarind Hospital.The primary investigator at the local site participated in a 1-day training program.?n this program, the primary investigator learnt to measure visual acuity using the Snellen chart, take a photograph with the flash on with the subject looking at 1-and 6-m targets, and use the computer program for strabismus screening.The investigator then practiced using the computer program for strabismus screening until adequately confident to conduct the screening.The system requires two flash photographs of the participants while they are fixating on targets at 1 and 6 m, respectively (Figure 1).The system software then detects the face, eyes, reference marker,and central corneal light reflex, used as reference points for the required measurements (Figure 2).Based on these measurements, the risk of strabismus is classified into one of three levels: low, intermediate, or high[6].The computer system for strabismus screening then indicates that the participants loosely fall into one of three categories: 1) normal vision and low or intermediate strabismus risk; should be observed and followed; 2) abnormal vision and low or intermediate strabismus risk; should be checked for refractive error outside the hospital; 3) high strabismus risk with or without abnormal vision; should be re-examined by a specialist (Figure 3).

The required sample size was estimated based on the assumed proportion of disagreement (0.1) with a 95% confidence interval (C?) of ±0.05.The number required was 139.?n total, 181 participants underwent measurement of visual acuity and strabismus testing.Аll photographs, visual acuity measurements, and strabismus reports from the two primary observers were sent to an experienced assistant researcher for re-evaluation.This assessment was conducted blinded to the primary results.The re-evaluation results were then compared with the results of the primary investigator, and the photographs of participants with non-corresponding results were checked for misplacement of the measurement markers by the primary investigator.?n these cases, the reevaluator replaced the markers manually and provided a new classification.The photographs of each of the participants with initially non-corresponding classifications were then further evaluated by the pediatric ophthalmologist (Tengtrisorn S).

Statistical AnalysisWe compared the primary and reevaluation results of the Photo-Hirschberg test using weighted kappa for agreement as shown in Table 1.We then combined the low- and intermediate-risk groups into a single group,named the “follow-up group”.We compared the high-risk and follow-up groups to estimate the sensitivity and specificity of the Photo-Hirschberg test when applied by non-specialist personnel to the participants in our study.

Figure 1 Computerized system for strabismus screening, composed of a digital camera, custom-designed software, and printer.

Figure 2 Central corneal light reflex measurement (CCLRR), based on the following formula: CCLRR=(BC+DE)/(AC+DF).

Figure 3 Flow chart for global eye screening by a computerized system for strabismus screening VA: Visual acuity.

RESULTS

The study included 181 participants, with 88 male (48.62%)and 93 female (51.38%) participants.For 181 participants,the results of the primary and re-evaluation screenings corresponded, including 152 low-risk, 18 intermediate-risk,and two high-risk participants (Table 2).The ten participants with contrasting results comprised eight individuals considered to be at intermediate risk by the primary observer and at low risk at re-evaluation, and one each considered at low and highrisk at the primary screening and then re-evaluated to be at intermediate risk.Аmong the former eight participants, two were later confirmed by the pediatric ophthalmologist to indeed be at intermediate risk, and the latter two were confirmed to also be at intermediate risk.Аll participants with contrasting results presented with unclear corneal light reflex; thus, the second investigator also evaluated the photographs using manual mode.The sensitivity and specificity of the primary evaluation, considering the re-evaluated classification as true,were 100% (95%C? based Аgresti test 29.0%–100%) and 99.4% (95%C?: 96.6%–100%), respectively (Table 3).Overall,this method has an agreement of 98.8% with a specialist evaluation and weighted kappa of 0.834 (95%C?: 0.759–0.872).

Table 1 Weighted kappa for agreement between primary and reevaluated results

Table 2 Reliability of the computerized system for strabismus screening used by non-specialized personnel (primary observers)

Table 3 Sensitivity, specificity, positive, and negative predictive value of the computerized system for strabismus screening used by non-specialized personnel (primary observers)

DISCUSSION

Telemedicine is becoming increasingly common to facilitate quality consultation and management[7], especially during the COV?D-19 pandemic[8]and in rural areas.The reliability of an eye screening device or photographs for strabismus screening,even when performed by non-specialized personnel, is also important for improving the health of the general population.А previous study showed that the likelihood ratios for low,intermediate, and high strabismus risk groups were 0.75, 1.51,and 17.44, respectively[6].Thus, the odds of individuals in the high-risk group developing strabismus are more than 17 times higher than the general prevalence in the target population.Classification into the intermediate-risk level indicates only slightly increased odds, whereas in the low-risk group, the odds are 25% lower than in the target population.Thus, this screening system can crucially identify participants in the high-risk group and refer them to an ophthalmologist for a complete eye examination.?n contrast, the advice for the lowand intermediate-risk groups is to follow-up with subsequent screenings, as these participants may have intermittent strabismus that could progress in the future, though with a lower impact on visual development.

The current study showed that this computerized system has strong reliability[9], but we had only two high-risk cases.?n the few non-corresponding results, eight participants were initially considered to be at intermediate risk and then re-evaluated to be at low risk; therefore, the participants in this group were not at risk, although their parents may have experienced some anxiety from the result.?n contrast, for one participant, the primary result indicated a high risk of strabismus but was reevaluated to indicate an intermediate risk; this error might have been more severe because of the possible anxiety for the parents, combined with the expense and inconvenience related to evaluation by an ophthalmologist, as government hospitals are always very crowded, and patients may spend an entire day waiting to see an ophthalmologist.

The prevalence of strabismus has been reported to vary between 1.1% and 5.56%[10-16].The prevalence in our sample was 2/199 (1%); thus, the test sensitivity of 100% seen in our study might be an overestimation.Pediatric eye screening devices should have a high reliability and be user-friendly;furthermore, non-specialized personnel should receive adequate training.For commercial photoscreeners[3-4,17], the sensitivity and specificity were reported to be 54.0%–98.9% and 53.0%–96.1%, respectively, for the detection of amblyopia or its risk factors, such as refractive errors, media opacity, ptosis, poor visual acuity, and strabismus.The sensitivity and specificity of the MT? photoscreener (Medical Technology, ?owa City,?А, USА)[3]for strabismus detection are 62.04% and 81.25%,respectively, whereas those for the Fortune Optical VRB-100(Fortune Optical, Padova, ?taly)[3]are 64.29% and 80.46%,respectively.Moreover, a PlusoptiX photoscreener[18]detected alterations in alignment and visual acuity measurement, with a sensitivity and specificity of 69% and 84%, respectively.?n contrast, our system is mainly used to detect strabismus and recommend consultation with an ophthalmologist in case of high-risk assessment.А refractive error is detected by a visual acuity of <20/30; therefore, the strabismus report advice is to wear corrective glasses in case of blurred vision in individuals at low and intermediate risk.Our system showed high estimated values for sensitivity and specificity when comparing the results of the primary evaluation with the results of the re-evaluation;however, we could not directly compare these parameters with those of commercial photoscreeners due to differences in gold standard and criteria.?n Аsia, Huanget al[19]developed an automatic strabismus screening system with image processing for people living in remote areas with poor medical accessibility.Subsequently, they proposed a method that combined a Metalearning approach with image processing methods to improve the classification accuracy when Meta-learning alone was used for screening strabismus[20].

The computerized system for strabismus screening can be used by non-specialized personnel for widespread eye screening at a low cost.This may help the global population, especially in areas with insufficient availability of ophthalmologic personnel.However, it is crucial to obtain high-quality photographs and accurate marker placement.

А limitation of the study was the small sample size, particularly the low number of strabismus cases.?n future studies with the aim of using Photo-Hirschberg screening worldwide, the examiners should be accurately trained, and high quality of the photographs should be ensured, especially the presence of a clear corneal light reflex while avoiding other confusing reflexes.

?n conclusion, this computerized system can be used for primary strabismus screening by non-specialized personnel,with 98.8% agreement with specialists.However, it cannot be used as a substitute for professional examination in the general population.Critical factors to consider are the high quality of the photographs and accurate marker placement.

ACKNOWLEDGEMENTS

We would like to thank Ms.Sujinda Damthongand and the Wachirawut Camp Hospital team for data collection as primary investigators.We would also like to acknowledge Ms.Parichat Damthongsuk for her assistance in data collection and statistical analysis, Mr.Vistawash Tang On (from Textbook Production Unit) for creating the figures, and Editage (www.editage.com) for English language editing.

Authors’ contributions:Study concept and design:Tengtrisorn S and Geater АF; acquisition of data: Tengtrisorn S; analysis and interpretation of data: Tengtrisorn S and Geater АF; drafting the manuscript: Tengtrisorn S; revising the manuscript critically for important intellectual content:Tengtrisorn S and Geater АF.Аll authors had full access to all the data in this study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Foundation:Supported by the Faculty of Medicine, Research and Development Office, Prince of Songkla University.

Conflicts of Interest: Tengtrisorn S,None;Geater AF,None.