key: cord-0072511-unby0sex
authors: Zheng, Xiao-Ben; Ling, Bingo Wing-Kuen; Zeng, Zhi-Tao
title: Evaluation of effectiveness of eye massage therapy via classification of periocular images
date: 2021-12-29
journal: Multimed Tools Appl
DOI: 10.1007/s11042-021-11789-w
sha: 502d76c066e0c4721beeb8ed00e9e5ac135f60b8
doc_id: 72511
cord_uid: unby0sex

This paper proposes a method to evaluate the effectiveness of the eye message therapy. The existing methods are via the diagnoses conducted by the medical professions based on the measurements acquired by the optical instruments. However, this approach is very expensive. To address this issue, this paper performs the classification between the periocular images taken before performing the eye massage therapy and those after performing the eye massage therapy to address the above difficulty. First, the median filtering is used to suppress the solitary point noise with preserving the edges of the image without causing the significant blurring. Then, the Canny operator is employed to accurately locate the edges. Next, the circle Hough transform (CHT) is used for performing the iris segmentation. Finally, various classifiers are used to perform the classification. The computer numerical simulation results show that our proposed method can achieve the high classification accuracies. This implies that there is a significant difference on the iris before performing the eye massage therapy and after performing the eye massage therapy. In addition, the comparisons with the state of art Daugman method have been performed. It is found that the classification performance achieved by the CHT based method is better than those achieved by the Daugman method.

Eye diseases such as the dry eye diseases, the myopia, the diabetic eye diseases and the glaucoma are the worldwide prevalent diseases. According to the report issued by the World Health Organization, nearly one billion of people in the worldwide have the above preventable eye diseases. The eye diseases can be due to many factors [2, 12, 18] . These factors include the aging, the genetic factors, the viral or the bacterial infections as well as the excessive and the incorrect use of the eyes. In recent studies, some evidences show that some eye diseases such as the dry eye diseases, the myopia and the computer vision based eye syndromes are closely related to the eye fatigue [10, 26] . Since doing the eye exercises to protect the eyes [17, 20] was promoted in the past few decades, many people take the eye massage therapy to soothe the eyes to relieve the eye strain and to protect the eyes. As a result, the eye massage therapy becomes popular.

The evaluation of the effectiveness of the eye massage therapy plays an important role in the treatment of the eye diseases. However, the works in this area are very limited. At present, the evaluations are mainly dependent on the diagnoses conducted by the medical professions based on the measurements acquired by the optical instruments [15] . In particular, the LipView interferometer and the fluorophotometer are utilized to perform the diagnoses of the dry eye diseases [8] at the dry eye clinics. Here, the interferometer provides two important measurements. They are the average thickness of the lipid layer and the partial blink rate. Besides, the fluorophotometer measures the penetration of the corneal fluorescein. However, conducting the above measurements is very expensive and it takes a very long processing time. On the other hand, an image based evaluation method was proposed [25] . The diagnoses of the dry eye diseases were performed using the images acquired from the vivo confocal microscopic and the optical coherence tomographic instruments. Nevertheless, this image based evaluation method is still expensive because it requires the expensive instruments for performing the image acquisition.

This paper proposes a periocular image classification based method for performing the evaluation of the effectiveness of the eye massage therapy. Here, the periocular images are taken by the portable cameras. First, the iris segmentation is performed. Different techniques were used for detecting the location of the eye via detecting the location of the centre of the pupil for performing the iris segmentation. The common methods include the CHT based method and the Daugman method [1, 7, 29] . In particular, the Daugman integro-differential operator [6] was used to segment the iris in the Daugman method. Also, many improvements were achieved based on this approach [23, 24] . On the other hand, the CHT based method was used to segment the iris [14, 19] . Then, the change of the size of the iris in the image is computed. However, these methods were sensitive to the noises. Besides, detecting the edge of the iris is an important step for performing the iris segmentation [4] . Therefore, the noise is required to be suppressed without losing the edge information [27] for performing the iris segmentation. Different methods [28] were used to suppress the noises. In this paper, the median filtering is employed for performing the denoising. This is because it can keep the edge information while does not blur the image. Besides, the Canny operator can detect the iris edge effectively. Hence, the CHT based median filtering with the Canny operator is employed in this paper to achieve the high accuracy of the iris segmentation.

Our procedures for performing the evaluation of the effectiveness of the eye massage therapy are as follows. First, the periocular images are taken before performing the eye massage therapy and after performing the eye massage therapy. Then, the iris in the images is segmented out using the CHT based method. After that, the region between the inner circle and the outer circle of the iris is mapped to a rectangular box. Next, the discrete cosine transform is used to normalize the size of the rectangular box to 32 × 64. By mapping the feature matrix to a feature vector, various classifiers including the random forest based classifier, the k nearest neighbors based classifier, the support vector machine based classifier and the artificial neural network based classifier are employed for performing the classification. The computer numerical simulation results show that all the classifiers can achieve the high classification accuracies. Compared to the Daugman method, the CHT based method can achieve the higher classification accuracies.

The contributions of this paper are as follows. First, a method for performing the evaluation of the effectiveness of the eye massage therapy is proposed. The computer numerical simulation results scientifically validate the effectiveness of the eye massage therapy. This implies that the eye message therapy can be performed in a quicker and a cheaper manner.

The outline of this paper is as follows. Section 2 reviews the CHT and the normalization method. Section 3 presents the method for performing the evaluation of the effectiveness of the eye massage therapy. The computer numerical simulation results are shown in Section 4. Finally, a conclusion is drawn in Section 5.

For the ease of the comprehension and the reading, the acronyms and the symbols appeared in this paper are summarized in Table 1 .

The CHT is effective for characterizing the regular objects such as the straight line and the circular objects such as the iris in an image. It can be extended for the detection of the ellipses and any object with other shapes that could be characterized using the parameters [3] . Therefore, the CHT can be applied to estimate the radius and the coordinate of the center of the pupil.

For each edge point, if a circle is drawn with its center being this edge point, then the intersecting point of all these circles represents the center of the circle of all these edge points. Figure 1 illustrates this concept. Let r be the radius and (a,b) be the coordinate of the center of the circle. Let (x i ,y i ) be the coordinates of the edge points. Obviously, if x i ,y i is on the circle with its radius r and its center coordinate (a,b), then it should satisfy [5, 11, 22] : Here, H(a,b,r) counts the total number of the edge points on the circle. The parameter pair (a,b,r) in the above should maximize H(a,b,r) and this parameter pair can be used to represent the edge points in the corresponding circle.

Normalization is a very important step that can be used for mapping the points between the inner circle and the outer circle of the iris defined by the CHT to a two dimensional rectangular box. Here, the rubber sheet model is employed for performing the normalization [16] . Since only the points in the iris are extracted, it can remove the irrelevant information outside the iris for performing the iris classification. Let (r p ,θ) and (r i ,θ) be the coordinates of the points in the inner circle and the outer circle of the iris represented in the polar coordinate system, respectively. Let (r p (θ),y p (θ)) and (x i (θ),y i (θ)) be the coordinates of the points in the inner circle and the outer circle of the iris along the direction θ represented in the Cartesian coordinate system, respectively. Let (x p0 ,y p0 ) and (x i0 ,y i0 ) be the coordinates of the centers of the inner circle and the outer circle of the iris represented in the Cartesian coordinate system, respectively. Then, we have and Let x(r,θ) and y(r,θ) be the x co-ordinates and the y co-ordinates of the points between the inner circle and the outer circle of the iris represented in the Cartesian coordinate system, respectively. Let wϵ [0, 1] . Then, the rubber sheet model maps the points between the inner circle and the outer circle of the iris to a rectangular box as follow [5, 22] : and Figure 2 shows the rubber sheet model.

The objective of this paper is to evaluate the effectiveness of an eye massage therapy via classifying the periocular images taken before performing the eye massage therapy and those after performing the eye massage therapy. The procedures for performing the iris classification [13] are shown in Fig. 3 . Here, it includes (i) the image acquisition, (ii) the iris segmentation, (iii) the normalization, (iv) the feature extraction and (v) the iris classification.

Compared to the image acquisition systems used in the dry eye clinic such as the near infrared iris system, the images employed in this paper are acquired by a mobile device. Obviously, the images can be acquired easier and more convenient. First, a video with its duration equal to 15 s is taken around the eye before performing the eye massage therapy. Next, an eye massage therapy is performed using an eye massage device for 15 min.

(4b) y p ( ) = y p0 + r p sin , Then, a video with its duration equal to 15 s is taken around the eye after performing the eye massage therapy. In order to minimize the effects of the devices, the same mobile device is used to capture all the videos and the same eye massage device is employed to perform the therapy for all the subjects. In order to ensure that the obtained results are 

Various components of the eye are shown in Fig. 5 . It can be seen that the eye mainly consists of three components [22] . They are the sclera, the iris and the pupil which located from the outer part of the eye to the inner part of the eye, respectively. In particular, the white outer area represents the sclera. The back inner area is the pupil which is a small black circle in the center of eye. The iris lies in the annular region between the sclera and pupil. It is worth noting that the lower part and the upper part of the iris region are sometimes occluded by the eyelids and eyelashes, respectively.

In order to perform the iris segmentation, the CHT based median filtering with the Canny operator is proposed. The block diagram for performing the iris segmentation is shown in Fig. 6 . First, the median filtering is applied to suppress the noise. Then, the Canny operator is employed to perform the edge detection. Next, the CHT is employed to characterize the iris. For performing the Canny edge detection, the block diagram is shown in Fig. 7 . In particular, the gradients of the pixel values are computed. Then, a thresholding operator is applied to obtain the edge map.

The details of each step are discussed below.

Let (i,j) be the starting index of a block of an image. Let x p,q be the grayscale pixel value at the index (p,q). Let median(•) be the median operator. Let r × s be the size of a window. Let y i,j be the output of the median filter. Then, From here, it can be seen that the median filtering is to sort the pixels in a window and to take the pixel value located in the middle of the sorted pixels as the output of the filter. Figure 8 shows an example of output after performing the median filtering. It is worth noting that the median filtering is a typical nonlinear operator used to suppress the impulsive noise. Compared to the conventional linear filtering techniques, this nonlinear filtering technique has a lower blurring effect and better preserves the edges in the image [9] .

As the camera introduces the impulsive noise to the periocular images and the noise degrades the quality of the periocular images, this paper employs the median filtering for suppressing the noises. First, each frame is extracted from each video. Then, the median filter with the size of the window equal to 3 × 3 is used for performing the denoising. The original image and the image denoised by performing the median filtering are shown in in Fig. 6 The block diagram for performing the iris segmentation Fig. 7 The block diagram for performing the Canny edge detection Fig. 8 The median filtering Fig. 9a and Fig. 9b , respectively. It can be seen from Fig. 5 that the eyelashes near the iris are removed. Hence, it can facilitate the iris classification.

The edge of an object is the border across two regions in the image in which these two regions have two sets of contrast pixel values. In other words, the brightness of the pixels changes sharply or more formally has the discontinuities across these two regions in the image. Performing the edge detection is to identify the pixels at the border. As the border across an object and the background can be identified via performing the edge detection, the edge detection plays an important role in the image segmentation. Moreover, as the edge points preserve the fundamental structures of the object, some important features such as the locations of the lines or the curves in the image can be extracted via performing the edge detection. In addition, only processing these edge points can reduce the required computational power for some image processing applications.

The Canny edge detection is the most common edge detection algorithm. It performs the following steps [21] .

(i) Computing the gradients: Let G x and G y be the horizontal gradient and the vertical gradient of an object in the image, respectively. Let G be the gradient of the object in the image. Then, we have (ii) Thresholding: Only the pixels with the large gradient values are regarded as the edge points. Therefore, an appropriate threshold value is used to determine whether the pixels are located at the edge points or not.

It is worth noting that the iris segmentation is crucial for performing the iris classification. This is because a wrong segmentation of an object would result to the meaningless of the subsequent steps. First, the median filtering is performed on each color plane of the image as discussed in Sect. 3.2.1. Then, the Canny edge detection is applied to denoised image. Figure 10a and Fig. 10b show the inner edge points and the outer edge points of the iris in the image after performing by the Canny edge detection algorithm, respectively. It can be seen from Fig. 10 that the closed shape of the iris can be detected. Finally, the iris region is segmented from each color plane. Fig. 9 The original image and the denoised image

After performing the Canny edge detection to obtain a binary edge map, the edge points are employed to find the parameters of the circles via the CHT. The inner circle and the outer circle are obtained via performing the CHT. Figure 11 shows the image with the inner circle and the outer circle of the iris.

After mapping the points between the inner circle and the outer circle of the iris to the two dimensional rectangular box, the sizes of the rectangular boxes are different for different images. In order to eliminate the size inconsistence issue, the DCT is applied to normalize the size of the rectangular box to 32 × 64. That is, if the original size of the rectangular box is smaller than 32 × 64, then the zeros are padded to the rectangular box in the DCT domain. On the other hand, if the original size of the rectangular box is larger than 32 × 64, then the high frequency components of the rectangular box are discarded in the DCT domain. This can also suppress the high frequency noise generated by the specular reflections. Next, the inverse DCT is applied to the processed rectangular box and the rectangular box with the unified sizes are obtained. Finally, the resized rectangular box is represented as a vector with its dimension equal to 2048 × 1. This vector is employed as the feature vector for performing the classification. Figure 12a shows that the circular region 

In this paper, the images are divided into two categories. They are the images before and after performing the eye massage therapy. Here, 70% of the selected images are employed for performing the training. On the other hand, the rest 30% of the selected images are employed for performing the testing. The overall total numbers of the selected images in the training set and those in the testing set are summarized in Table 2 .

Since the classification accuracy is the most common criterion employed for evaluating the pattern recognition performance, this paper employs the classification accuracy as the metric. In order to demonstrate the effectiveness of performing the eye massage therapy, several classifiers are employed. In particular, the random forest based classifier, the KNN based classifier, the SVM based classifier and the ANN based classifier are employed. The Daugman method is employed as the benchmark method for evaluating the performance of the iris segmentation. Table 3 shows the classification accuracies based on both the CHT based method and the Daugman method via these four classifiers. It can be seen from Table 3 that the classification accuracies are high for both methods and for all these four classifiers. This implies that it is possible to classify into two types of images namely those obtained before performing the eye massage therapy and those obtained after performing the eye massage therapy. In other words, there is a significant difference on the iris between Fig. 12 a The circular region in the iris before performing the normalization. b The resized rectangular box Table 2 The overall total numbers of the selected images in the training set and those in the testing set Total number of the selected images 518 Total number of the selected images in the training set 363 Total number of the selected images in the test set before performing the eye massage therapy and after performing the eye massage therapy. Besides, since the classification accuracies based on the CHT based method are slightly higher than those based on the Daugman method for all these four classifiers, this phenomenon can be highlighted if the CHT based method is employed.

The confusion matrices are employed to further evaluate the classification performances achieved by various methods. In particular, the confusion matrices achieved by the CHT based method and the Daugman method via these four classifiers are shown in Fig. 13 and Fig. 14, respectively. It can be seen from Fig. 13 and Fig. 14 that the obtained results are consistent with those shown in Table 3 . That is, the classification performances achieved . 13 The confusion matrix achieved by the CHT based method via a the RF based classifier, b the KNN based classifier, c the SVM based classifier and d the ANN based classifier by both the CHT based method and the Daugman method are good. Besides, the classification accuracies based on the CHT based method are slightly higher than those based on the Daugman method for all these four classifiers. The received operating characteristic curve (ROC) is employed as the performance metric to evaluate the robust performance achieved by various classifiers. The ROC plots the true positive rate (TPR) against the false positive rate (FPR). If the FPR is small, then the classification accuracy will be high. Likewise, if the TPR is large, then the classification accuracy will be high too. The ROC can be divided into two parts. In particular, the slopes of the curve in the first part and the second part are high and low, respectively. Therefore, the ROC is also known as the sensitivity curve. Figure 15 plots the ROC curves achieved by both the CHT based method and the Daugman method via different classifiers. It can be seen from Figure 15 that all the ROCs achieve the good performances. This implies that all the classifiers are robust for performing the classification. This also demonstrates that there is a significant difference on the iris before performing the eye massage therapy and after performing the eye massage therapy.

The area under the ROC (AUC) is used to evaluate the prediction accuracy. In general, the higher the AUC refers to the higher the prediction accuracy. Since the quicker response of the Since the range of the AUC is between 0 and 1, the area under the 45° straight line in the plot is 0.5. If the AUC is equal to 0.5, then the model likes performing a random guess such as flipping a coin. In this case, the prediction does not tell any information. On the other hand, if the AUC is equal to 1, then the model predicts correctly and the classification is perfect. However, there is not perfect classification in practice. Hence, the classification is effective if the AUC is between 0.5 and 1. Table 4 shows the AUCs achieved by both the CHT based method and the Daugman method via various classifiers. It can be seen from Table 4 that the AUCs achieved by both the CHT based method and the Daugman method are large for all these four classifiers. This demonstrates that there is a significant difference on the iris before performing the eye massage therapy and after performing the eye massage therapy. Besides, the CHT based method is superior than the Daugman method. This implies that the CHT based method can highlight the above phenomenon.

In order to evaluate the reliability performances achieved by various classifiers, the ten fold cross validations are performed. Table 5 shows the mean classification accuracies based on these ten fold cross validations achieved by both the CHT based method and the Daugman method via different classifiers. It can be seen from Table 5 that the mean classification accuracies based on these ten fold cross validations achieved by both the CHT based method and the Duangman method are high. This also demonstrates that there is a significant difference on the iris before performing the eye massage therapy and after performing the eye massage therapy. Besides, the mean classification accuracies based on these ten fold cross validations achieved by the CHT based method are slightly higher than those achieved by the Duangman method, this implies that the CHT based method can highlight the above phenomenon. 

This paper evaluates the effectiveness of an eye massage therapy via performing the classification between the images taken before performing the eye massage therapy and those after performing the eye massage therapy. Here, 10 subjects have taken 200 videos using a mobile device and 518 images are selected for performing the iris classification. First, a median filtering is employed for suppressing the noises. Then, the Canny edge detection is employed for generating the edge maps. Next, the CHT is employed to characterize the inner circle and the outer circle of the iris in the images. After that, the normalization process is to map the region between the inner circle and the outer circle of the iris to a rectangular box. Then, the DCT is employed to generate the feature vectors. Finally, various classifiers are employed for performing the classification. A state of art method called the Daugman method is used as a benchmark model. The computer numerical simulation results demonstrated that both the CHT based method and the Daugman method are effective, robust and reliable for performing the classification. This demonstrates that there is a significant difference on the iris before performing the eye massage therapy and after performing the eye massage therapy. Besides, the CHT based method is slightly effective, robust and reliable than the Daugman method. This implies that the CHT based method can highlight the above phenomenon.

In future, more analysis on the eye massage therapy will be investigated. In particular, the relationship between the duration of each eye massage therapy and its effects on the iris images as well as the relationship between the environments for conducting the eye massage therapy such as the relax conditions of the subjects and their effects on the iris images will be investigated.

Technology Commission, Enterprise Support Scheme (no. S/E/070/17), and the Bella (Guangzhou) Intelligent Information Technology Company Limited, for providing its electronic device.

Management and Optimization. He has published an undergraduate textbook, a research monograph, several book chapters, 200 + internationally leading journal papers and 130 + highly rated international conference papers. His research interests include the time frequency analysis, the optimization theory, the symbolic dynamic, the multimedia signal processing and the biomedical signal processing.

Zhi-Tao Zeng is the president of the electronic technology industry association of the Zengcheng District, Guangzhou, since 2018, and awarded a China excellent honest entrepreneur in 2019. He founded the Xunlei Intelligence (Guangzhou) Technology Company Limited in 2010, which is a national high technology enterprise. He also founded the Bella (Guangzhou) Intelligent Information Technology Company Limited in 2017. The company was awarded the "Private Enterprise with Outstanding Contribution to the COVID-19 Epidemic in Guangzhou". He research interests include the human health based medical devices such as the eye massage device and the sleep meter.

Iris segmentation analysis using integro-differential and hough transform in biometric system

Sleep and mood disorders in dry eye disease and allied irritating ocular diseases

Improved hough transform for fast iris detection

Proc. of National Conf. on Information Technology and Computer Science (CITCS)

New methods in iris recognition

How iris recognition works. The essential guide to image processing

Review on the proportional study of segmentation techniques for iris acknowledgment

Fluorophotometry as a diagnostic tool for the evaluation of dry eye disease

A survey on various median filtering techniques for removal of impulse noise from digital image

Computer vision syndrome: a review

A survey on Hough transform, theory, techniques and applications

Myopia, an underrated global challenge to vision: where the current data takes us on myopia control

An efficient iris recognition system

Reliable iris localization using Hough transform, histogram-bisectio, and eccentricity

Evaluation of incomplete blinking as a measurement of dry eye disease

Iris segmentation and normalization using Daugman's rubber sheet model

Chinese eye exercises and myopia development in school age children: a nested case-control study

The definition and classification of dry eye disease

The pupil location based on the OTSU method and hough transform

Eye exercises of acupoints: their impact on refractive error and visual symptoms in Chinese urban children

A survey on feature extraction techniques for color images

Recognition of human iris patterns for biometric identification. Dissertation

An enhanced iris segmentation algorithm using circle Hough transform

An improved Daugman method for Iris recognition

Image-guided evaluation and monitoring of treatment response in patients with dry eye disease

Computer vision syndrome among computer office workers in a developing country: an evaluation of prevalence and risk factors

EDGE detection techniques using Hough transform

Iris segmentation along with noise detection using hough transform

Daughman's algorithm method for iris recognition-a biometric approach

 Table 5 The mean classification accuracies based on these ten fold cross validations achieved by the CHT based method and the Daugman He has also served as the guest editor-inchief of several special issues of highly rated international journals such as the Circuits, Systems and Signal Processing. He is currently the associate editors of the Journal of Franklin Institute, the Circuits, Systems and Signal Processing, the Measurement, the IET Signal Processing, and the Journal of Industrial