key: cord-0045451-as1q6ps7
authors: Forero, Manuel G.; Ávila-Navarro, Julián; Herrera-Rivera, Sergio
title: New Method for Extreme Color Detection in Images
date: 2020-04-29
journal: Pattern Recognition
DOI: 10.1007/978-3-030-49076-8_9
sha: f22482cc1cf83979943b710232af96380bbc06a9
doc_id: 45451
cord_uid: as1q6ps7

In image processing and computer vision, it is common to find applications, in which it is necessary to detect reference points characterized by extreme color, i.e., a primary color RGB or complementary CMY with very high saturation. Thus, there are cases in which a certain class of objects can be distinguished according to their characteristic extreme color, which can be used as landmarks or to identify objects. Therefore, there is an interest in identifying landmarks characterized by extreme colors. In this paper, a new method for detecting objects with an extreme color is introduced and compared with other approaches found in the literature. The methods are analyzed and compared using a color palette in which a transition between R, G, B, C, M and Y colors is generated. The results obtained show that the methods studied allow the specific colors to be adequately discriminated, while the proposed method is the only one that allows the full range of extreme colors R, G, B, C, M and Y to be detected, being more selective than the others, by taking practically the areas corresponding to each color separately .

Color is a feature that has been exploited to a great extent in digital image processing, since it is a powerful tool that often facilitates the classification and identification of objects, which can be discriminated based on the large number of appreciable color tones [1] . In the area of computer vision it is common to find problems in which it is required to use the color information to carry out the detection of reference points that allow the tracking and definition of the behavior of objects that present particular characteristics and that are observed through sequences of images, obtained in a controlled environment [2] [3] [4] [5] . On the other hand, in other areas such as agriculture and biology, the need has arisen to use color-based image processing techniques in order to apply them to problems such as the detection of weeds in crops [6, 7] , the classification and study of different types of fruits that present significant changes in their color during the different stages of ripening, or due to the presence of defects or associated pests [8] [9] [10] , counting of organisms [11] , among others.

One of the problems that has become relevant in image studies is associated with the detection and segmentation of landmarks with high color intensity, mainly related to primary colors and their complements, which are used in the definition of color spaces, since there are specific applications, in which the objects of interest are easily distinguishable due to their high saturation in one of the RGB color components (Red-Green-Blue) [2, 6, 7] . Therefore, in the literature there are several techniques used for the detection of landmarks with a high saturation in one of the primary or complementary colors CMY (Cyan-Magenta-Yellow), which are defined as extreme colors [2, 6, 7] .

After an extensive literature review it was found that there is no study on the techniques of detection of RGB extreme colors and their complements. Therefore, in this study we analyze the techniques found in the literature for the detection of the extreme colors that constitute the components of the RGB color space. In addition we introduce a new method for detecting the R, G, B, C, M, Y extreme colors. The exposed techniques are compared using a color table and a photograph, to study the quality of the approached methods and to define the cases in which their use is more appropriate, taking into account the detection ranges of the analyzed extreme colors.

In this work, forty images were employed to evaluate the methods studied. Two of them are shown in Fig. 1 . The first one, shown in Fig. 1(a) , consists of a synthetic colour palette, created with a transition between 0 and 255, which allows the visualization of each of the primary colors and their complements. The second, illustrated in Fig. 1(b) , consists of a 2736 × 1824 pixel photograph, taken in a controlled environment, of several extreme color objects. In addition, three frames from a video, shown in Fig. 4 , were used to demonstrate the properties of the method proposed in this work for object tracking. The photographs were acquired with a Canon 6D camera. In the development of the research, a computer HP-2840, with 15.6GiB memory, Intel Xeon(R) CPU E5-2650v4@ 2.20 GHz × 24, NVIDIA Quadro P600 graphics card and Linux-Ubuntu 16.04LTS operating system was used. The image processing algorithms were written in Java as plugins of the free access software Imagej.

The proposed method starts with the analysis of the selected color channel with respect to the remaining channels in the RGB color space. To detect an extreme color, the channel of interest must be compared against the other two. Thus, if a basic color is extreme its value must be very high compared to the other two channels. For example, if an extreme red colored object is to be detected, the value of the red component must be compared to the values of the green and blue channels, as shown in Eqs. (1) and (2) . If the value of the red channel is close to or equal to 255 and much higher than the values of the green and blue components, it is an extreme color. If the value of red is greater than the other two, but not much greater, it means that red is not extreme. If the value of red is lower than either of the other two components, the pixel hue is not red and at least one of the terms C 1 or C 2 will be negative.

From the above, it is possible to define an equation that combines the terms C 1 and C 2 into one and that allows determining if a color is extreme. Since C 1 and C 2 must be high values, a conjunction is made between these terms, using the fuzzy AND operation, since they are not binary values. This operation is equal to obtaining the minimum value between C 1 and C 2 . In this way, the expression presented in Eq. (3) is found, which allows to determine if the color red is extreme.

Based on the procedure described above, this idea can be extended to the complement colors CYM. In contrast to the previous procedure, the opposite operation is performed. The comparisons seen in Eqs. (1) and (2) are reversed as seen in Eq. (4), obtaining as a result that the green and blue colors of a point are highlighted and that mixing them with the operation And the opposite color cyan is obtained.

Three other methods found in the literature were compared with the proposed technique to evaluate their results. The first method studied, proposed by Arnal [11] , a subtraction is made between the green and blue channels of an image (see Eq. (5)) in an attempt to detect the yellow color characteristic of the Trialeurodes vaporariorum. This method is a simple approximation that is not extended to the other basic colors or their opposites.

In the work of García-Santillán et al. [7] , a method is introduced for the detection of perennial weeds, characterized by their extreme green color, for which the chromatic components of the RGB space are initially normalized, dividing the value of each component by the maximum value of the spectral channel, as shown in Eqs. (6)- (8) .

The value of each pixel is then normalized using Eqs. (9)-(10) so that the sum of the components of the normalized channels is equal to 1, as shown in Eq. (12) .

Once the color components of each pixel have been normalized, objects characterized by an extreme green color can be segmented using the Eq. (13). This method requires empirically finding a threshold in the resulting image above which a color is considered extreme. This method can be used for the other colors in the RGB color space by replacing the first term of Eq. (9) with the color to be highlighted.

In the work described by Forero et al. [6] a method is presented which, like the previous one, seeks to find the extreme green color of perennial weeds in cereal crops. For this purpose, the monochromatic image obtained with the BT.601 standard is subtracted from the green channel, as shown in Eq. (14). This method, unlike the previous ones, does not require the search for a threshold value for the detection of extreme color since a fixed value can be used. It can also be used to detect the other two colors in the RGB color space, as was used in the work of García-Vanegas et al. [2] for the detection of extreme red colored landmarks.

The color palette shown in Fig. 1. (a) was used to compare the results obtained with each method, which are presented in Fig. 2 . As seen in Sect. 3, only the method proposed here can be used to detect the R, G, B, C, Y, M extreme colors. Figure 3 presents the results obtained with the photograph covering several of the extreme colors presented in this work, which is studied to show the performance of the methods studied. Fig. 2 . Images obtained with the extreme color detection methods. The columns correspond to the methods studied, the rows correspond to the color channels R, G, B, C, M, Y respectively. Column (1) corresponds to the proposed method, column (2) to the technique used by Forero et al. [6] , Column (3) to the method introduced by García-Santillán et al. [7] using the IsoData method [12] to find an automatic threshold value and Column (4) with manual threshold adjustment R(2;5), G(2;8), B(2;8), column (5) to the method presented by Arnal [11] . The empty boxes correspond to cases where the corresponding method does not provide for the detection of that color. (Color figure online) Fig. 3 . Images obtained from the photo using the extreme color detection methods described in the text. The columns correspond to the methods studied; the rows correspond to the color channels R, G, B, C, M and Y respectively. Column (1) corresponds to the proposed method, column (2) to the technique used by Forero et al. [6] , Column (3) to the method introduced by García-Santillán et al. [7] using the IsoData method [12] to find an automatic threshold value and Column (4) with manual threshold adjustment R(0;0), G(2;8), B(0;0), column (5) to the method presented by Arnal [11] . The empty boxes correspond to cases where the corresponding method does not provide for the detection of that color. (Color figure online) Table 1 shows the average execution times obtained with each method over a total of 20 tests. García-Santillán et al. method [7] using the IsoData threshold adjustment 1.696

García-Santillán et al. method [7] with manual threshold adjustment 1.696

Arnal method [11] 1.462 Figure 4 presents the results obtained in the detection of an extreme yellow object using the method proposed in a video in which the object of study changes its position, lighting and surrounding objects. 

As shown in Fig. 2 , the only method that allows detecting extreme colored objects is the one proposed here, which, in addition, is more effective in detecting pure extreme colors and does not require the selection of a threshold. The method worked out by Forero et al. [6] detected blue shades in the green color and the same is true for the other colors, so this method is less selective. Concerning the method proposed by García-Santillán et al. [7] it can be seen that without the selection of the appropriate threshold the method fails as seen in Column 3, and even if a threshold is empirically selected, the method is not selective with red shades (column 4). With respect to Arnal's method [11] , can be seen in column 5 which is the method that only considers an extreme color and that is not selective for the detection of extreme yellow color leaving tolerant green shades.

In Fig. 3 it can be seen that the proposed method detects the extreme colors for each shade being the most selective of the methods studied here. The method worked out by Forero et al. [6] was less selective with the shades as seen in Column 2, where it detects the extreme green color and lets blue shades through. In the method proposed by García-Santillán et al. [7] it is required to manually find an appropriate threshold to determine the extreme colors, which makes it impractical. As can be seen in Column 5 of Fig. 3 , the method developed by Arnal [11] allows for the detection of extreme yellow color, but also erroneously detects green and red tones as yellow. Table 1 shows that the proposed method is the fastest, taking less than half the time to perform than the other methods studied here. It is evident that the Forero et al. [6] method is the next fastest method and that the García-Santillán et al. [7] and Arnal [11] methods have similar execution times. Figure 4 shows the result obtained with the proposed method in the detection of an extreme yellow object in a video. As can be seen, the proposed method allows the object of interest to be detected under different light conditions and with different objects in the scene.

Extreme colors are used in the detection of landmarks in computer vision techniques. In this work, a new method for detecting extreme colors in images was introduced and compared with three other techniques. The method introduced here is the only one that allows the detection of extreme colors in the 6 channels R, G, B, C, Y, M, and it is also the most selective method in the detection of extreme colors compared to the other methods analyzed. It should be noted that the method does not require the adjustment of a threshold for each channel. The proposed method is a very fast one, with similar or even higher speeds than other techniques proposed previously, and can be used in computer vision applications for the detection of extreme colored objects in real time.

Digital Image Processing

Automatic vision based calibration system for planar cable-driven parallel robots

Mean shift tracking with multiple reference color histograms

An unmanned ground vehicles experimental setup for image-based object tracking

Bio-inspired perception sensor (BIPS) concept analysis for embedded applications

Color classification methods for perennial weed detection in cereal crops

On-line crop/weed discrimination through the Mahalanobis distance from images in maize fields

Immature green citrus fruit detection using color and thermal images

Heredities on fruit color and pigment content between green and purple fruits in tomato

Banana detection based on color and texture features in the natural environment

Using digital image processing for counting whiteflies on soybean leaves

Picture thresholding using an iterative selection method