doi:10.1016/j.eswa.2006.01.028 www.elsevier.com/locate/eswa Expert Systems with Applications 32 (2007) 674–686 Expert Systems with Applications Expert system for low frequency adaptive image watermarking: Using psychological experiments on human image perception Ho Seok Moon a, Taewoo You b, Myung Ho Sohn b, Hye Soo Kim c, Dong Sik Jang a,* a Industrial Systems and Information Engineering, Korea University, 1, 5-ka, Anam-Dong, Sungbuk-Ku, Seoul 136-701, South Korea b Business Administration, Myongji College, 356-1, Hongeun 3-Dong, Seodaemun-Ku, Seoul 120-776, South Korea c Department of Electronics and Computer Engineering, Korea University, 1, 5-ka, Anam-Dong, Sungbuk-Ku, Seoul 136-701, South Korea Abstract This paper proposes a new system for low frequency adaptive image watermarking based on the statistical data from psychological experiments on human image perception. The new approach can lead to a reduction of degrading the subjective image quality that often occurs when watermark is embedded into low frequency area. In order to reduce the degrading of image quality, the new approach deter- mines the strength of watermark according to local image characteristics such as brightness and contrast. By conducting a behavioral experiment on human image fidelity based on the psycho-visual image association technique, we were able to infer the relationship between the watermark strength and the different levels of image brightness and contrast information. The exact watermark is extracted according to edge characteristics by adopting a so-called edge mask that exploits the coefficients of subbands in the subsampled discrete wavelet transform images. Thus, our new approach does not require original images for watermark. We also show the new approach is practically validated against some standard images. � 2006 Elsevier Ltd. All rights reserved. Keywords: Blind image watermarking; Discrete wavelet transform; Psychological experiments; Human image perception; Edge mask 1. Introduction One of the salient issues in this internet era is that digital contents, such as audio, image, and video, may be copied in an unauthorized manner. This stimulated the development of copy protection technologies like digital watermarking. The digital watermarking is now receiving a widespread attention as a possible solution to the protection applica- tions of intellectual property rights (Cox & Miller, 1997; Dai, Zhang, & Yang, 2003; Lin & Chen, 2000; Swanson, Kobayashi, & Tewfik, 1998). Digital watermarking is defined as a technique of embedding some hidden informa- tion called watermark into digital multimedia without loss of perceptual quality of watermarked data. Once created, 0957-4174/$ - see front matter � 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2006.01.028 * Corresponding author. Tel.: +82 2 925 3655; fax: +82 2 3290 3776. E-mail addresses: bawooi@korea.ac.kr (H.S. Moon), taewooyou@ mail.mjc.ac.kr (T. You), totalsol@mail.mjc.ac.kr (M.H. Sohn), hyesoo@ korea.ac.kr (H.S. Kim), jang@korea.ac.kr (D.S. Jang). the watermark can be detected or extracted for the purpose of owner identification or/and integrity verification of tested data (Levicky & Foris, 2004). In this paper, we pro- pose a new watermarking technique that is not only appli- cable to images, but also easily expandable to other host media. It is generally argued that a watermark should at least meet the following conditions (Chen & Lin, 2003): (1) It should be perceptually invisible (or transparent). (2) It should be difficult to remove without seriously affecting the image quality. (3) It should robustly resist to image distortions caused by attacks such as common image processing opera- tions and lossy image compression. The major objective of this paper is to propose a new approach to low frequency adaptive image watermarking for security applications that can be distinct from others mailto:bawooi@korea.ac.kr mailto:taewooyou@mail.mjc.ac.kr mailto:taewooyou@mail.mjc.ac.kr mailto:totalsol@mail.mjc.ac.kr mailto:hyesoo@korea.ac.kr mailto:hyesoo@korea.ac.kr mailto:jang@korea.ac.kr Fig. 1. Two-level wavelet decomposition. H.S. Moon et al. / Expert Systems with Applications 32 (2007) 674–686 675 in utilizing the statistical data from psychological experi- ments on human image. It is worthwhile to point out three main contributions that can be drawn from the current study. First, the fidelity of images was enhanced through controlling the watermark strength with the data associ- ated with brightness and contrast as local image character- istics, without sacrificing the robustness of images. To our best knowledge, we considered people’s perception on image that can vary by visual characteristics. The key infor- mation on the strength of image brightness and contrast was acquired through an experiment on human being’s image fidelity. Second, we devised a technique that does not require original images by considering the characteris- tics of edge. The embedding and extracting of watermark according to the edges of images was implemented through the so-called edge mask. The term ‘edge mask’ is defined as a mask representing the edge direction of images, say hor- izontal or vertical. The edge mask plays a role as the best predictor of the original image in extracting the watermark. The edge mask is determined by choosing the direction that takes the lower pixel value between the watermark position and its neighborhood pixels in each direction. Third, the experimental results demonstrate that the proposed method is resistant to some image processing operations and to Joint Photographic Experts Group (JPEG) lossy compression, which usually do not degrade the image qual- ity. Fig. 1 illustrates the overview of the proposed system. The organization of the paper is as follows: A survey of existing studies is made in Section 2. Section 3 discusses the embedding method. Section 4 describes the extracting method. Section 5 contains experimental results and appli- cation. Finally, Section 6 draws a conclusion. 2. Literature review Existing literature reveals two techniques for the water- marking of images: frequency domain and spatial domain (Wu & Shih, 2004). Most of the recent watermarking schemes employ mainly the frequency domain approach because it is superior to the spatial domain approach in robustness and stability (Hsu & Wu, 1998; Lin & Chen, 2000; Miu, Lu, & Sun, 2000; Shih & Wu, 2005). However, there is a crucial question that should be answered: which frequency band in frequency domain can be robust and imperceptible to various attacks? According to Weber’s rule, the low frequency area is more robust than high and middle frequency areas (Shi & Sun, 1999). There have been various methods to embed the watermark into the low fre- quency area (Huang, Shi, & Shi, 2000; Joo, Suh, Shin, & Kikuchi, 2002). It is known that these prior approaches are robust to various attacks. Despite their robustness, the existing methods still leave a problem unanswered. The key concern is that, if the watermark is embedded arbi- trarily in the low frequencies, without adapting to local image characteristics, the image quality could be visually degraded. Thus, of importance is the amount of modifica- tions that can be made by watermark embedding in low frequency pixels. This problem can be solved by using the Human Visual System (HVS) model in digital watermark- ing. The HVS model determines the maximal amount of watermark signal that can be tolerated at every location without affecting the visual quality of the image (Barni, Bartolini, & Piva, 2001; Chen & Lin, 2003; Levicky & Foris, 2004). To embed the watermark with a minimal loss in image fidelity, the watermark strength modulation should reflect sufficiently the local image characteristics such as texture, edge presence, and luminance (Taskovski, Bogdanova, & Bogdanov, 2002). In order to determine the optimal strength of water- mark, we should be not only well aware of local image characteristics, but also be knowledgeable on how human beings feel the fidelity of image visually. The sensitivity of human being to noise strength does not remain constant, but changes according to the surround modulation charac- teristics of noise. The brightness difference required to detect a noise in a particular area increases with the bright- ness of the area near the noise (i.e., watermark embedded) (Stiles, 1978; Wyszecki & Stiles, 1982). In other words, the probability of not distinguishing noise increases as the average brightness rises. Contrast also affects the possibil- ity of detecting noise. Even in the case that the mean brightness is fixed at noise surroundings, the perceived con- trast of the noise embedded at center tend to weaken (Chubb, Sperling, & Solomon, 1989; Olzak & Laurinen, 1999). Thus, we can predict that the strength of watermark is less likely to be perceived by human beings as the level of brightness and contrast increases. Such a prediction indi- cates that the impact of embedded watermark on the fidel- ity of images can vary by the level of brightness and contrast. Therefore, it is less likely that the images with higher level of average brightness and contrast are dis- cerned even at the same strength of watermark embedded. Also, watermarking techniques can be divided into two distinct categories depending on the necessity of original images for the watermark extraction. Although the exis- tence of the original image may facilitate the watermark extraction to a certain extent, two problems can come out: (1) At the risk of insecurity the owners of original images may be compelled to share their work with anyone who wants to check the existence of the watermark and (2) 676 H.S. Moon et al. / Expert Systems with Applications 32 (2007) 674–686 it is time-consuming and cumbersome to search out the originals within the database that correspond to a given watermark. Thus, in order to overcome these problems we need a method for extracting the embedded watermark without requiring the original image (Chen & Lin, 2003). This method is called a blind watermarking technique. The blind watermarking techniques appear far more useful since the availability of an original image is usually unwar- ranted in real-world scenarios (Wang & Pearmain, 2004). As a blind watermarking technique, Lin and Chen (2000) proposed a method in which binary watermark is embedded directly into the Lease Significant Bit (LSB) of Discrete Cosine Transform (DCT) coefficients in an image. This method is not robust because the LSB value of coeffi- cients can be easily forged from various attacks of image processing. Chu (2003) and Wang and Pearmain (2004) suggested a blind watermarking technique that can estimate the pixels of an original image on the supposition that adjacent pixels in the image were similar in a gray level. Chu (2003) intro- duced a method using a random perturbation to subimages obtained by subsampling the image. To extract the water- mark, this method selects randomly a horizontal or vertical subimage and exploits it to estimate an original image. Wang and Pearmain (2004) obtained the estimated pixel value by computing the average of adjacent values (for example, the pixel values in the mask size of 3 · 3, 5 · 5, 7 · 7) of an original image pixel. Though adjacent pixels at gray levels are generally alike in image natures, some regions that can be subject to abrupt changes of pixel values such as edge areas cannot be estimated correctly because of the discrepancy between the value of an original pixel and an estimated pixel. In Chu’s method (2003), when the original image pixel is esti- mated with a pixel of a horizontal (or vertical) subimage, regions with horizontal (or vertical) edges can still be a problem. Also, the method of Wang and Pearmain (2004) Fig. 2. The overall watermarkin that uses average information in the mask can cause false estimations in the edge regions. In addition, their methods do not account for the characteristics of images in deter- mining the strength of watermark. The results of overall literature review can be summa- rized in two-fold: First, the fidelity of images should be enhanced through controlling the watermark strength with the data associated with local image characteristics, without sacrificing the robustness of images. Second, the blind watermarking is more effective than non-blind watermarking. But without considering the edge of image, the extracting process will not be exact. So, the edge of image is important to estimate the original image in blind watermarking. This paper attempted to solve these problems. 3. The embedding method Our embedding strategy is based on a Discrete Wavelet Transform (DWT). In the DWT, an image is first decomposed into four sub- bands, LL1, LH1,HL1 and HH1(L: i.e., low, H: i.e., high) by cascading horizontal and vertical two-channel critically subsampled filter banks (Chen & Lin, 2003). Each entry in subbands LH1, HL1 and HH1 represents the finest scale wavelet coefficients. To obtain the next coarser scale of wavelet coefficients, the subband LL1 is further decom- posed and critically subsampled. The process continues until some final scale is reached. Fig. 2 shows the image decomposed into seven subbands for two scales. The lowest frequency subband is at the top left corner and the highest frequency subband is at the bottom right corner. Thus, in our implementation, we modified the wavelet coefficients selected from subband LL2. In the proposed watermarking method, the original image is a gray-level image of size N1 · N2 and the digital watermark is a binary image of size M1 · M2. g procedure of the system. H.S. Moon et al. / Expert Systems with Applications 32 (2007) 674–686 677 Given the image X[n1, n2], n1 = 0, . . . ,N1 � 1, n2 = 0, . . . , N2 � 1, then X 1½k1; k2� ¼ X ½2n1; 2n2�; X 2½k1; k2� ¼ X ½2n1 þ 1; 2n2�; X 3½k1; k2� ¼ X ½2n1; 2n2 þ 1�; X 4½k1; k2� ¼ X ½2n1 þ 1; 2n2 þ 1� ð1Þ for k1 = 0, . . . ,N1/2 � 1, k2 = 0, . . . ,N2/2 � 1 are four subimages obtained by subsampling. These are trans- formed via DWT to obtain sets of coefficients Yi[k1, k2], i = 1, 2, 3, 4. Since the subimages Xi’s are highly correlated, it is expected that Yi � Yj, for i 5 j except edge (Chu, 2003). The embedding and extracting methods are based on comparisons among the DWT coefficients of the four DWT subimages obtained by subsampling. By making dif- ferent modifications to the DWT coefficients pertaining to different subimages, the watermark without comparison with the original image is extracted. Our overall embedding steps are described in Fig. 3. The algorithm works according to the following steps. 3.1. DWT of the subimage generated by decomposing the original image The original image is decomposed into four subimages according to Eq. (1). Each of the four subimages is trans- Fig. 3. Watermark e formed via two-level DWT to obtain four LL2 subbands, one of which is selected for embedding a watermark. 3.2. Making edge mask The edge mask is used to consider the edge of local image characteristics. The clone mask is constructed according to the pixel-wise relationship between the selected subband and neighboring two subbands. If the LL2 subband of Y1 is selected to embed the watermark, the edge mask is constructed based on the pixel-wise rela- tionship between the selected band and the neighboring two LL2 subbands of Y2 and Y3. In general, DWT subimages have approximately same coefficients at the same spatial location except edge areas. Thus, after embedding a watermark into one of the sub- images, we can easily extract the watermark later by com- paring the watermarked subimage with the rest of the subimages. Thus this method does not need an original image for extracting the watermark. Once we select a subimage to embed the watermark, we have to select a subimage to compare for later watermark extraction. Selecting just a horizontal or vertical neighbor- ing subimage for comparison purpose will be problematic because after DWT there are always some differences between the images around the edges. Thus, to deal with the problem, the proposed method uses an edge mask as in Fig. 4(a). For example, Fig. 4(a)–(d) shows how to select mbedding steps. Fig. 4. Selecting neighbor subimages and an edge mask: (a)–(d) subimage selection (a gray block is a subimage to be watermark-embedded and the two white blocks enclosed in dashed line are neighboring subimages), (e) the example of an edge mask. 678 H.S. Moon et al. / Expert Systems with Applications 32 (2007) 674–686 neighboring subimages (white blocks enclosed in the dashed line) after selecting a subimage to be embedded (gray block) and Fig. 4(e) shows one example of an edge mask. In Fig. 4, H (i.e., horizontal) means the coefficient of a selected subband Y1 at that location is more similar to that of Y2 than to that of Y3, whereas V (i.e., vertical) means the coefficient at that location is more similar to that of Y3 than to that of Y2. In the procedure, it should be noted that, to strengthen ownership, we can have the degree of freedom of which the subimage is selected for embedding the watermark. Thus, selecting a subimage can be used as an author key to preventing a hacker’s attack. 3.3. Pixel-based pseudo-permutation of the watermark In this paper, the binary image of size M1 · M2 of our laboratory logo was used as a watermark. In order to dis- perse the binary pattern on a space, pseudo-random per- mutation is performed as follows: (1) number each pixel from zero to M1 · M2, (2) generate each number in random order, and (3) generate the coordinate pairs by mapping the random sequence number into a 2-D sequence. 3.4. Determining the watermark strength according to image characteristics As mentioned earlier, this paper exploits subjective psy- chological factors when determining watermark strength, based on the psychological presumption that human beings tend to exhibit different sensitivity to noise by levels of brightness and contrast. In order to prove this proposition, we conducted an experiment under quasi-environments. The aim of the experiment is to see how human being’s sen- sitivity to noise, i.e., the strength of watermark that can be detected by human beings, changes by different levels of brightness and contrast. We provided eight graduate stu- dents with different combinations of watermark strength, brightness, and contrast. And we recorded the level of watermark strength they can marginally recognize the dif- ference between original images and watermarked images. In our experiment, we analyzed the brightness and con- trast for about 1200 images in the image database of Corel Draw. Each image was transformed into gray images with 256 intensity levels, and then the mean brightness and mean contrast were calculated by applying Eqs. (2) and (3) to each pixel value. As a result, the average brightness and contrast were 113.21(±42.56) and 56.82(±18.03). The levels of image brightness were classified as three: average level (115), average �1 standard deviation (72), and aver- age +1 standard deviation (158). The levels of contrast were grouped in the same way: average (57), average �1 standard deviation (39), and average +1 standard deviation (75). Mean ðlÞ¼ 1 N 1N 2 XN 1�1 i¼0 XN 2�1 j¼0 X ij ð2Þ Contrast ðSÞ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPN 1�1 i¼0 PN 2�1 j¼0 ðX ij � lÞ 2 N 1N 2 � 1 s ð3Þ Moon, Sohn, and Jang’s (2004) proposed method is used as watermark embedding method in this experi- ment. The watermark is embedded into lowest resolu- tion representation (LL2) of the original image obtained after applying two-level wavelet decomposition in this method. In this experiment, we used conventional images such as Airplane, Lena, and Baboon. We generated nine combina- tions of images by the three levels of brightness and con- trast. Watermark image was generated by embedding different strength-of-noise (T). By increasing T from 1 to 15 by one, we made watermark images with differing strength. We limited the number of images to three, because the objects of experiments, if they are exposed to too many images, may be distracted, thus result in a distortion of the experimental results. The choice of three images in experiment is better applicable than others as we can verify in Section 5 where the experiment were applied to various 12 images. Fig. 5 shows the images of Lena varying accord- ing to various combinations of contrast and brightness in this experiment. All images were presented on 21 inch monitors at 512 · 512 pixels. Students were allowed to watch the mon- itors at the distance of 50 cm. The participants were unable to recognize what kinds of noise are embedded in images since they did not have any prior experience in the water- marked images. Participants were asked to tell water- marked images from non-watermarked images. Each participant estimated the strength-of-noise 18 times (6 times per an image) under nine combinational con- ditions derived from brightness and contrast. The average of the values was used as the strength measure of noise. The analysis of variance in Fig. 6, demonstrates that the strength-of-noise increases as the mean brightness rises (F(2, 14) = 87.86, p < 0.001) and as the mean contrast rises (F(2, 14) = 103.38, p < 0.001). However, there was no sig- Fig. 5. Lena images used in psychological experiments. Fig. 6. Predicted watermark strength according to the mean brightness and contrast of image. H.S. Moon et al. / Expert Systems with Applications 32 (2007) 674–686 679 nificant interaction effect between brightness and contrast (F(4, 28) = 1.01, p > 0.05). The experiment shows that the sensitivity of human being to noise could be the same though levels of strength-of-noise are physically different. In other words, though the values of peak signal-to-noise ratio (PSNR) based on physical difference of images are the same, the perceived watermark strength can be different across the brightness and contrast of image. Therefore, the water- mark strength should be adjusted properly by the bright- ness and contrast of image. Using the ordinary least square (OLS) method we regressed the mean brightness (l) and the mean contrast (S) on the perceived watermark strength (T). The result of the OLS estimation is summarized in Eq. (4). We present the estimation results using the whole sample correspond- ing to our three raw images. It appears that both the mean brightness and the mean contrast are significantly related to watermark strength. The unit change in brightness leads to 0.025 change in watermark strength, while one unit change in contrast results in 0.114 changes in watermark strength. In this paper we used the estimated regression Eq. (4) to determine the watermark strength. The numbers in parenthesis are t statistics: Watermark strength ðTÞ¼ 2:880 þ 0:025l þ 0:114S ð3:10Þ ð4:54Þ ð7:41Þ R2 ¼ 0:35 ð4Þ 3.5. Modification of coefficients in the selected subimage using edge mask By modifying the coefficients of LL2 based on the edge mask and watermark, we embed the watermark into the LL2 subband of the selected subimage. Fig. 7 shows exam- ples of the embedding procedure, where (a)–(c) represent the LL2 subband of Y1 and two neighboring LL2 subbands of Y2 and Y3, respectively. Fig. 7(d) is the edge mask obtained from (a) to (c). Fig. 7(e) is a permuted binary watermark with size i = 0, . . . , M1 � 1, j = 0, . . . , M2 � 1. If we change the coefficients of LL2 subband of Y1 in accordance with the edge mask and watermark, we come up with four cases: (1) the associated pixels of edge mask and watermark have H and 1, (2) the associated pixels of edge mask and watermark have H and 0, (3) the associated pixels of edge mask and watermark have V and 1, and (4) the associated pixels of edge mask and watermark have V and 0. In each case, the coefficients of LL2 subband of Y1 are modified by Eqs. (5) and (6). Here, Y01 is the LL2 sub- band after the watermark W is embedded, and T is water- mark strength to be determined according to Eq. (4). Fig. 7. Embedding example: (a) The LL2 subband of Y1(Y1(i, j)). (b) The LL2 subband of Y2(Y2(i, j)). (c) The LL2 subband of Y3(Y3(i, j)). (d) Edge mask (S(i,j)), (e) Watermark (W(i, j)). (f) Modified coefficients (Y 01(i, j)). 680 H.S. Moon et al. / Expert Systems with Applications 32 (2007) 674–686 If Sði; jÞ¼ H and W ði; jÞ¼ 1 Y 01ði; jÞ¼ Y 1ði; jÞ if Y 1ði; jÞ > Y 2ði; jÞ Y 2ði; jÞþ T otherwise � Else if Sði; jÞ¼ H and W ði; jÞ¼ 0 Y 01ði; jÞ¼ Y 2ði; jÞ� T if Y 1ði; jÞ > Y 2ði; jÞ Y 1ði; jÞ otherwise � ð5Þ Else if Sði; jÞ¼ V and W ði; jÞ¼ 1 Y 01ði; jÞ¼ Y 1ði; jÞ if Y 1ði; jÞ > Y 2ði; jÞ Y 3ði; jÞþ T otherwise � Else if Sði; jÞ¼ V and W ði; jÞ¼ 0 Y 01ði; jÞ¼ Y 3ði; jÞ� T if Y 1ði; jÞ > Y 3ði; jÞ Y 1ði; jÞ otherwise � ð6Þ The basic rule of Eqs. (5) and (6) is that, if W(i, j) = 1, then Y1(i, j) should be higher than a subband to be compared. Otherwise, Y1(i, j) should be modified to have a higher value. If W(i, j) = 0, Y1(i, j) should be lower than a com- pared subband. If higher, it should be modified to have a lower value. For example, consider the circled pixel in Fig. 7. The edge mask takes H and the watermark takes Fig. 8. Watermark e 1. In this case, the horizontal subimage Y2 is similar to Y1. Then Y1(0, 0) and Y2(0, 0) are compared. Because Y1(0, 0) is lower than Y2(0, 0),Y1(0, 0) should be changed to have higher value than Y2(0, 0) by the amount of strength T according to Eq. (5). In order to explain embed- ding method easily, we set T value to ‘6’ in this example. 4. The extraction method Fig. 8 shows the overall extraction steps. The detailed steps are as follows. 4.1. Decomposing watermarked image and DWT The watermarked image is decomposed into four sub- images by subsampling. then each subimage is transformed via two-level DWT. 4.2. Extracting the watermark using the edge mask There are two steps in this process. First, we select a watermarked subimage and two neighboring subimages. Second, we extract a watermark by performing a pixel-wise xtraction steps. Fig. 9. Extraction example: (a) The LL2 subband of a watermarked subimage Y 0 1 (i, j). (b) The LL2 subband of Y2(i, j). (c) The LL2 subband of Y3(i, j). (d) Edge mask S(i, j). (e) Extracted watermark W*(i, j). H.S. Moon et al. / Expert Systems with Applications 32 (2007) 674–686 681 comparison among the images. More specifically, if the coefficient of a watermarked subimage is higher than that of a neighboring subimage, the watermarked pixel at the same location is set to ‘1’, otherwise its value is set to ‘0’ (see Eqs. (7) and (8)). This is the reverse procedure of the embedding scheme described in the previous section. For example, Fig. 9 shows one extraction procedure where (a) is the LL2 subband (Y 0 1) of the watermarked subimage. For the circled pixel at (0, 0) in (a), the associated pixel of the edge mask has H, see (d). Thus, the coefficient of a horizontal neighboring subimage (i.e., Y2(0, 0) = 213) is selected for comparison. Because the coefficient of Y 01(0, 0) is higher than that of Y2(0, 0), the watermark pixel at (0, 0) is set to ‘1’. If Sði; jÞ¼ H W �ði; jÞ¼ 1 if Y 01ði; jÞ > Y 2ði; jÞ 0 otherwise � ð7Þ Else if Sði; jÞ¼ V W �ði; jÞ¼ 1 if Y 01ði; jÞ > Y 3ði; jÞ 0 otherwise � ð8Þ 4.3. Reversing the permutation and similarity measurement Next process is to reverse the pseudo-random permuta- tion for extracting a visually recognizable watermark according to the predefined pseudo-random order. The extracted watermark is a visually recognizable pat- tern. Subjectively, the viewer compares the result with the referenced watermark. However, the subjective measure- ment is dependent on factors such as the expertise of the viewers and the experimental conditions, etc. Thus, as a proxy for similarity measure we use the normalized corre- lation (NC) between the extracted watermark, W*[i, j] and the referenced watermark W [i, j] given in Eq. (9), where M1 and M2 are the horizontal and vertical sizes of the watermark image, respectively. NC ¼ PM 1�1 i¼0 PM 2�1 j¼0 W ½i; j�W �½i; j�PM 1�1 i¼0 PM 2�1 j¼0 W ½i; j�W ½i; j� ð9Þ To establish a more quantitative measure of impercepti- bility, we make use of the PSNR metric. Although this measure is generally not very accurate, it serves as a good rule of thumb of the invisibility of the watermark. The PSNR is defined as Eq. (10) in units of dB, where X is the original image, X* is the watermarked image, and NT is the number of pixels in X. In general, if the PSNR value is greater than 35 dB then the perceptual quality is accept- able, i.e., the watermark is almost invisible to human eyes (Chen & Lin, 2003). PSNR ¼ 10 log 255 � 255 1 N T PN T �1 i¼0 ðX i � X � i Þ ð10Þ 5. Experimental results and application Fig. 10 shows an example of embedding and extracting results for images of ‘Lena’. In our experiment, 12 standard gray images (Airplane, Bridge, Baboon, Cameraman, Church, Girl, Girl2, House, Lena, Man, Man2, Peppers), as shown in Fig. 11, are used as original images. All of them were sized 512 · 512. Watermarks are embedded in these images. The pattern with ‘I I S LAB’, sized 64 · 64, was used as a watermark. The performance of the proposed method was evaluated in terms of the imperceptibility and robustness against various attacks: (1) the decision of watermark strength regarding image local characteristic, (2) the validity of edge mask, (3) image processing opera- tion (image contrast and histogram equalization), and (4) JPEG compression. Fig. 10. Example of the proposed watermarking approach: (a) test image, (b) watermark, (c) watermarked image (with PSNR = 40.35 dB), (d) extracted watermark with NC = 1.0. Fig. 11. Standard images used in our experiment. 682 H.S. Moon et al. / Expert Systems with Applications 32 (2007) 674–686 5.1. Decision of watermark strength regarding image local characteristic We conduct an experiment on how the estimates from Eq. (4) for various levels of brightness (mean) and contrast (standard deviation) affect the imperceptibility of images. We compute the strength of watermark by partitioning an image into equal slices, calculating the mean and standard deviation of each partition, and plugging the data into (4). The partitioning of image is made on various sizes of partition, as shown in Table 1. Image should not be over- lapped by partitioning. Table 1 reports the PSNR of water- marked images calculated from different strength of watermark according to local image characteristics. That is, the strength of watermark does not remain con- stant, but it is changed based on Eq. (4). The image mean and standard deviation entries in Table 1 represent those for un-partitioned 512 · 512 images. Table 1 shows that, after our proposed method is applied on various mean and standard deviations of images, the subjective perceived image quality was enhanced, and the PSNR stayed high at 40 dB. We also changed the size of partitions, but the results did not change significantly. However, we should note that the quality of image can be degraded, if the size of partition is too small. Thus, it is unnecessary that we partition an image into too small pieces and apply the mean and standard deviations of the partition to the equation. Table 1 Imperceptibility results (PSNR) according to different segmented image size Image PSNR according to segmented size Image mean Standard deviation 64 · 64 32 · 32 16 · 16 8 · 8 4 · 4 1 38.73 38.64 38.54 38.36 38.13 178.4 46.5 2 39.22 39.11 39.04 38.92 38.82 132.8 41.7 3 40.52 40.14 40.07 39.79 39.52 113.8 52.88 4 40.55 40.12 40.11 40.45 39.79 118.7 62.2 5 40.41 40.19 40.08 39.91 39.58 121.4 56.9 6 39.4 39.06 38.92 38.71 38.56 132.3 67.9 7 42.28 42.07 41.7 41.49 41.27 73.6 42.6 8 40.38 40.09 40.14 39.97 39.75 138.1 46.7 9 40.41 40.21 40.08 40.02 39.68 125.2 45.2 10 40.88 40.39 40.13 40.01 39.75 117.8 69.8 11 40.61 40.47 40.33 40.04 39.78 111.4 47.4 12 39.78 39.64 39.39 39.18 39.04 129.9 51.9 H.S. Moon et al. / Expert Systems with Applications 32 (2007) 674–686 683 5.2. The validity of edge mask In watermarking method considering the relationship of neighboring pixels, if the watermark is embedded into area that the values of neighboring pixels are considerably dif- ferent (edge area), it is obvious to degrade image quality because of embedding more strong watermark into that area. In order to prevent degrading image quality, water- mark should not be embedded into that area generally. But this method resulted in inexact watermark extraction. In this paper, we solve the problem through edge mask. Because we embed and extract watermark according to edge mask, we can select neighboring pixel that is not edge Fig. 12. The validity of the edge mask, (a), (d), (g) and (j) are watermarked ima (c), (f), (i) and (l) are extracted watermarks using edge mask. for embedding and extracting. Also, we can prevent severely degrading image quality and extract exact watermark. Fig. 12 shows the validity of edge mask. Fig. 12(a), (d), (g), and (j) are watermarked images with PSNR 40 dB. Fig. 12(b), (e), (h), and (k) are extracted watermarks with- out the edge mask. Here, original pixels were uni-direction- ally estimated with a horizontal or vertical direction. As expected, the quality of extracted watermarks is poor, because estimation cannot be accurate around image edges. Contrary, Fig. 12(c), (f), (i), and (l) are extracted water- marks with edge mask. Since original pixels can be esti- mated with edge mask that reflects image characteristics, ges, (b), (e), (h) and (k) are extracted watermarks without using edge mask, 684 H.S. Moon et al. / Expert Systems with Applications 32 (2007) 674–686 the quality of extracted watermarks is better than the pre- vious cases. Fig. 14. The extracted watermarks of the JPEG compressed version of image number 6: (a) NC = 96.17 for compression ratio 2.83, (b) NC = 95.07 for compression ratio 4.68, (c) NC = 92.85 for compression ratio 5.33, (d) NC = 86.85 for compression ratio 6.85. Table 3 5.3. Image processing operation Fig. 13 and Table 2 show the results of applying the con- trast and histogram equalization to the watermarked image. Table 2 contains the results for the partitioned area by different size of partitions. It is shown that the results are robust to the contrast and histogram equalization attacks regardless of the size of partitioning. Watermark extraction results (NC) after applying JPEG compression Image number NC results according to JPEG compression ratios 2.83 4.68 5.33 6.85 1 88.31 86.5 83.5 59.99 2 85.44 74.51 69.53 62.74 3 95.61 94.53 91.72 83.22 4 91.58 90.7 85.18 76.29 5 94.55 93.9 91.97 83.84 6 96.17 95.07 92.85 86.85 7 93.43 91.41 81.84 67.26 8 89.58 89.33 86.82 79.54 9 87.72 85.91 82.71 77.37 10 91.7 91.36 86.89 77.81 11 94.53 93.7 90.45 80.98 12 86.43 81.05 78.9 74.02 5.4. JPEG lossy compression and application Fig. 14 and Table 3 show the extracted watermarks from the JPEG compressed version at various compression ratios and the corresponding NC values of the extracted watermarks. In this experiment we employed the mean and standard deviation from 16 · 16 partition. Obviously our proposed method also appears robust to the high level of compression. Fig. 15 shows the Graphic User Interface (GUI) which was used in the proposed system. The programs have been implemented in Visual C++. Fig. 13. Watermark extraction results after applying contrast and histogram equalization: (a) contrast enhanced version of Fig. 7(c), (b) extracted watermark from (a) with NC = 84.28, (c) histogram equalized version of Fig. 7(c), (d) extracted watermark from (c) with NC = 90.50. Table 2 Watermark extraction results (NC) after applying contrast and histogram equalization according to different segmented image size Image number NC results according to watermark extraction Contrast attack Histogram equalization attack 64 · 64 16 · 16 4 · 4 64 · 64 16 · 16 4 · 4 1 88.5 85.16 85.21 88.4 88.4 88.23 2 93.09 92.97 92.85 92.26 91.99 92.46 3 85.22 85.35 85.28 93.55 93.68 93.53 4 70.95 70.95 70.95 86.5 86.55 86.25 5 75.39 75.42 75.07 91.6 91.72 91.65 6 69.14 69.53 69.58 93.12 93.04 92.9 7 79.42 79.35 79.35 88.6 88.75 88.99 8 80.86 80.88 80.66 85.45 85.6 85.55 9 84.28 84.18 84.35 90.5 90.58 91.11 10 60.57 60.52 60.52 86.11 86.13 86.79 11 83.13 83.2 83.23 91.4 91.09 91.55 12 83.37 83.35 83.33 93.41 93.41 93.27 Fig. 15. The GUI of the proposed system. H.S. Moon et al. / Expert Systems with Applications 32 (2007) 674–686 685 6. Conclusion This paper proposes a new approach to low frequency adaptive image watermarking based on the statistical data from psychological experiments on human image percep- tion. The subjective image quality tends to be degraded when watermark is embedded into low frequency area. In order to reduce the degrading of image quality, we devised a new approach that can determine the strength of water- mark according to local image characteristics such as brightness and contrast. The relationship between the watermark strength and the different levels of image bright- ness and contrast was statistically established based on a behavioral experiment on human image fidelity using the psycho-visual image association technique. Also, we extracted the exact watermark according to edge character- istics by adopting a so-called edge mask that exploits the coefficients of subbands in the subsampled DWT images. Our new approach is distinct in that the original images are not required for watermark. By experiments to some standard images, we showed the new approach is practi- cally validated. The potential contributions of the paper are summarized as follows. First, the fidelity of images was remarkably enhanced through controlling the watermark strength with the data associated with brightness and contrast as local image characteristics, without sacrificing the robustness of images. In this scheme we took into account people’s perception on image that can vary by visual characteristics. The key information on the strength of image brightness and contrast was acquired through an experiment on human being’s image fidelity. Second, we devised a technique that does not require the original images by con- sidering the characteristics of the edge. Third, the experi- mental results demonstrate that the proposed method is robust to various image processing operations and to JPEG lossy compression and does not degrade the image quality. References Barni, M., Bartolini, F., & Piva, A. (2001). Improved wavelet-based watermarking through pixel-wisa masking. IEEE Transaction on Image Processing, 10(5), 783–791. Chen, L. H., & Lin, J. J. (2003). Mean quantization based image watermarking. Image and Vision Computing, 21, 717–727. Chu, W. C. (2003). DCT-based image watermarking using subsampling. IEEE Transaction on Multimedia, 5(1), 34–38. Chubb, C., Sperling, G., & Solomon, J. A. (1989). Texture interactions determine perceived contrast. Proceeding of the National Academy of Sciences, 86, 9631–9635. Cox, I., & Miller, M. L. (1997). A review of watermarking and the importance of perceptual modeling. Proceedings of Electronic Imaging, 3016, 92–99. Dai, Y. J., Zhang, L., & Yang, Y. X. (2003). A new method of MPEG video watermarking technology. Proceeding of IEEE ICCT, 2, 1845–1847. Hsu, C. T., & Wu, J. L. (1998). Multiresolution watermarking for digital images. IEEE Transaction on Circuits and Systems, 45(8), 1097–1101. Huang, J., Shi, Y. Q., & Shi, Y. (2000). Embedding image watermarks in DC components. IEEE Transaction on Circuits and Systems for Video Technology, 10(6), 974–979. Joo, S. H., Suh, Y. H., Shin, J. H., & Kikuchi, H. (2002). A new robust watermark embedding into wavelet DC component. ETRI Journal, 24(5), 401–404. Levicky, D., & Foris, P. (2004). Human visual system models in digital image watermarking. Radio Engineering, 13(4), 38–43. Lin, S. D., & Chen, C. F. (2000). A robust DCT-based watermarking for copyright protection. IEEE Transaction on Consumer Electronics, 46(3), 415–421. Miu, X. M., Lu, Z. M., & Sun, S. H. (2000). Digital watermarking of still images with gray-level digital watermarks. IEEE Transaction on Consumer Electronics, 46(1), 137–145. Moon, H. S., Sohn, M. H., & Jang, D. S. (2004). DWT-based image watermarking for copyright protection. In AIS 2004 (pp. 490–497). 686 H.S. Moon et al. / Expert Systems with Applications 32 (2007) 674–686 Olzak, L. A., & Laurinen, P. I. (1999). Multiple gain control processes in contrast–contrast phenomena. Vision Research, 39, 231–256. Shi, Y. Q., & Sun, H. (1999). Image and video compression for multimedia engineering: Fundamentals, algorithms, and standards. Boca Raton, FL: CRC Press. Shih, F. Y., & Wu, Y. T. (2005). Enhancement of image watermark retrieval based on genetic algorithms. Journal of Visual Communication and Image Representation, 16, 115–133. Stiles, W. S. (1978). Mechanisms of color vision. London: Academic Press. Swanson, M. D., Kobayashi, M., & Tewfik, A. H. (1998). Multimedia data embedding and watermarking technologies. Proceeding of the IEEE, 86(6), 1064–1087. Taskovski, D., Bogdanova, S., & Bogdanov, M. (2002). Improved low frequency image adaptive watermarking scheme. In ICSP’02 (pp. 26– 30). Wang, W., & Pearmain, A. (2004). Blind image data hiding based on self reference. Pattern Recognition Letters, 25, 1681–1689. Wu, Y. T., & Shih, F. Y. (2004). An adjusted-purpose digital watermark- ing technique. Pattern Recognition, 37(12), 2349–2359. Wyszecki, G., & Stiles, W. (1982). Color science: Concepts and methods, quantitative data and formulae. New York: Wiley. Expert system for low frequency adaptive image watermarking: Using psychological experiments on human image perception Introduction Literature review The embedding method DWT of the subimage generated by decomposing the original image Making edge mask Pixel-based pseudo-permutation of the watermark Determining the watermark strength according to image characteristics Modification of coefficients in the selected subimage using edge mask The extraction method Decomposing watermarked image and DWT Extracting the watermark using the edge mask Reversing the permutation and similarity measurement Experimental results and application Decision of watermark strength regarding image local characteristic The validity of edge mask Image processing operation JPEG lossy compression and application Conclusion References