135_158_____Colin C. Venters.hwp Query by Colour : Investigating the Efficacy of Query Paradigms for Visual Information Retrieval 색에 의한 질의: 시각정보 검색을 위한 질의 패러다임의 유용성 측정 Colin C. Venters* ABSTRACT The ability of the searcher to express their information problem to an information retrieval system is fundamental to the retrieval process. Query by visual example is the principal query paradigm for expressing queries in a content-based image retrieval environment yet there is little empirical evidence to support its efficacy in facilitating query formulation. The aim of this research was to investigate the usability of the query by colour method in supporting a range of information problems in order to contribute to the gap in knowledge regarding the relationship between searchers’ information problems and the query methods required to support efficient and effective visual query formulation. The results strongly suggest that the query method does not support visual query formulation and that there is a significant mismatch between the searchers information problems and the expressive power of the retrieval paradigm. 초 록 탐색자가 정보 요구를 정보검색시스템에 표 하는 능력은 검색과정의 기본이다. 시각 시에 의한 질의는 내용기반 이미지 검색환경에서 질의 표 을 한 요한 패러다임이다. 하지만, 이 방법이 질의 구성의 편의성에 있어 얼마나 효과가 있는지에 한 실험 입증은 아직 미미하다. 이 연구의 목 은 다양한 정보요구와 효율 이고도 효과 인 시각 질의 구성을 지원하기 해 탐색자의 정보요구와 질의 방법 사이에 발생하는 지식 격차를 고려하여 색을 이용한 질의 방식의 사용성을 조사하기 한 것이다. 본 연구결과를 통하여 색을 이용한 질의 방법이 시각 질의 구성을 히 지원하지 못한다는 것과 탐색자의 정보요구와 검색 패러다임의 표 력 사이에 뚜렷한 불일치가 있다는 것을 알 수 있다. Keywords: colour retrieval, content-based image retrieval, image retrieval, query by visual example, query by colour, visual information retrieval, usability 컬러 검색, 내용기반 이미지 검색, 이미지 검색, 컬러에 의한 질의, 시각 정보검색, 유용성 * Assistant Professor, Department of Library and Information Science, Kyungpook National University(venters@knu.ac.kr) ￭ Received : 6 May 2011 ￭ Revised : 24 May 2011 ￭ Accepted : 3 June 2011 ￭ Journal of the Korean Society for Information Management, 28(2): 135-158, 2011. [DOI:10.3743/KOSIM.2011.28.2.135] 136 Journal of the Korean Society for Information Management, 28(2), 2011 1. Introduction Advances in technology have been the catalyst for an unprecedented increase in the volume of institu- tional and personal image data. In the UK, approx- imately three million mammograms are taken and stored each year as part of the Breast Screening Programme with the number estimated to increase to six million per year under new recommendations, which require two mammograms per breast to im- prove breast cancer detection rates (Brady, et. al. 2004). However, while there have been significant technological advances with image data capture and storage, developments in effective image retrieval have not kept pace. To address this, research into the retrieval of image data in the last two decades has focused primarily on the use of visual properties and characteristics; commonly referred to as con- tent-based image retrieval. Within this field, a number of key research areas have been identified that are advocated as crucial to its advancement. Despite a long recognition as a key area of research, query interfaces to support effective information retrieval including their ex- pressive power and the impact of the query on re- trieval performance, especially with real end users, has still to receive serious attention by the research community (Gudivada and Raghaven 1995; Flickner et. al. 1997; Rui, Huang, and Chang 1999; Venters et. al. 2003; Hanjalic, Sebe, and Chang 2006; Datta et. al. 2008). A plethora of content-based image retrieval sys- tems supporting a range of retrieval features and functionality have been reported in the literature (Venters and Cooper 2000; Veltkamp and Tanase 2002). This increasing number of systems is a re- flection of the research activity and growing interest in the field. Despite this, the majority of systems are research prototypes designed to test and demon- strate the performance of a retrieval algorithm. Nevertheless, Jaimes and Chang (2002) state that content-based image retrieval systems should consist of three basic components at the highest level of abstraction: the database, the indexing information, and the user interface. When users’ conduct searches, they interact with the system to express their in- formation requirements through a user interface. They suggest that two important aspects of the user interface are its expressive power and its ease of use; where expressive power relates to what can be expressed using the language and ease of use is concerned with how difficult it is for the user to formulate the desired query using the language. This paper reports the results of an investigation that aimed to explore the usability of the query by colour method from a user satisfaction perspective in supporting a range of information problems. Section two provides the background to the research outlining the retrieval method and query paradigm for colour retrieval, image information seeking be- haviour, and related experimental evaluation studies. Section three outlines the methods and procedures used in the experiment. Section four presents the results that emerged from the experiment. Section five discusses the implications of the results and draws conclusions. Query by Colour : Investigating the Efficacy of Query Paradigms for Visual Information Retrieval 137 2. Research Background The origins of content-based image retrieval lie in the fields of artificial intelligence, computer vision, image processing, pattern recognition, and signal processing. Advancement of the approach has been attributed to the early experiments conducted by Kato (1991) into the automatic retrieval of images by col- our and shape feature. The process involves a direct matching operation between a query image and a database of stored images. For each image a feature vector is computed for its unique visual properties f I f1 I , f 2 I , f 3 I ,... f d I( ) Image Feature Vector where f is the feature vector, I is the representation of the image content, and d is the dimensionality. Similarity is calculated by comparing the feature vector of the query image against the feature vectors of images stored in a database using a distance model such as Euclidean distance. Other commonly used metric distance functions include Kullback-Leibler divergence, Mahalanobis distance, and minkowsky distance. The most common image matching feature characteristics are colour, shape, and texture. A num- ber of review papers have surveyed the feature and matching techniques utilized in the process of con- tent-based image retrieval and readers are directed to those publications for detailed descriptions of the techniques and approaches used (Idris and Panchanathan 1997; Del Bimbo 1999; Lew and Huang 2001; Castelli and Bergman 2002; Feng, Siu, and Zhang 2003). Nevertheless, the ability to provide effective and efficient image retrieval using this ap- proach has proved challenging and its efficacy as a primary retrieval method is open to debate. 2.1 Colour Retrieval Colour retrieval has been a major focus of con- tent-based image retrieval since its inception (Kato 1991) and research into the development of effective retrieval approaches continues unabated despite a lack of empirical evidence regarding its efficacy as a re- trieval mechanism (Hua 2010; Yu, Luo, and Lu 2011). Colour for humans is a visual perceptual property corresponding to the categories called red, green, blue etc., derived from the electromagnetic spectrum of light interacting with the eye, which is sensitive to the spectral energy at particular wavelengths between 380-750 nm. Colour retrieval techniques allow images to be retrieved on the basis of their global or local distribution of colour. Global colour features analyse the overall distribution of both the dominant colour and variations of colour within the entire image, in- dependent of location. Colour histogram is the most commonly used colour feature representation and de- notes the joint probabilities of the intensities of the three colour channels, red R, green G, and blue B (Rui, Huang and Chang 1999). Smith and Chang (1996) defined the colour histogram as hR,G,B[r,g,b]=N.Prob{R=r,G=g,B=b} Colour Histogram 138 Journal of the Korean Society for Information Management, 28(2), 2011 where R, G, and B are the three colour channels and N is the number of pixels in the image . The colour histogram is computed by separating the colours within the image into bins and counting the number of pixels of each colour. Since the number of colours is finite, the three colour channels can be transformed into a single variable histogram. Alternative approaches to colour histogram include colour histogram intersection (Swain and Ballard 1991), colour moments (Stricker and Orengo 1995), and colour sets (Smith and Chang 1996). The posi- tions of the colour values are not significant within global colour retrieval methods and images that have similar chromatic property values regardless of posi- tion are retrieved. To improve the precision of the retrieval results, local colour techniques divide an image into ni sub-blocks and then extract the colour values from each block. Image similarity is based on the position of the colour values. Examples of this approach include quadtree-based colour layout (Lu, Ooi, and Tan 1994), colour tuple histogram (Rickman and Stonham 1996), and colour correlo- gram (Huang et al. 1997). Chromatic-based retrieval is effective in retrieving images containing identical and similar regions or distributions of colour within an image but can result in a high level of false colour positives. Readers are directed to the paper by Smith (2002) for a detailed description of the techniques and approaches used in colour image retrieval. It is suggested that this approach is suitable for in- formation problems where the user is knowledgeable about the chromatic content of the image and where there are large homogenous colour regions within the data. The field of remote sensing provides an exemplar application area where colour retrieval could be effectively applied but has yet to be inves- tigated thoroughly
. Nevertheless, it is suggested that the effectiveness of the method de- pends largely on the users’ ability to express their information problem using purely chromatic features (Del Bimbo 1999). While there is some evidence to support this further exploration is required to test the boundaries of the findings. a. Three Level Classification b. Four Level Classification
False Colour Composite Landsat Data Query by Colour : Investigating the Efficacy of Query Paradigms for Visual Information Retrieval 139 2.2 Query by Visual Example Image retrieval is the process of identifying rele- vant images from a database that satisfies an in- formation need. In the context of this research, an information need is a requirement to find relevant information in response to a searcher’s specific in- formation problem. This is formally expressed as a query Qn using the input language provided by the retrieval system. To provide both structure and access mechanisms to image data, concept-based im- age retrieval queries are expressed as keywords using either controlled or uncontrolled vocabularies sup- ported by the retrieval system. In contrast, con- tent-based image retrieval requires searchers to ex- press their information problems as visual queries. Query by visual example (QVE) is the general term used to describe the overall approach and was first coined by Hirata and Kato (1992) to describe the interaction paradigm of their sketch query tool devel- oped for the Art Museum image retrieval system developed by Kato, Kurita and Shimogaki (1989). Under this umbrella term, a number of query para- digms have been proposed for specifying a query Qn in an image database I. Del Bimbo (1999) and Lew and Huang (2001) distinguish between different methods of QVE including query by browsing, query by icon, query by image, and query by sketch. This research explores the usability of the query by colour method, which is outlined in Section 2.2.1. To initiate a content-based image retrieval search, the user provides, selects or creates the visual repre- sentation of their information need. Smeulders et. al., (2000) defined an abstract query space to represent user interaction in a content-based image retrieval system: Q={IQ,FQ,SQ,ZQ} Abstract Query Space where IQ is the selection of images from the data- base I, FQ is the selection of features derived from the images in IQ, SQ is the similarity function, and ZQ is a set of labels to capture the goal dependent semantics. When no knowledge about past or antici- pated use of the system is available, the initial query space Q0 should not be biased toward specific images or made to make some image pairs a priori more similar than others. Rui and Huang (2001) proposed an alternative model to represent the query . They state that an image object O can be modeled as a function of the image data D, features F, and representations R: O=O(D, F, R) Object Model where D is the raw image data, F is a set of visual features associated with the image object, such as colour, texture or shape, and R is a set of representa- tions for a given feature such as colour histogram or colour moments. However, they suggest that speci- fying object O, feature F, and R representation level weights imposes a dual burden on the user of the retrieval system and its developers. For example, specifying the object level weights U imposes a bur- 140 Journal of the Korean Society for Information Management, 28(2), 2011 den on the user, as it requires them to have compre- hensive knowledge of the visual features used in the retrieval system and how they relate to their information need. Similarly, specifying features and their representation level weights Vi and Wij imposes a burden on system developers who do not know which rij and rijk match users’ perceptions of the image content. In general, the majority of content-based image retrieval systems exemplify the interaction model proposed by Rui and Huang (2001). The models emphasize that user interaction is currently a com- plex interplay between the user and the system. Eidenberger and Breitender (2002) argue that user interfaces for content-based image retrieval systems are overly complicated and that casual users would be overtaxed by the demands of selecting similarity measures, retrieval features, and setting of weights. They suggest that in order to improve the acceptance of future systems, simpler user interfaces are required, proposing a framework for their design as part of the VizIR project; an open-framework to develop a Java API for visual information retrieval. The frame- work includes a description of the visual components and their class structure, and the communications between panels, visual components and the query engines based on the Multimedia Retrieval Mark-up Language (MRML) proposed by Müller et. al., (2000). The proposed framework implements two methods of the QVE paradigm, which they consider are the most intuitive: query by image and query by sketch. While there is considerable merit in their proposal to develop a framework for user interfaces for visual information retrieval, their decision to im- plement the two query methods does not take into account whether these methods support the for- mulation of searcher’s information problems. 2.2.1 Query by Colour Query by colour allows the user to specify the percentage or the percentage and distribution of col- our within the query image: query by colour percent- age and query by colour layout. Percentages of col- our within an image can be expressed by selecting colours from a colour space or by mixing colour components and then specifying the percentage of colour, e.g. fifty percent green and fifty percent white
. Query by colour layout
extends this by allowing the distribution and absolute location of the primary colour features to be specified on a virtual query canvas. For example, the user may specify ni colour values in positions xy on the query canvas. Examples of this approach have been utilised by Flickner et. al., 1995; Colombo, Del Bimbo, and
Query by Colour Percentage Query by Colour : Investigating the Efficacy of Query Paradigms for Visual Information Retrieval 141 Genovesi, 2000; Lai, 2000.
Query by Colour Layout Bird, Elliott, and Hayward (1999) investigated the usability of QBIC developed by IBM at the Almaden Research Centre (Niblack et al. 1993). Twenty-five participants from a variety of back- grounds and with a range of experience took part in the evaluation, including curators, graphic design- ers, photographers, librarians, and IT specialists. Query formulation was restricted to query by internal image and query by colour layout. Participants were required to complete twelve search tasks, ranked approximately in order of difficulty, which involved searching or browsing for a target image. The results of the experiment revealed that participants varied significantly in their ability to formulate a visual abstraction of their information requirement. The most common expression of this was a tendency to include too much colour detail, in the form of small blocks, which significantly affected the re- trieval performance of the system. One of the most interesting findings was that participants demon- strated remarkably little irritation with the retrieval performance of the system when the retrieval results deviated from their expectations, blaming their own ability to formulate an accurate representation of the information need as the primary reason for the poor results. However, it is likely that continued exposure to non-relevant retrieval results will lead to significant user frustration with the system. In a study that aimed to test the hypothesis H1 that structured browsing would facilitate effective image retrieval where the user had only a vague concept of their information need, Lai, Tait, and McDonald (2000) compared the CHROMA system’s query tools: a colour navigation tool and a query by colour location tool. The CHROMA navigation tool provides access to a hierarchical set of ten colour groupings based on an extended perceptual colour model proposed by Berlin and Kay (1991) with col- ours transformed using the HSV (Hue, Saturation, Value) colour model. Twenty-four undergraduates participated in the study and were required to com- plete fourteen search tasks: seven specific and seven non-specific. The results show that there was no significant difference, p>0.01, in user satisfaction measures for tool type or in the number of successful searches between the tools in either task condition. Overall, tasks were performed significantly quicker using the navigation tool than when using the colour location tool particularly when the task was non- specific. As a result, they concluded that this was an indication that the provision of structured browsing facilitates fast and effective image retrieval where the user had only a vague concept of their information need. This is not a surprising result given the nature of the task and tool, and reaffirms the relationship 142 Journal of the Korean Society for Information Management, 28(2), 2011 between browsing and non-specific queries (Batley 1988). However, the results suggest that their ap- proach improves the browsing process by limiting the range and depth of the colour groups, thereby minimising the risk of the user becoming disoriented. In a second experiment, McDonald, Lai, and Tait (2001) compared the performance of the CHROMA colour navigation tool against the QBIC colour layout tool to test the hypothesis H1 in their previous experiment. The evaluation involved ten under- graduates who had no experience with either user interface and were required to complete fourteen search tasks: seven specific and seven non-specific. In contrast to their previous experiment, the results show that while there was no significant difference in user satisfaction with the retrieval results using the tools, there was a significant difference in the ratings of the participants’ satisfaction with the search tool, Z=-2.46, p>0.01. The restriction to the ten perceptual colours of CHROMA’s navigation tool clearly affected user satisfaction in a positive direction over the more extensive colour combina- tions that could be selected from QBIC’s colour space. This suggests that limited retrieval features may not be a barrier to effective retrieval and supports the findings of Bird, Elliott, and Hayward (1999) that the amount of colour information contained in the query Qn affects retrieval performance. However, it was acknowledged that the image collection used in both experiments only contained one thousand images and evidence suggests that the viability of such an approach is heavily dependent on the size of the image collection (Enser 1995; Markkula and Sormunen 2000). In a two-part experiment to investigate users’ apti- tude to differentiate between single and multiple block colours, Chan and Wang (2006) demonstrated that participants average estimates for both blocks was higher than the real size in both cases and that there was greater uncertainty with more dispersed estimates for mid-size colour blocks than for small or large. What effect colour size estimates have on retrieval performance is unclear and further inves- tigation focusing on retrieval performance is required to understand the significance of these findings. However, it is highly likely that over and under esti- mates of colour in the query Qn will significantly affect the retrieval performance of the system. To compound the problem further, Purves and Lotto (2011) argue that colour perception is ‘a reflex- ive manifestation of past behavioral success rather than the result of a logical processing of present stimuli’. In other words, what we see is based on the history, experience and knowledge of the in- dividual as a means of translating the present light stimuli. This is a profound and controversial position, which raises serious questions regarding searchers ability to accurately represent their information prob- lems using chromatic features as well as having sig- nificant and far-reaching consequences for how the human visual system is modelled now and in the future. 2.3 Image Queries In contrast to the field of text retrieval, studies Query by Colour : Investigating the Efficacy of Query Paradigms for Visual Information Retrieval 143 of end users and their information seeking behaviour are strictly limited. The seminal work on image query analysis was conducted by Enser and McGregor (1992). Their analysis of two thousand seven hundred and twenty two requests from the Hulton Deutsch picture collection revealed that approximately sev- enty percent of requests were for a unique person, object or event and that the majority of other request included refinements based on time. From this exami- nation they concluded that queries could be allocated either unique or non-unique categories and were sub- ject to refinement
. An example of non- unique would be castles
and an example of unique would be Edinburgh Castle
. Therefore, an example of a non-unique refined would be castles at night whilst an example of a unique refined query might be Edinburgh Castle at Hogmanay, 2010
. However, they acknowledged there was a process of pre-meditation in the ex- pression of the query, which was filtered and reformu- lated through an intermediary picture expert. Armitage and Enser (1997) extended this work by analyzing the image queries from seven picture libraries in order to develop a general-purpose catego- risation of image information behaviour. They ana- lysed one thousand seven hundred and forty nine image queries using the unique, non-unique and re- fined categorization proposed by Enser and McGregor (1992) and found that there was a notable degree of overlap between the sample data and the previous results. However, despite the synergy with the pre- vious work, they argue that the simplicity masked the complex nature of the relationship between unique and non-unique objects in the visual domain as it was not obvious at what level uniqueness of an entity was encountered in a hierarchy of related super-con- cepts and sub-concepts. Fidel (1997) conducted an investigation which analysed one hundred requests submitted to a stock photograph collection using Jörgensen’s (1995) twelve classes: literal object; people; people related attributes; art historical information; colour; visual a. Castle b. Edinburgh Castle c. Edinburgh Castle at Hogmanay, 2010
Non-Unique, Unique and Unique Refined Categories 144 Journal of the Korean Society for Information Management, 28(2), 2011 elements; location; description; abstract concepts; content or story; external relationships; viewer response. As noted previously by Armitage and Enser (1995), it was observed that attribute assignment was challenging, as the distinction between objects and the abstract concepts they could represent was an intractable and complex problem to resolve. This led Fidel to consider the nature of the retrieval task, concluding that images can be used as both sources of data or objects which can be classified as extreme poles on a continuum of retrieval tasks with varying degrees of significance determined by the searcher
.
Retrieval Task Continuum It was suggested that although there was no empiri- cal evidence on which to base further explorations, these polar constructs could be used as a conceptual framework for future retrieval experiments. Impor- tantly for the field of content-based image retrieval, it was also noted that none of the queries included requests for shape or texture, with only two from the sample referring to a specific colour, which raises serious doubts concerning its efficacy as a viable approach to address the challenge of providing effec- tive and efficient image retrieval. The studies of user queries suggest that information needs’ fall into three broad categories: specific items or instances, general topical or subject category, and abstract or affective responses. Broadly translated as specific, semi-specific and non-specific where spe- cific is when only one image will satisfy the request e.g. a specific painting or X-ray; semi-specific is where only one of a subset is acceptable e.g. specific conditions or criteria are attached, such as Impres- sionist paintings; non-specific is where the searcher is unsure exactly what is required or has only fuzzy criteria (Conniss, Ashford, and Graham 2000). How- ever, Taylor (1968) argued that the analysis of user information needs’ solely through the analysis of query data as an approach to understanding in- formation behaviour should be treated with a degree of caution. He suggested that as a result of the original information problems exposure to a process of nego- tiation, interpretation, and synthesis through an inter- mediary, it potentially represents a compromised need as a result of the filtering process against the contents of the image collection, the capabilities of the retrieval system, and the human searcher. Nevertheless, the results provide a useful insight into information seeking behaviour and provide a founda- tion for future research. 3. Query by Colour Evaluation Information retrieval systems have traditionally been evaluated by measuring their retrieval effective- ness using two measures: precision and recall. Since their inception these measurements have been central to numerous text retrieval experiments including the TREC experiments. However, a number of objections have been raised regarding their suitability as in- Query by Colour : Investigating the Efficacy of Query Paradigms for Visual Information Retrieval 145 dicators of retrieval performance. The major criticism levelled at the measures has largely focused on the pre-mathematical concept of relevance (Korfage 1997). van Rijsbergen (1979) stated that evaluation of information retrieval systems has proved ex- tremely difficult and, despite numerous measures of effectiveness that have been proposed, a general theory of evaluation has not emerged; a position that still holds true. In contrast, Lancaster and Warner (1993) report that studies have consistently demon- strated that usability is the principal factor affecting the selection of an information source. Emerging attempts have shifted their focus away from measures of retrieval effectiveness to consider user satisfaction and to understand its importance (Dunlop 2000; Johnson, Griffiths, and Hartley 2001; McDonald, Lai, and Tait 2001). This experiment focused on satisfaction as defined in the ISO 9241 standard; the level of comfort that the user feels when using a product or system and how effective that product or system was in supporting their goals. The aim of the experiment was to assess whether query by colour supported visual query expression of a range of information problems. The experiment was a within-subjects experimental design. The in- dependent variable was tool type and the dependent variable was user satisfaction with a focus on overall reaction, learning and system capabilities. The sys- tem used in the experiment was the version of QBIC was that available at The State Hermitage Museum, which supports query formulation by colour percent- age and colour layout (Mintzer et. al. 2001). The evaluation involved twenty-eight, senior Library and Information Science undergraduate students who had no previous exposure to the system and included twenty females and eight males with an age range between 20-27. Participants were required to com- plete six search tasks; two specific, two semi-specific, and two non-specific. Specific search tasks where to find the specific images in Figure 6. The semi-specific tasks were to find paintings of sunflowers and Paris. Non-specific tasks were to find paintings of the “grandeur, simplicity and indestructibility of nature” and a “festival.” The tasks were ordered from the specific to non-specific to provide an increasing level of task difficult. Data
Specific Search Tasks 146 Journal of the Korean Society for Information Management, 28(2), 2011 was collected via a post-search questionnaire which contained twenty nine items on a ten-point differential negative and positive rating scale i.e. -5, -4, -3, -2, -1, 1, 2, 3, 4, 5. The post-session questionnaire was a modified version of the Questionnaire for User Interaction Satisfaction (Chin, Diehl, and Norman 1988) and was designed to ascertain the subjective satisfaction of participants. The granularity of the rating scale was modified from its linear scale of 1-9 and NA to a negative and positive scale, as studies have suggested that this scale reading corresponds more accurately to the opinions of users (Hix and Hartson 1993). The scale contained no neutral value in order to reduce the error of central tendency thereby forcing participants’ to make a negative or positive choice. Statistics were calculated for measures of central tendency, dispersion, and distribution. The data was analysed using the Wilcoxon signed-rank test at an alpha level of p=0.001. 4. Results Table 1 shows the mean x and standard deviation σ for overall reaction to the query tools. The results show that the colour layout tool was rated positively on two out of four measures and the colour search tool was rated positively on one out of four measures. The colour layout tool was considered wonderful ( x d=2.25) and easy ( x d=1.75) in spite of being rated frustrating ( x d=-0.38) and inadequate ( x d=-0.63). Similarly, despite being considered easy to use ( x d =1.38), the colour percentage tool was rated as terrible ( x d=-1.50), frustrating ( x d=0.13), and inadequate ( x d =-1.50). However, scores for both tools are clustered around low mean negative and positive values. The difference between the scores on three out of four semantic differential measures is negligible: frustrat- ing-satisfying ( x df =0.25), difficult-easy ( x df =0.38), and inadequate-adequate ( x df =0.87). The scores be- tween the query tools on the terrible-wonderful se- mantic differential are diametrically opposed with the colour layout tool being rated positively and the colour search tool rated negatively ( x df =3.75). Figures 7 and 8 show the distribution of scores for overall reaction. The results show that there is a clustering between the individual scores within range of one or two points in the inter-quartile range with the exception of the frustrating-satisfying se- mantic differential for the colour layout tool, which was more widely distributed in comparison to the other measures suggesting that participants’ opinion Colour Layout Colour Percentage Terrible-Wonderful 2.25 (0.886) -1.50 (1.773) Frustrating-Satisfying -0.38 (2.446) -0.13 (2.100) Difficult-Easy 1.75 (1.832) 1.38 (1.996) Inadequate-Adequate -0.63 (2.200) -1.50 (2.563) Overall Reaction, Mean and Standard Deviation Query by Colour : Investigating the Efficacy of Query Paradigms for Visual Information Retrieval 147
Overall Reaction, Colour Layout
Overall Reaction, Colour Percentage was more widely divided with regards to how frustrat- ing or satisfying participants perceived it (Q1=-2min, Q3=1.5max). A Wilcoxon signed rank test for the combined scores for overall user reaction revealed that there was no significant difference, p>0.001, in overall reaction between the two query tools, Z=-2.142, p=0.032. Correspondingly, a Wilcoxon signed rank test on individual measures also shows that there was no significant difference, p>0.001, of user sat- isfaction between the two tools
. Wilcoxon Signed Rank Terrible-Wonderful Z=-2.384, p=0.017 Frustrating-Satisfying Z=-0.512, p=0.609 Difficult-Easy Z=-0.108, p=0.914 Inadequate-Adequate Z=-0.921, p=0.357
Wilcoxon Signed Rank Scores for Individual Measures of Overall Reaction Table 3 shows the mean x d and standard deviation σ for learning the query tools. The results show that the colour layout tool was positively rated on all four measures and the colour percentage tool being rated positively on three out of four measures. Both tools were considered easy and fast to learn, and easy to correct errors. The colour percentage tool has a negligible higher mean rating for overall learning and time to learn measures than the colour layout tool with a difference of x df =0.87. Notably, both query tools were rated poorly with regards to task performance. Despite the colour layout tool hav- ing a having a positive rating score for task perform- ance it is only marginally positive at x d=0.12. Tasks using the colour percentage tool had a negative mean rating score although the score is not highly negative at x d=-1.38; the difference between the two scores is also negligible at x df =1.50. Figures 9 and 10 show the distribution of scores for learning. The results show responses are not widely distributed suggesting some broad consensus of opinion. 148 Journal of the Korean Society for Information Management, 28(2), 2011 Colour Layout Colour Percentage Learning (Difficult-Easy) 2.13 (1.885) 3.00 (1.309) Time to Learn (Slow-Fast) 2.13 (2.100) 3.00 (2.268) Error Correction (Difficult-Easy) 1.63 (2.326) 1.38 (2.615) Tasks Performance (Never-Always) 0.12 (1.642) -1.38 (2.387)
Learning, Mean and Standard Deviation
Learning, Colour Layout
Learning, Colour Percentage A Wilcoxon signed rank test for the combined scores for learning revealed that there was no sig- nificant difference in overall leaning between the two tools, Z=-0.456, p=0.648. Similarly, a Wilcoxon signed rank test for individual measures also shows no significant difference in individual learning meas- ures between the two tools
. Table 5 shows the mean x d and standard deviation σ for the capabilities of the query tools. The results show that the colour layout tool has a positive mean rating on three out of four measures as opposed to the colour percentage tool, which has a negative mean rating across all measures. However, the neg- ative x d ratings of the colour percentage tool gravitate to the low-end of the scale suggesting that despite the participants’ rating, the degree of their dissat- isfaction was not highly negative. Both tools were considered not to support visual query formulation with the colour layout and colour percentage tools receiving mid-level negative scores of x d=-2.38 and x d=-2.13 respectively. The implications of this result will be discussed in Section 5. The usability of the colour layout tool was not considered to effect either visual query expression or the retrieval results. As observed by Bird, Elliott, and Hayward (1999), this may be the result of users blaming their own ability to formulate an accurate representation of their in- formation need rather than the systems ability to Query by Colour : Investigating the Efficacy of Query Paradigms for Visual Information Retrieval 149 Wilcoxon Signed Rank Learning (Difficult-Easy) Z=-1.552, p=0.121 Time to Learn (Slow-Fast) Z=-1.823, p=0.068 Error Correction (Difficult-Easy) Z=-0.333, p=0.739 Tasks Performance (Never-Always) Z=-1.802, p=0.072
Wilcoxon Signed Rank Test for Individual Measures of Learning Colour Layout Colour Percentage Tools Supports Visual Query Expression(Never-Always) -2.38 (1.685) -2.13 (2.031) Usability Effects Query Expression(Always-Never) 1.25 (1.035) -1.75 (1.581) Usability Effects Retrieval Results(Always-Never) 3.00 (0.756) -2.13 (2.295) RGB Values Effects Results(Always-Never) 3.38 (1.188) -2.25 (1.909)
Capabilities, Mean and Standard Deviation
Colour Layout, Capabilities
Colour Percentage, Capabilities support them in this activity. While the mean value for usability effecting query expression is low at x d=1.25, the mean value with regards to usability effecting the retrieval results is rated high at x d=3.00. In contrast, the usability of the colour percentage tool was considered always to effect both query ex- pression and the retrieval results, x d=-1.75 and x d =-2.13 respectively. Participants considered that the RGB colour values used with the colour percentage tool also considerably effected the retrieval results with a mean score of x d=-2.25 which is diametrically opposed to the colour layout tool at x d=3.38 given a difference of x df =5.63. Figure 11 and 12 show the distribution of scores for system capabilities. A Wilcoxon signed rank test for the combined scores of system capability revealed a significant 150 Journal of the Korean Society for Information Management, 28(2), 2011 Wilcoxon Signed Ranks Tools support visual query expression? Z=-0.324, p=0.746 The usability of the tool effects visual query expression? Z=-2.322, p=0.020 The usability of the tool effects the retrieval results? Z=-2.410, p=0.016 Retrieval results are effected by the RGB colour values? Z=-2.533, p=0.011
Wilcoxon Signed Rank Test for Individual Measures of Capability difference in overall capabilities between the two tools, Z=-4.197, p<0.001. However, a Wilcoxon sign- ed rank test on individual measures showed no sig- nificant difference in individual capability measures between the two tools
. 5. Discussion This works has important implications for the de- sign of query methods for content-based image re- trieval systems or hybrid systems that include a con- tent-based retrieval features and in particular query by colour. Overall reaction by participants to both query methods was mixed. The data revealed that there was no significant difference, p>0.001, between the two query methods on two of the three measures: overall reaction (Z=-2.142, p=0.032) and learning (Z=-0.456, p=0.648). In contrast, the combined re- sults for system capability show a significant differ- ence between the two query methods at Z=-4.197, p<0.001, although not at an individual level. In gen- eral, the colour layout tool was rated more positively than the colour percentage tool. It has been suggested that two important aspects of a query interface are its expressive power and its ease of use i.e. what can be expressed using the systems language combined with how difficult it is for the searcher to articulate their information prob- lem using that language. In terms of ease of use, the results of this experiment suggest that both query methods were considered easy to use by the partic- ipants although the mean rating scores for this was at the lower end of the positive rating scale suggesting considerable scope for improvement. That both query methods were considered frustrating and inadequate suggests that while they may have been easy to use their expressive power was somewhat questionable. This is supported by the data concerning learning and system capabilities, which revealed that partic- ipants in this experiment considered that tasks using either tool could not be performed in a straight for- ward manner or that they did not support visual query expression. This strongly suggests that there is a significant mismatch between searchers information problems and the query methods available to support efficient and effective visual query formulation using this approach. In addition to the data generated via the post-ses- sion questionnaire a number of phenomena was also observed. Firstly, participants expressed having a mental image of the information need for the semi-specific and non-specific information tasks. For example, one tasks was to retrieve paintings of Query by Colour : Investigating the Efficacy of Query Paradigms for Visual Information Retrieval 151 sunflowers. A number of participants expressed that their mental image of this information problem was that of van Gogh’s painting of Sunflowers
. This was also noted by Jose, Furner, and Harper (1998) who observed that users had a well-defined mental image of their information need. However, within the dataset only two images of sunflowers exist: Vase of Sunflowers (circa. 1898-99) by Henri Matisse
and Sunflowers (1901) by Paul Gauguin. Despite all being pictures of sunflowers, the colour properties and their distribution within these images differ quite significantly which is illustrated in Figure 13. This in itself will have a significant effect on retrieval performance of the system. The use of purely chromatic features to represent an information prob- lem presents unique challenges to the searcher. It requires the searcher to think about their information needs and its representation in a way that deviates significantly from what they are used to. While this was not measured empirically, it is also clear that a searchers mental representation of their information problem can also have a significant effect on retrieval performance as it influences the chromatic features selected including their quantity and distribution. This is an important aspect to consider with regards to information problems that are less well clearly defined. What effect the mental representation of an information problem plays in the retrieval process presents an interesting avenue for research as well as the emerging theories related to the processing of colour stimuli and its effect on colour selection. Similarly, it was also noted that participants’ abil- ity to formulate a visual abstraction of the experiment information problems varied significantly. It was ob- served that despite significant efforts to reproduce a viable representation of their information need a number of participants were unable to accurately a. Sunflowers, van Gogh b. Vase of Sunflowers, Matisse
Sunflowers 152 Journal of the Korean Society for Information Management, 28(2), 2011 represent their query. This was both the product of the expressive power of the query tool and the searchers mental model of the information problem. For example, the colours selected for the sunflower query did not represent either the colours used in the target image or the colours of the physical flower head. While all participants used yellow for the outer petals the colour used for the kernel of the flower head was generally orange. Depending on the type of sunflower e.g. Helianthus Annuus, the kernel col- our can range from various shades of green to black. As observed by Bird, Elliott, and Hayward (1999), there was also a tendency to include too much detail in the form of small blocks, which affected the re- trieval performance of the system. As a result, re- trieval performance was severely affected making it challenging, if not impossible, for the searcher to refine the retrieval results in a relevance feedback loop. These findings have important implications for the efficacy and utility of content-based image re- trieval as an approach although there is a need to examine the usefulness of both the tools and retrieval features in context. Ultimately, the effectiveness of the colour retrieval method depends largely on the users’ ability to express their information problem using purely chromatic features. While the results are based on a small sample conducting generic search tasks on an art image database the data indicates that there are limits in which query by colour can be used effectively. The results of this experiment suggest that the effective and efficient retrieval of relevant images is not only dependent on robust re- trieval algorithms but on supporting the querying needs of the searcher. This experiment focused solely on participant’s satisfaction as defined in the ISO 9241 standard focus- ing on their overall reaction, learning and system capabilities as measures of user satisfaction. However, it is acknowledged that all three measures of usability - effectiveness, efficiency, and satisfaction - may be required in order to reduce the risk of making un- founded assertions about the overall usability of the system under investigation (Frøkjær, Hertzum, and Hornbæk 2000); where effectiveness is the extent to which a goal or task can be achieved and is indicated by a quantitative measure of the number of errors and efficiency is the amount of effort required to complete a task and is reflected in a quantitative measures of task completion time and learning time. To what extent the use of indicators such as time play in measuring usability in information retrieval experiments, particularly where there is a significant element of subjective interpretation required to com- plete a task, is open to debate. However, the number of errors and in particular their type gives an insight into the usability of systems. Employing an error classification with a fine level of granularity, such as that proposed by Norman (1981), may reveal more interesting data regarding measures of effec- tiveness experienced by participants during the course of an experiment. Nevertheless, the develop- ment of such taxonomy was beyond the scope of this research. Query by Colour : Investigating the Efficacy of Query Paradigms for Visual Information Retrieval 153 6. Conclusions This paper presented the results of an experiment that investigated the usability of the query by colour method in supporting a range of information problems. The ability for users’ to communicate their infor- mation requirements effectively and efficiently is cru- cial in any information retrieval system. The success and value of any information resource can therefore be maximised by ensuring that its content is effectively exploited and is accessible to those users whose in- formation requirements it is intended to support. A powerful image retrieval system with no effective method by which the searcher can formulate visual queries of their information problems has little value. Content-based retrieval research has generally oper- ated in a vacuum isolated from the real activities and tasks of the searcher focusing primarily on the technical aspects of delivering effective image retrieval. As a consequence, the design of query inter- faces for content-based image retrieval has been driv- en by the underlying retrieval mechanism and not by the requirements of the end user. To address the challenges of and deliver effective image retrieval requires more than the sum of retrieval algorithms. The importance of the user in the design and the evaluation of content-based image systems have been grossly underestimated by the research community. What the information needs of users of image collec- tions are and how successfully these can be expressed in a content-based image retrieval environment is far from clear. This is a complex field and no single solution or approach will solve the multi-dimensional issues to achieve effective and efficient retrieval of the data. The fusion of work from a diverse range of disciplines, including mathematics and statistics, information and computer science, neuroscience and psychology, will all contribute to the development of accessible image retrieval systems. Nevertheless, research into query interfaces for content-based image retrieval is still in its infancy and substantial research is required to understand the relationship between the user’s information problem and methods of query formulation and refinement. References Armitage, L. H. and P. G. B. Enser. 1997. “Analysis of user need in image archives.” Journal of Information Science, 23(4): 287-299. Batley, S. 1988. “Visual information retrieval: brows- ing strategies in pictorial databases.” In: Online Information '88: Proceedings of the 12th International Online Information Meeting, 6th-8th December 1988, London, England: 373-381. Berlin, B. and P. Kay. 1991. Basic Color Terms: Their Universality and Evolution. University of California Press. 154 Journal of the Korean Society for Information Management, 28(2), 2011 Bird, C., P. J. Elloitt, and P. M. Hayward. 1999. “Content-based retrieval for European image libraries.” In: The Challenge of Image Re- trieval: CIR99, The 2nd UK Conference on Image Retrieval, Forte Posthouse Hotel, Newcastle upon Tyne, United Kingdom, 25th-26th February 1999, edited by D. J. Harper and J. P. Eakins. Brady, M., D. Gavaghan, A. Simpson, R. Highnam, A. Knox, S. Lloyd, A. Simpson, and D. Watson. 2004. “Grid computing for digital mammo- graphy.” In: AHM’2003: Proceedings of UK e-Science All Hands Meeting 2003, Notting- ham, UK, 2-4th September 2003, edited by Simon Cox: 723-730. Castelli, V. and L. D. Bergman. 2002. Image Data- bases: Search and Retrieval of Digital Imagery. John Wiley & Sons, 1-10. Chan, H. C. and Y. Wang. 2006. “Human factors in color-based image retrieval: an emprical study on size estimate accuracies.” Journal of Visual Communication & Image Representation, 15(2): 113-131. Chin, J., V. Diehl, and L. Norman. 1988. “Develop- ment of an instrument measuring user sat- isfaction of the human-computer interface.” In: CHI '88: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Washington, D.C. USA, 15-19 May 1988, edited by J. J. O’Hare, ACM Press: 213-218. Conniss, L. R., A. J. Ashford, and M. E. Graham, 2001. “Information seeking behaviour in im- age retrieval: VISOR I.” Institute for Image Data Research, 1-147, (Library and Information Commission Research Report 95). Datta, R., D. Joshi, J. Li, and J. Z. Wang. 2008. “Image retrieval: ideas, influences, and trends of the new age.” ACM Computing Surveys, 40(2). Del Bimbo, A. 1999. Visual Information Retrieval. Morgan Kaufmann. Dunlop, M. 1991. “Multimedia information retrieval.” PhD Thesis, University of Glasgow. Eidenberger, H. and C. Breiteneder. 2003. “VizIR: a framework for visual information retrieval.” Journal of Visual Languages and Computing, 14(5): 443-469. Enser, P. G. B. 1995. “Progress in documentation pictorial information retrieval.” Journal of Documentation, 51(2): 126-170. Enser, P. G. B. and C. G. McGregor. 1992. “Analysis of visual information retrieval queries.” London: British Library, (British Library Research & Development Report, No. 6104). Feng, D., W. C. Siu, and H. J. Zhang. 2003. “Multi- media information retrieval and management: technological fundamentals and applications.” Springer-Verlag. Multimedia Signal Processing Book Series: 1-26. Fidel, R. 1997. “The image retrieval task: implications for the design and evaluation of image databases.” The New Review of Hypermedia and Multimedia, 3: 181-199. Frøkjær, E., M. Hertzum, and K. Hornbæk. 2000. “Measuring usability: are effectiveness, effi- ciency, and satisfaction really correlated?” Query by Colour : Investigating the Efficacy of Query Paradigms for Visual Information Retrieval 155 In: CHI’00: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, The Hague, The Netherlands, 01- 06, April, 2000, ACM Press: 345-352. Flickner, M., H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker. 1995. “Query by image and video content: the QBIC system.” IEEE Computer, 28(9): 23-32. Gudivada, V. N. and V. V. Raghaven. 1995. “Content- based image retrieval systems.” IEEE Com- puter, 28(9): 20. Hanjalic, A., N. Sebe, and E. Y. Chang. 2006. “Multi- media content analysis, management and re- trieval: trends and challenges.” In: Electronic Imaging 2006, Multimedia Content Analysis, Management and Retrieval 2006, IS&T/SPIE International Symposium, SPIE Vol. 6073, San Jose Marriott and San Jose Convention Centre, San Jose, California, USA, 15-19 January 2006, edited by Edward Y. Chang, Alan Hanjalic and Nicu Sebe, SPIE. Hirata, K. and T. Kato. 1992. “Query by visual example.” In: EDBT’92: Advances in Database Technology: Proceedings of the 3rd Interna- tional Conference on Extending Database Technology, Vienna, Austria, 23rd-27th March 1992, edited by A. Pirotte, C. Delobel and G. Gottlob, Springer-Verlag, 56-71. Hix, D. and H. R. Hartson. 1993. Developing User Interfaces: Ensuring Usability Through Product and Process. John Wiley. Hua, X-S. 2010. “How to realize content analysis in web-scale multimedia search.” In: CIVR 2010: ACM International Conference on Image and Video Retrieval, Xi'an, China 5-7 July, 2010. Huang, J., S. Kumar, M. Mitra, W-J. Zhu, and R. Zabih. 1997. “Image indexing using color cor- relogram.” In: CVPR '97: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, San Juan, Puerto Rico, 17-19 June, 1997, IEEE Computer Society, 762-768. Idris, F. and S. Panchanathan. 1997. “Review of image and video indexing techniques.” Journal of Visual Communication and Image Represen- tation, 8(2): 146-166. Jaimes, A., and S-F. Chang. 2002. “Concepts and techniques for indexing visual concepts.” In: Castelli, V. and Bergman, L. D. (ed). Image Databases: Search and Retrieval of Digital Imagery, John Wiley & Sons Inc, 497. Johnson, F. C., J. R. Griffiths, and R. J. Hartley. 2001. “DEVISE: a framework for the evaluation of internet search engines.” The Council for Museums Archives and Libraries (Library and Information Commission Research Report No. 100). Jörgensen, C. 1995. “Image attributes: an investi- gation.” PhD thesis, Syracuse University. Jose, J. M., J. Furner, and D. J. Harper. 1998. “Spatial querying for image retrieval: a user-oriented evaluation.” In: SIGIR: Proceedings of the 21st Annual International Conference On Research And Development in Information Retrieval, University of Melbourne, 24-28 156 Journal of the Korean Society for Information Management, 28(2), 2011 August 1998, ACM Press, 232-240. Kato, T. 1991. “Intelligent visual interaction with im- age database systems: toward the multimedia personal interface.” Journal of Information Processing, 14(2): 134-143. Kato, T., T. Kurita, and H. Shimogaki. 1989. “Multi- media interaction with image database Systems.” In: Advanced Database System Symposium ’89, Kyoto, Japan, 1989, 271-278. Korfhage, R. R. 1997. Information Storage and Retrieval. New York: John Wiley & Sons. Lai, T-S. 2000. “CHROMA: a photographic image retrieval system.” Ph.D. Thesis. University of Sunderland, UK. January 2000. Lai, T-S., J. Tait, and S. McDonald. 2000. “A user-cen- tred evaluation of visual search methods for CBIR.” In: CIR2000: The Challenge for Image Retrieval, Third UK Conference on Image Retrieval, Old Ship Hotel, Brighton, UK, 4th-5th May 2000, edited by J. P. Eakins and P. G. B. Enser. Lancaster, F. W. and A. J. Warner. 1993. Information Retrieval Today. Arlington: Information Re- sources Press. Lew, M. S. and T. S. Huang. 2001. “Visual information retrieval: paradigms, applications, and re- search issues.” In: Lew, M. S. (ed.) Principles of Visual Information Retrieval. Springer- Verlag, Advances in Pattern Recognition, 3-9. Lu, H., B-C. Ooi, and K-L. Tan. 1994. “Efficient image retrieval by color contents.” In: ADB- 94: 1st International Conference on Applica- tions of Databases, Vadstena, Sweden, 21-23 June, 1994, edited by Witold Litwin and Tore Rischor, Springer, 95-108. McDonald, S., T-S. Lai, and J. Tait. 2001. “Evaluating a content-based image retrieval system.” In: SIGIR’01: Proceedings of the 24th Annual International ACM Special Interest Group on Information Retrieval Conference on Re- search and Development in Information Re- trieval, New Orleans, Louisiana, US, ACM Press, 232-240. Markkula, M. and E. Sormunen. 1998. “Searching for photos: journalist practices in pictorial IR.” In: CIR98: The Challenge of Image Retrieval Workshop, University of Northumbria at Newcastle, Newcastle upon Tyne, United Kingdom, 5th February 1998, edited by J. P. Eakins, D. J. Harper and Joemon Jose. Mintzer, F., G. W. Braudway, F. P. Giordano, J. C. Lee, K. A. Magerlein, S. D’Auria, A. Ribak, G. Shapir, F. Schiattarella, and A. Zelenkov. 2001. “Populating the hermitage museums new web site.” Communications of the ACM, 44(8): 52-60. Müller, W., H. Müller, S. Marchand-Maillet, T. Pun, D. M. Squire, Z. Pecenovic, C. Giess, and A. P. de Vries. 2000. “MRML: an extensible communication protocol for interoperability and benchmarking of multimedia information retrieval systems.” Visual Information and Information Systems, 300-311. Niblack, C. W., R. Barber, W. Equitz, M. D. Flickner, E. H. Glasman, D. Petkovic, P. Yanker, C. Faloutsos, and G. Taubin. 1993. “The QBIC Query by Colour : Investigating the Efficacy of Query Paradigms for Visual Information Retrieval 157 project: querying images by content using col- or, texture and shape.” In: Storage and Retrieval for Image and Video Databases, Proceedings of the Society of the Photo- Optical Instrumentation Engineers Vol. 1908, San Jose, CA, USA, 31 January-5 February 1993, edited by Carlton W. Niblack, 173-187. Norman, D. A. 1981. “Categorisation of action slips.” Psychological Review, 88(1): 1-15. Purves, D. and R. B. Lotto. 2011. Why we see what we do redux: a wholly empirical theory of vision. Sinauer Associates. C. J. van Rijsbergen. 1979. Information Retrieval. London: Butterworths. Rickman, R. M. and T. J. Stonham. 1996. “Content- based image retrieval using colour tuple histograms.” In: Storage and Retrieval for Still Image and Video Databases IV, Proceedings of the Society of the Photo-Optical Instrumen- tation Engineers Vol. 2670, San Diego/La Jolla, CA, USA, March 1996, edited by Ishwar K. Sethi and Ramesh C. Jain, 2-7. Rui, Y. and T. S. Huang. 2001. “Relevance feedback techniques.” In: Lew, M. S. (ed.) Principles of Visual Information Retrieval. Springer- Verlag, Advances in Pattern Recognition, 219-258. Rui, Y., T. S. Huang, and S-F. Chang. 1999. “Image retrieval: current techniques, promising direc- tions, and open issues.” Journal of Visual Communication and Image Representation, 10(1): 51. Smeulders, A. W. M., M. Worring, S. Santini, A. Gupta, and R. Jain. 2000. “Content-based im- age retrieval at the end of the early years.” IEEE Transaction on Pattern Analysis and Machine Intelligence, 22(12): 1349-1380. Smith, J. R. 2002. “Color for image retrieval.” In: Lew, M. S. (ed.) Principles of visual infor- mation retrieval. Springer-Verlag, Advances in Pattern Recognition, 219-258. Smith, J. R. and S-F. Chang. 1996. “Tools and techni- ques for color image retrieval.” In: Proceed- ings of IS&T/SPIE Symposium on Electronic Imaging: Science and Technology (EI’96)― Storage and Retrieval for Image and Video Databases IV, vol.2670: 426-437. Stricker, M. A. and M. Orengo. 1995. “Similarity of color images.” In: Storage and Retrieval for Image and Video Databases III, Proceedings of the Society of the Photo-Optical Instru- mentation Engineers, Vol. 2420, San Diego/La Jolla, CA, USA, March 1995, edited by Wayne Niblack and Ramesh C. Jain, 381-392. Swain, M. and D. Ballard. 1991. “Color indexing.” International Journal of Computer Vision, 7(1): 11-32. Taylor, R. S. 1968. “Question-negotiation and in- formation seeking in libraries.” College and Research Libraries, 29: 178-194. Veltkamp, R. C. and M. Tanase. 2002. “A survey of content-based image retrieval systems.” In: Marques, O. and Furht, B. (ed), Content-Based Image and Video Retrieval, Kluwer, 47-101. Venters, C. C. and M. D. Cooper. 2000. “A review of content-based image retrieval systems.” 158 Journal of the Korean Society for Information Management, 28(2), 2011 JISC Technology Applications Programme, JTAP Report, University of Manchester. Venters, C. C., R. J. Hartley, and W. T. Hewitt, 2003. “Mind the gap: users interfaces and content- based image retrieval.” In: Deb, S. (ed.) Multimedia Systems and Content-Based Image Retrieval. Idea Publishing, 322-355. Yu, F.-X., H. Luo, and Z.-M. Lu. 2011. “Colour image retrieval using pattern co-occurrence matrices based on BTC and VQ.” Electronics Letters, 47(2): 100-101.