College and Research Libraries CAROL A. JOHNSON and RICHARD W.' TRUE SWELL The Weighted Criteria Statistic Score: An Approach to Journal Selection The problem of effective journal selection has always been a difficult one to solve. With increasing subscription costs resulting in ever-decreasing buying power, its solution has become critical. The large proportion of library budget dollars spent on journal subscriptions, as well as the continuing re- quirement to satisfy user needs, mandate the use of a technique for journal selection that is both practical and systematic. The weighted criteria statistic score provides a decision-making approach that is reliable and widely appli- cable for identifying, evaluating, and utilizing journal selection criteria. THE AIR FORCE GEOPHYSICS Research Library serves the information/bibliographic needs of the Air Force Geophysics Labora- tory (AFGL) and the Air Force Electronic Systems Division personnel at Hanscom Air Force Base. It is also a major resource of li- brary materials for numerous Department of Defense contractors located in the met- ropolitan Boston area, as well as throughout the United States. For more than twenty years the library has collected. books ana journals in a wide range of subjects to meet the varied re- search requirements of laboratory projects, including such subject areas as environmen- tal sciences, materials sciences, meteorol- ogy, geophysics, chemistry, physics, math- ematics, astrophysics, electronics, geodesy, terrestial sciences, and ionospheric re- search. Over the years comprehensive and com- plete holdings of journals and monographs in fields of interest have been accumulated, Carol A. johnson is chief, Analysis Division, National Agricultural Library, Beltsville, Mary- land. At the time this study was conducted, she was director, Air Force Geophysics Research Li- brary, Hansc011t. Air Force Base, Bedford, Mas- sachusetts. Richard W. Trueswell is professor, Department of Industrial Engineering and Oper- ations Research, University of Massachusetts at Amherst. and very little has been discarded. Journals have been collected and bound without con- sideration of the increasing costs and ever- decreasing space available. Current ,journal subscriptions have not been significantly re- duced for the past six years, and none have been purchased in microform. Correction of the problems associated with this expanding journal collection had not occurred. As a re- sult, some specific measures for immediate action were necessary. Some of these efforts are described in this paper. APPROACH It is increasingly true that librarians are facing uniformly and unilaterally the prob- lem of significantly increased costs of scien- tific monographs and serials. Similarly, the problem of available space and increased costs of providing it in today' s inflationary economic situation present problems that demand immediate resolution. The AFGL Research Library is no excep- tion to the above · set of conditions. It had therefore become mandatory to formulate an approach toward solving, or at least re- ducing the intensity of, financial and space problems for library materials, while at the same time continuing to support user re- search needs. As an initial step in gathering information about use of the collection, it was decided I 281 288 I College & Research Libraries • july 1978 to confront the problems relating to journal titles, because of the overall greater prob- lems with increasing costs and space re- quirements associated with these materials. The objective was to obtain usage statistics in order to determine whether or not the size of the journal collection is appropriate for meeting user needs. Many approaches are possible in the solu- tion of the problem, but they all must relate to the basic question of which serials are to be eliminated and which are to be retained. It is obvious that there are certain meas- urements or dimensions that will describe the relative importance of each journal sub- scription title. On the other hand, it is equally obvious that these dimensions are very difficult, if not impossible, to define and measure; e.g., it would be useful to be able to predict the subject areas and appro- priate titles for new or changing subject areas of user interest. This is extremely difficult to do because very few of the scien- tists and researchers know with any degree of certainty the answers to questions regard- ing future changes of emphasis in research and development activities any more than the managers and administrators. There is one thread of information that continues to exist in the literature and re- search regarding the importance and use of journal titles. This thread of information can be simply stated: Much of the use of a given commodity is expressable or satisfied by a small portion of that commodity. For exam- ple, Trueswell has shown this in his 80/20 rule, where he has statistically demon- strated that approximately 80 percent of the monograph circulation is satisfied by 20 per- cent of the holdings. 1 This concept is useful in attacking the problem of journal selection if we assume that there is a core of journal titles that satisfy most of the user require- ments. The hypothesis being tested in this study is that the library does in fact subscribe to more titles than are used, or identified as used, by its clientele. One strategy, there- fore, is to identify as many criteria as possi- ble that relate to the use of journal titles. The titles that are identified by these parameters could be considered as the core of journal title holdings. Thus by application of the basic concept of the 80/20 rule, we will be identifying a relatively small number of titles that satisfy a rather high proportion of the user requirements. In effect, a strategy for selection of rele- vant journal titles could be expressed as fol- lows: Defining any user-related criteria, select those journal titles that satisfy one or more of these criteria. The strategy decided upon in this study was to identify as many criteria as possible related to user require- ments for journals. Applying these user-related criteria, jour- nals are identified as relatively more impor- tant when they satisfy one or more of the criteria. A criteria list is then prepared rank- ing journal titles according to their Criteria Statistic Score, which is defined as the number of criteria satisfied by each title. The titles not included in the selected list will be those that can be considered for re- moval from the collection. This is essentially the approach applied at AFG L. Another approach to journal selection was taken by Holland. 2 She utilized journal usage data collected at · the University of Michigan Engineering Library and de- veloped a formula based on access time to measure the effect of journal budget reduc- tion on service to users. In this instance it was found that 50 percent of the paid sub- scriptions could be cut with only an 8 per- cent service reduction. Wenger and Chil- dress address the problem of journal evalua- tion somewhat differently in their study. 3 They gathered data from a use study, circu- lation and interlibrary statistics, a core list, and librarian and patron input in order to determine an efficient methodology for sub- scription renewals and enhanced collection relevance. Numerous means have been used for defining an optimum journal collec- tion. These studies may be found in the lit- erature. 4 The preceding examples merely serve to highlight some of these other ap- proaches. METHODOLOGY It is recognized that circulation statistics are valuable user-related criteria. However, journals do not circulate from the AFGL Research Library, and thus data for this cri- terion were not available. It was felt that journal circulation data from another library would not be appropriate for AFGL because of the rather unique mission of the AFG L library. Therefore, other user-related criteria had to be developed. Two types of criteria were developed: the first relating to actual use and the second relating to the user's perception of use or importance. The first set of criteria was developed from statistics collected over a six-month period regarding use of the library photo- copying machine. Each time a user made a photocopy of an article, the journal title, ar- ticle, pages, and user name were recorded by the user on a form maintained at the li- brary copying machine. A second set of six attitudinal criteria was developed through a literature use survey, which was formulated and circulated by the Library Advisory Committee to 700 scien- tists and engineers in the Geophysics Labo- ratory and Electronic Systems Division. In each of the first three questions only ten titles were requested. While this might be viewed as a limiting aspect, in fact, no re- spondent listed journals up to that number. Participants were requested to answer the following six questions: 1. List those journals (up to ten) that you used most often during the past year (three or more times) . 2. List those journals (up to ten) that you used last year but not as often (once or twice). 3. List other journals (up to ten) that you did not use last year but you know that they contain references of some interest to you . 4. List all the journals in which you pub- lished during the past five years. 5. List those journals that contain articles that you referenced in your publications of the past five years. 6. List those journals that have published articles citing your publications. Approximately 60 percent of those queried responded to this questionnaire. MethodQlogy of data compilation was based on availability of machine-readable files for all titles held in the library and computer · hardware and software cap- abilities. After collecting the data from questionnaires and photocopying use, the information was recorded on 3-by-5-inch cards according to journal title, criteria cited, and office affiliation of user. Prior to keypunching data from these cards, titles Weighted Criteria I 289 were reviewed for accuracy of citation. At the same time, a unique identification number was assigned from the library's existing machine-readable files of journal titles. It should be noted that all journal ti- tles cited in the questionnaire results were already held in the library's collection. This situation would be less likely to occur in a library having less extensive journal hold- ings. Data collected for each title were keypunched from the 3-by-5-inch cards . They included unique identification number, journal title (abbreviated, if neces- sary, to fit into thirty columns allocated), and criteria codes . Keypunched cards were manually reviewed to assure accuracy of identification numbers. No correction was made to title abbreviations that had been left to the keypuncher with very few guidelines, since this was not a major con- cern as long as the title could be readily identified, if necessary, by cross-checking against the eomplete citation in the master record otherwise available. Computer p~ograms were written to gen- erate machine listings ranked by descending frequency of criteria referenced in questions 1-6 and photocopy use, to list journals ranked by the Criteria Statistic Score, and finally to list all titles not appearing in any of the criteria lists. This last list, called a "noncriteria" list of titles, is derived by machine matching of the cumulated criteria lists against the master list of all titles held in the library. In this way, a list is gener- ated from the total data base for all titles that do not meet any of the criteria defined. The calculation for weighted criteria statistic scores was done manually. Criteria developed from the questionnaire resulted in the following ranked lists : 1. Journal titles used three or more times in past year (question 1). 2. Journal titles used one or two times in past year (question 2). 3. Journal titles not used but which have known references of interest (question 3). , 4. Journals in which AFGL published in past five years (question 4). 5. Journals cited by AFGL in their pub- lications over past five years (question 5). 6. Journals known to have published ar- .290 I College & Research Libraries • July 1978 tides citing AFGL publications (question 6). 7. Summation ranked score for titles ap- pearing in questions 1-3. 8. Summation ranked score for titles ap- pearing in questions 4-6. 9. Summation ranked score for titles ap- pearing in questions 3 and 5. 10. Journals ranked by Criteria Statistic Score. 11. "Noncriteria" list of journal titles in AFGL Research Library. CRITERIA STATISTIC SCORE In an effort to develop an overall ranking that reflects all criteria, a Criteria Statistic Score (CSS) was developed. This score is simply a ranking of the titles according to the number of criteria each journal title satisfies. If a title appears in the photocopy list and in each of the first nine criteria, it would have a CSS of 10. Table 1 shows a partial listing of the journals and their CSS. It will be noted that a number of titles have the same CSS, indicating a need for a more discriminating statistic. WEIGHTED CRITERIA STATISTIC SCORE Analysis of the results of the compilation of the CSS Statistic Score reveals that a weighted score might be more appropriate. To this extent the original CSS has been modified to provide a second ranking of criteria; namely, the Weighted Criteria Statistic Score (WCSS). The WCSS is calcu- lated in much the same way as the original CSS. However, the calculation is revised as follows: the summation of the frequency (or the number of times) that a given journal title appears in one of the ten ranked list- ings. (Thus if a title appears fourteen times in criterion 1 and twelve times in criterion 2 and sixteen times in criterion 3, it would have a weighted criteria score of forty-two.) It should be noted that there is some concern about using such a calculation to develop a weighted score. Obviously, the units involved are not entirely compatible. However, the attempt here is to develop a ranking of the journals according to criteria that relate to user requirements. In this re- spect, the WCSS provides a technique of showing the relative importance of ranking over a wider scale of values than the CSS statistic. Table 1 contains the WCSS that has been calculated for the first thirteen journal titles. TABLE 1 CRITERIA RANKING ANALYSIS Weighted Criteria Criteria Rank Photo- Statistic Statistic by Copy Journal Score (CSS) Score (WCSS) wcss Q. 1-3 Q. H Use 1. Journal of Geofthysical Research 10 181 1 1 1 2 2. Proceedings o the IEEE 10 92 2 4 3 8 3. ~lied Optics 10 90 3 2 6 (14) 4. ·o Science 10 84 4 (18) 2 9 5. Planetary and Space Science 10 81 5 7 4 5 6. Physical Review: A journal of Experimental an Theoretical Physics . 10 77 6 5 7 1 7. Nature. P~sical Science 10 75 7 3 9 3 8. Journal of e Atmospheric Sciences 10 74 8 9 5 (36) 9. Science 10 60 9 6 (16) (11) 10. Bulletin of the American Meteorological Society 10 56 10 (11) 10 (300) 11 . Review of Scientific Instruments 10 50 a~~ 8 a~~ ~~~ 12. Canadian Journal of Physics 10 53 10 13. Quarterly ~ournal of the Royal (11) (21) 8 (94) Meteoro ogical Society 10 54 14. Scientific American 10 15. Journal of the Optical Society of America 4 16. Journal of the American Chemical Society 6 17. Applied Physics Letters 7 CONCLUSION AND RESULTS The primary objective of this study was to identify those journal titles that could be de- leted from subscriptions or removed from the library holdings in order to satisfy the increased intensity of both financial and spatial constraints. The overall strategy of the study was to identify criteria that relate to user requirements and in a meaningful way to develop a list of journal titles that in one way or another relate to or satisfy these criteria. This was done, and 648 titles were identified as satisfying one or more of the criteria using the criteria obtained from the questionnaire, as well as . from photocopy use. Expressed another way, 648 titles had WCSS values ranging from 1 to 181. However, this leaves a very large number of titles on the noncriteria list. The strategy of the approach is to use the noncriteria list for the necessary weeding or streamlining of journal titles. Other criteria will of necessity have to be applied to the noncriteria list in order to reduce it to a more workable size. The first of these would be an analysis by members of the library staff to identify titles that from their experience might remain within the holdings of the library. Second, a review by the scientists at AFG L of titles suggested for deletion would be desirable before eliminating any titles from the hold- ings. Titles identified by these user criteria could be added to the ranked criteria list of titles to be retained. Regardless of how many criteria are applied, there will still remain a list of noncriteria journal titles. This list must then be analyzed according to the parameters dictated by the financial and spatial constraints. Specifically, these noncriteria journals should be ranked according to the cost of subscriptions arid according to the space available in the stacks. If the resulting list is truly a non-user-related criteria list, then we can analyze it relative to the financial and spatial constraints. At that point, those journals appearing on the noncriteria list that have the highest subscription costs would be eliminated first; those requiring the largest amount of space per title would be withdrawn on a high priority basis. An interesting outcome of the study is shown in table 1. The perceived ranking of Weighted Criteria I 291 journal titles is shown in five columns of the table while the last column shows figures more closely related to actual use (i.e., photocopying). Some of the items that are in the top ten for perceived need or use really rank very low in terms of photocopy use. Of course, this could be due to a number of factors that were not examined in the scope of this study and are not consid- ered critical as long as all criteria are used. In any case, the relationship between the critera was a secondary result of the study, as opposed to the primary objective of de- veloping a non-user-related criteria listing of journal titles. Another interesting relationship is that six of the top journal titles are common to all of the criteria (i.e., questions 1-6 and photo- copying and overall WCSS and CSS), shown in the table. A similar analysis of the top fifty journals, cited in the questionnaire or photocopy criteria, shows within these fifty titles there are twenty-seven journals that have all criteria in common. As inferred above, it is apparent that additional criteria should be explored relative to user-related criteria for the journal titles to remain in the library. The extent to which other criteria are de- veloped depends considerably on the mis- sion of the library. If the library is to be strictly user-requirement-oriented in terms of the current population of users, then the number of journal titles can be reduced drastically. However, if the mission of the library is to be a truly effective research li- brary, then additional parameters must be applied to the noncriteria list developed. We find, for example, at this stage of the study, only 25 percent of the current sub- scription journal titles are identified by any of user-related criteria. If we compare . the user-related journals to the total number of titles held in the library, namely 7,306, we find that only 8 percent appear in user- related criteria. It is obvious, therefore, that further analysis of the noncriteria list is nec- essary and that scientists and engineers at AFGL must be consulted before any journal titles are deleted. In summary, the ultimate goal of this study was to identify those journal titles that could be withdrawn from the library's hold- ings in order to alleviate critical financial 292 I College & Research Libraries • july 1978 and spatial constraints. The overall strategy was to identify criteria. that relate to user requirements and to develop a list of journal titles that in one way or another satisfy these criteria. A methodology for helping to solve the problems raised by constraints in the financial and spatial aspects of the library's journal holdings has been developed, and an attempt was made at providing an ultimate list of journals for subscription cancellation and possible with- drawal from the journal collection. It is clear that additional criteria must be de- veloped and applied but that the meth- odology described in the study allows for this development. REFERENCES 1. Richard W. Trueswell, "Some Behavioral Pat- terns of Library Users: The 80120 Rule," Wil- son Library Bulletin 44:458 (Jan. 1969). 2. Maurita Peterson Holland, "Serial Cuts vs. Public Service: A Formula," College & Re- search Libraries 37:.54348 (Nov. 1976). 3. Charles B. Wenger and Judith Childress, "Journal Evaluation in a Large Research Li- brary," Journal of the American Society for Information Science 28:293-99 (Sept. 1977). 4. Selected examples of other relevant articles include: B. C . Brookes, "Optimum P% Li- brary of Scientific Periodicals," Nature 232:458--61 (Aug. 13, 1971); B. Houghton and C. Prossen, "Rationalization of Serial Holdings in Special Libraries," As lib Proceedings 26:226-35 (June 1974); and E. F. Hockings, "Selection of Scientific Periodicals in an Indus- trial Research Library," Journal of the Ameri- can Society for Infonnation Science 25:131-32 (March-April 1974). ·.-4