College and Research Libraries The Application Of Social Judgment Analysis to Library Faculty Tenure Decisions Anne McCartt Using the analytical tool of Social Judgment Analysis, the author examines individual tenure policies. In a hierarchical judgment task, State University of New York at Albany librarians rated profiles of hypothetical candidates. Multivariate regression was used to devise weights and functional forms relating tenure criteria to the assigned ratings. Substantial differences were found for the over_all tenure decision and for judgments of research/ publication records. More similarities were identified for ratings of university service and organizational participa- tion. An illustrative candidate work sheet is presented. The author contends such an approach would increase the consistency and fairness of tenure decisions. m ach faculty member in an aca-demic institution is regularly called upon to evaluate col-. leagues for the purpose of granting promotion, continuing appoint- ment, or renewal of contract. Not only are individuals asked to make peer review judgments, but a group recommendation is usually made at the departmental level. Although the broad criteria for promotion and tenure are generally consistent across universities and colleges (i.e., research, teaching, and university service), the weights and interpretations applied to these criteria vary across campuses and, to some extent, across departments within a single institution. Within a department, individual faculty members may vary greatly in their respective judgment poli- cies, even if written guidelines exist. Be- cause faculty members may apply their judgment policies inconsistently, two in- dividuals with similar values may still dif- fer in their evaluation of a particular candi- date. This inconsistency, together with real disagreement over academic values, results in considerable group conflict, which may be iterated each time a col- league is evaluated. The present climate of cutbacks in higher education exacerbates this conflict, as faculty compete for fewer tenured slots. Recent court cases have questioned the cloak of confidentiality which has tradi- tionally surrounded the process of tenure and have focused attention on the rights of an individual faculty member within the context of academic freedom. 1 While one may argue that the tenure process is an inextricable part of academic institu- tional independence and a flexible tool for applying changing academic standards and goals, one could also question the "fairness" of a process which often in- volves many vague criteria that may be used in an inconsistent way. A fairer pro- cess might involve more clearly defined expectations for tenure, including the ranking or weighting of criteria and more precise measures for fulfillment of the cri- teria. The process would also ensure more consistent application of the criteria. While judgments of the quality of a per- son's work are always subjective to some degree, the way in which these judgments are integrated into an overall r;valuation Anne McCartt is a research associate with the Institute for Traffic Safety Management and Research, State University of New York at Albany. 345 346 College & Research Libraries need not be subjective or inconsistent. More precise, consistent policies and procedures would allow the peer review group to focus on the specific qualifica- tions of each candidate. More specific guidelines would also assist nontenured faculty members in understanding aca- demic expectations. Junior faculty mem- bers who are not meeting standards might then be more effectively identified prior to a tenure decision. ACADEMIC STATUS FOR LIBRARIANS Over the past few decades it has become increasingly common for universities and colleges to grant faculty status to academic librarians. 2 The precise definition of ''fac- ulty status" varies by institution, but the implementation of "full faculty status" usually means that librarians are ex- pected, officially at least, to fulfill the same criteria for promotion and tenure as teach- ing faculty. Full faculty status also implies that tenure, promotion, and term renewal judgments involve some degree of peer • 3 review. Peer evaluation may be a drastic change from the way in which personnel deci- sions in libraries were previously made; academic libraries have been more similar to traditional hierarchical, bureaucratic or- ganizations than have other academic de- partments. Thus, the collegial judgmental role may be an unfamiliar and uncomfort- able one for librarians. 4 Especially ~ the areas of teaching and research, librarians have also had difficulty fitting their func- tions into the professorial role model. In many academic libraries today, the spe- cific criteria and procedures for the ap;>li- cation of the criteria are still evolving. Because faculty status is a relatively re- cent phenomenon (and still a controver- sial one) and because librarians must fulfill criteria originally established for profes- sors, one would speculate that the promo- tion and tenure decisions in an academic library are characterized by more interper- sonal conflict than in other academic de- partments. In traditional academic depart- ments, the application of criteria to a particular candidate may produce heated September 1983 debate, but the overall judgment policy has a much longer history and has achieved a degree of stability over time. Compared to traditional academic facul- ties, members of library faculties have had less opportunity to approach consensus on general criteria for promotion and ten- ure and on measures for these criteria, and these criteria are less likely to be clearly understood and accepted by faculty mem- bers. The basis for group conflict often may not be the qualifications of a particu- lar candidate and the degree to which a candidate meets faculty standards, but rather, a lack of consensus on which stan- dards have been, and should be, applied. This paper examines tenure policies ex- ercised by individual librarians at the State University of New York at Albany (SUNYA). Descriptions of three librari- ans' individual tenure policies are de- rived, using the analytical tool of Social Judgment Analysis. Differences between policies are examined, and a model pre- sented which could be used to facilitate deliberations for a tenure candidate. While this paper reports on judgments within a specific environmental context, the methodology is applicable to other ac- ademic environments. SOCIAL JUDGMENT ANALYSIS Judgment can be defined as an inferen- tial cognitive process whereby a person uses available information to draw conclu- sions about unknown qualities or quanti- ties. 6 The process by which pieces of infor- mation are integrated into a single judgment is. often referred to as an indi- vidual's judgment policy. Two basic ap- proaches are taken to discover a person's judgment policy. The first is to elicit a de- scription of the policy directly from the person. While this is the most commonly used method, an accurate description can- not be obtained until an individual is fully aware of what are typically, very complex intuitive processes. A large body of re- search has shown that such subjective ac- counts often produce inaccurate represen- tations of the actual internal judgment process. 7 The second method, embodied in Social I Judgment Analysis, is to infer the policy through an empirical analysis of actual judgments. Social Judgment Theory fol- lows from Tolman and Brunswick's ap- proach to cognition. 8 Social Judgment the- orists posit that the limitations of human cognition are best reduced through the ex- ternalization of the internal judgment pro- cess. According to Social Judgment The- ory, the integration of information forming judgment includes (a) placing a "weight," or measure of relative impor- tance, on each piece of information; (b) de- veloping a specific relation between each piece of information and the overall judgment-referred to as "function form"; and (c) using a particular method for integrating the dimensions of the problem-called the ''organizing princi- ple.' ' 9 Social judgment researchers have found that individual differences in these aspects of information usage, and incon- sistencies in the application of judgment policies, lead to disagreement in judg- ments and interpersonal conflict. 10 Quantitative procedures have been de- veloped to provide externalizations of judgment policies. Based on repeated judgments under a variety of well- specified conditions, an individual's judg- ment policy can be captured mathemati- cally through the use of multiple regression procedures. The criterion vari- able is the individual's judgment, and the predictor variables are the dimensions of the situation judged. The regression equa- tion provides the organizing principle, as well as the weight and function form for each dimension of the problem. 11 Social Judgment Analysis provides a means to empirically derive descriptions of judgment policies, thereby revealing in- terpersonal similarities and differences. Clarification of judgment policies not only provides insight for an individual, but it also facilitates the resolution of conflict by allowing individuals to focus on the real differences and similarities of their judg- ment policies. The technique can also be used to predict a person's future judg- ments. Finally, Social Judgment Analysis can be used as the basis for developing a model for structuring future decisions. The Application of Social Judgment 347 BACKGROUND ON SUNYA LIBRARY CRITERIA FOR TENURE State University of New York at Albany (SUNY A) librarians have had full faculty status for more than a decade. The four ac- ademic ranks are as follows: assistant li- brarian, senior assistant librarian ( estab- lished in 1977), associate librarian, and librarian. Librarians are typically given initial appointment at one of the two lower ranks. Because persons must be given one "grace year," the tenure process usually begins during a person's fifth year. 12 The SUNYA Library academic faculty has extensively revised its criteria and pro- cedures several times, most recently in 1978. The current document is designed to abide with campus and statewide SUNY policies, but reflects the special role of li- brarians within the institution. Thus, the introduction to ''Evaluation of Library Ac- ademic Faculty for Promotion and Contin- uing Appointment" states: The Policies of the Board of Trustees of the State University of New York define librarians as aca- demic faculty, and thereby recognize that li- brarians make a fundamental intellectual con- tribution to the educational, research, and public service missions of the university. The Policies also provide that librarians are a sepa- rate and distinct Academic Faculty group. This provision recognizes that the intellectual contri- butions of the Library Academic Faculty of the University differ in kind and emphasis from those of the teaching faculty. Thus, although the criteria by which librarians are judged must relate to the general criteria ap- plied to all Academic Faculty, they must also re- late to the unique role of librarians in the aca- demic community. The introduction to the specification of criteria states that "the basic quality which must be evident for promotion in aca- demic rank and/or continuing appoint- ment is the ability to perform at a high pro- fessional level in areas which contribute to the mission of the institution. II Three broad criteria, "intended to serve as gen- eral guidelines, II are given, along with some explanatory comments. These three criteria are as follows: 1. Effectiveness as a librarian (job perfor- 348 College & Research Libraries mance and continuing growth and devel- opment) 2. Contributions to the advancement of the profession (participation in professional/ scholarly organizations, research and publication, consultancies, presentations) 3. University service (involvement in li- brary or university committees and '' ap- propriate" community organizations) The "Guidelines for Application of Cri- teria for Continuing Appointment" state that ''Effectiveness as a librarian'' must have been fulfilled "in an outstanding manner." Additionally, the candidate must also demonstrate "professional ac- tivities of high quality'' in fulfilling the re- maining criteria and ''show evidence that such contributions will continue." In summary, the library guidelines are very general and allow a great deal of individ- ual discretion in applying the criteria to tenure candidates. The internal library procedures for pro- motion and tenure decisions involve a meeting of the library academic faculty, where each candidate is discussed and a secret vote taken. The results of that meet- ing and the recommendation of the library director are then forwarded to the campus-wide University Council on Pro- motion and Tenure. PARTICIPANTS Three librarians were asked to partici- pate in the study. While no attempt was made to use a true sample of the entire professional staff, the participants repre- sented a variety of backgrounds and di- verse work experiences. Participants in- cluded a nontenured senior assistant, a tenured senior assistant, and a tenured as- sociate . Of the four library departments, only the technical services department was not represented. All three persons had participated in a large number of ten- ure and promotion decisions . TASK The participants were given profiles of hypothetical tenure candidates and asked to rate each candidate. The pool of actual candidates over the past few years was not large enough to permit use of real cases, but great care was taken to use profiles September 1983 which were realistic representations of typical candidates considered for tenure. Based on discussions with the partici- pants, five important and discriminating candidate characteristics were chosen. Al- though these characteristics may not be all-inclusive, it was felt that these five cri- teria are the most salient considerations upon which tenure judgments are based, and usually provide sufficient information to make a decision. The five criteria are as follows: 1. Job performance 2. Educational credentials 3. Participation in professional or schol- arly organizations 4. Research and publication record 5. University service An additional criterion, library position, was considered, but eliminated because of its '' configurality. '' That is, it was felt that the position of a candidate (e.g., cataloger) completely changes the way in which the other information is utilized. To deal with this issue, the profiles all represented li- brarians from a single department, refer- ence and collection development services, by far the largest department. Once the major factors affecting tenure judgments were identified, cases were constructed which were representative of the environment. When Social Judgment Analysis is used, it is first necessary to es- tablish the precise range of values for each criterion. Second, the values on the crite- ria must systematically vary in such a way that the interrelationships between crite- ria and the distribution of criterion values are realistic . When based on participant input, these two steps help ensure that the criteria are well defined, representative, and are uniformly interpreted by all the judges and by the decision analyst. Quality of job performance and educa- tional credentials were easily defined and understood by the participants. The other three criteria, however, were more com- plex variables and definition of these be- came judgment subtasks. For example, the meaning of a "superior" research and publication record is not immediately ob- vious; evaluation of a candidate's research and publication record is itself a complex judgment, where various types of publica- tions and other related activities are weighted differently, and · trade-offs are made between the quantity and quality of different types of achievements. There- fore, a hierarchical judgment task was constructed (see figure 1). Subtasks were administered to clarify the precise mean- ing of a high or low value for · the other three criteria. As seen in figure 1, a rating for participa- tion in professional organizations was based on the extent of an individual's in- volvement in organizations at the local, state, and national levels, while a univer- sity service record has two underlying di- mensions: the level of service within the li- brary, and level of service within the Level of Involvement Level of Involvement · . The Application of Social Judgment 349 wider university community. Both tasks consisted of 25 hypothetical candidate profiles. The subtask for research and publication consisted of six variables, as follows: number of presentations at pro- fessional meetings, number of library ''in- house" publications, number of book re- views, quality of journal or newsletter edited, number of articles in refereed jour- nals, and number of articles in nonrefer- eed journals. Participants were asked to judge 36 research/publication profiles. For the overall tenure judgment, 37 candidate · profiles were presented. For each judgment task the judges were asked to assign a number between 1 and 20 to each candidate profile, where 20 rep- in Local Organizations ?7 in State Organizations Participation in Professional Level of Involvement · Organizations in National Organizations Level of Library Service ~ University Service Level of External Service Number of Presentations at Professional Meetings Quality of Journal or Newsletter Edited Number of Articles in Refereed Journals Number of Articles in Non-Refereed Journals Record of Research and Publication Educational Credentials FIGURE 1 Hierarchical Tenure Judgment Task TENURE JUDGMENT 350 College & Research Libraries September 1983 resented a superior rating. In all four criterion ranges. Participants were asked tasks, the cases were created in such a way to assume that the evidence in each case that the criteria were totally uncorrelated. clearly supported the record presented. Then unrealistic candidate profiles were RESULTS eliminated, introducing a small degree of intercorrelation between criteria. The Descriptions of judgment policies for in tercorrela tions, however, remained each individual were derived using a non- quite low, ranging from .00 to -.24 for the linear, additive multiple regression three subtasks and from . 00 to -.18 for the model. Qqadratic and linear forms of each overall tenure task. criterion were included as the indepen- All four tasks were presented in the dent variables. For each task the regres- same format and administered at the same sion analysis identified the weight and time. An example of a judgment task is function form relating each candidate shown in figure 2. For this task the partici- characteristic to the appropriate rating as- pant was given a series of hypothetical re- signed. Once the individual policies were search and publication profiles and asked delineated, it was then possible to com- to rate each profile with a number be- pare the policies of the participants and to tween 1 (very poor record) and 20 (supe- develop a system for evaluating future rior record). The instructions accompany- tenure candidates. ing the task included a description of the Each regression analysis produced a Quality Presentations Library of Journal/ Articles in Articles in at Professional "In-house" Book Newsletter Refereed Nonrefereed Meetin~s Publications Reviews Edited Journals Journals Cases Rating: 1-20 1 1 1 14-15 2 1 1 2 2 5 4-6 2 1 4 3 7 1 4-6 1 3 0 4 2 1 2-3 6 0 4 5 2 0 14-15 0 3 3 6 1 5 0-1 5 1 2 7 6 0 12-13 3 1 3 8 0 4 7-9 2 1 2 9 5 2 4-6 3 1 3 10 3 1 7-9 2 1 3 11 2 2 4-6 3 1 0 12 3 1 7-9 3 0 3 13 2 2 7-9 0 2 1 14 3 2 0-1 4 0 3 15 3 0 10-11 1 1 1 16 0 3 2-3 3 0 1 17 4 0 7-9 1 0 3 18 3 0 2-3 4 1 0 19 1 0 10-11 3 0 4 20 4 2 0-1 6 3 1 21 3 0 14-15 5 0 4 22 5 4 14-15 1 0 2 23 5 5 2-3 0 1 3 24 1 0 4-6 4 3 3 25 0 1 10-11 5 2 3 26 2 3 12-13 4 2 2 27 4 4 10-11 4 1 4 28 5 3 10-11 2 3 0 29 4 1 12-13 0 2 4 30 2 4 0-1 3 3 4 31 3 5 10-11 6 3 3 32 6 3 14-15 6 2 3 33 4 5 14-15 3 2 0 34 7 2 10-11 0 0 1 35 3 3 0-1 0 0 2 36 4 0 0-1 1 1 4 Cue Ranges 0-7 0-5 0-15 0-6 0-3 0-4 FIGURE2 Research and Publication Judgment Task The Application of Social Judgment 351 TABLE 1 MULTIPLE CORRELATION COEFFICIENTS Organizational University Research/ Participation Service Publication Tenure Judge 1 .87 .93 .69 .95 Judge 2 .90 .95 .87 .96 Judge 3 .93 .97 .94 .96 TABLE 2 MEAN JUDGMENTS Organizational ParticiEation Judge 1 12.7 Judge 2 10.7 Judge 3 11.7 multiple correlation coefficient, R, an in- dex of the fit of the judgment regression model to the person's actual judgments. A high multiple R 13 means that the model provides a good representation of the in- ternal judgment policy, that the model can be used to predict future judgments accu- rately, and that the judge is applying the policy consistently. However, a low multi- ple R could be due either to inconsistency on the part of the judge, or to the model's failure to capture the judge's policy. The multiple correlation coefficients are shown in table 1 and are generally quite high, with only three less than . 90 and only one less than .85. This means that the judgment policy descriptions derived are excellent representations of the actual judgment policies exercised by the three participants. DESCRIPTION AND COMPARISON OF POLICIES One way in which individuals may vary is the number of candidate profiles identi- fied as "tenurable." In a general sense, some judges set a higher standard for ten- ure than others. Mean judgments for each judge were computed and are shown in table 2. With the exception of the overall tenure judgment, judge 1 had the highest mean judgment and judge 2 the lowest. Judge 1 rated the candidate profiles most favorably in completing the three judg- ment subtasks. The differences between judges, however, were relatively small. On the tenure judgment task, judge 3 had the highest mean rating, with judges 1 and 2 having the same mean judgment. University Research/ Service Publication Tenure 10.4 13.5 9.6 9.8 12.5 9.6 10.0 12.8 10.9 Interestingly, the mean judgment for ten- ure for all three judges fell close to 10, the arbitrary cutoff point for a negative tenure decision. Individuals may also differ in the rela- tive importance given a particular crite- rion. Relative criterion weights for the three subtasks are listed in table 3. Each weight represents the relative contribu- tion of the criterion to the overall judg- ment and captures the contributions of both the linear and quadratic forms of the criterion. All three judges had similar weighting schemes for the university service sub- task. Service external to the library was al- ways weighted more heavily than internal library service, although the weights as- signed to external service range from .56 (judge 1) to .69 (judge 3). For the organizational participation sub- task, judges 1 and 2 distributed the weights fairly evenly across the three criteria-level of involvement in local, state, or national organizations. Judge 3, however, assigned no importance to the level of local organizational participation and gave a weight of .63 to activities in na- tional organizations. Not unexpectedly, the judges differed dramatically in their weighting schemes for the research and publication subtask. None of the judges used all six criteria in rating the research and publication pro- files, and judge 1 and judge 2 based their judgments on completely different crite- ria. Judge 1 used three criteria, as follows: number of library publications, number of book reviews, and the quality of journal or 352 College & Research Libraries September 1983 TABLE3 RELATIVE CRITERION WEIGHTS Criteria Task 1: University Service Library service University service Task 2: Organizational Participation Local organizations · State o~anizations Nation organizations Task 3: Research and Publication Number of trresentations Number of "brary "in-house" publications Number of book reviews Quality of ;ournal or newsletter edited Number o refereed journal articles Number of nonrefereed journal articles Task 4: Overall Tenure Judgment Quality of job performance Educational credentials Partic~ation in professional organizations Recor of research and publication Unversi!Y service newsletter edited. An editorship was given a weight of .48, nearly equal to li- brary publications (.28) and book reviews ( .24) combined. Judge 2 used only two cri- teria: a weight of .70 was given to the number of presentations at professional · meetings, and a much lower weight of .30 was given to the number of articles in ref- ereed journals. Judge 3, using four crite- ria, found the number of articles in refer- eed journals (.45) most important. The number of articles in nonrefereed journals and an editorship of a newsletter or jour- nal were weighted almost equally at .24 and .23 respectively. A relatively low weight of .08 was given to the number of book reviews. The three judges also used markedly different weighting schemes in evaluating candidates' overall qualifications for ten- ure. Judge 1looked at job performance as the sole criterion when evaluating profiles for tenure. Judge 2 used all five criteria but also placed the greatest importance on job performance (.60). The remaining four cri- teria all received similar weights, ranging from .09 to .12. Judge 3 used all criteria ex- cept educational credentials. Most heavily weighted was the research/publication criterion (.53). Job performance was given Jud~e 1 Jud~e 2 Jud~e3 .44 .36 .31 .56 .64 .69 .28 .32 .0 .36 .26 .37 .36 .42 .63 0 .70 0 .28 0 0 .24 0 .08 .48 0 .23 0 .30 .45 0 0 .24 1.0 .60 .34 0 .12 0 0 .09 .08 0 .10 .53 0 .10 .06 a weight of .34, with university service and organizational participation consid- ered much less important with weights of .06 and .08, respectively. A weighting scheme alone does not- completely describe an individual's judg- ment policy. Another component is the functional form relating each criterion to the judgment. For the subtasks in organi- zational participation, university service, and for the tenure task, the functional forms for the three judges were all increas- ing functions, although not all were lin- ear. The function form for the research and publication record subtask are pre- sented in figure 3, along with the relative weights. With the exception of library publications, the function forms increase. That is, the higher the level on the cue, the higher the given rating. The function for number of library publications, used only by judge 1, decreases up to three publica- tions, and then increases. Taken together, the function forms and weights for each judge describe the judgment policy for evaluation of research and publication rec- ords. As figure 3 shows, the three judges had fundamental disagreements over how a particular record of research and publication should be evaluated. For ex- RESEARCH AND PUBLICATION RECORD Number of presentations at Professional Meetings 20 The Application of Social Judgment 353 FUNCTION FORMS RELATIVE WEIGHTS 0123t567 Judge1- Judge2 --- Judge 3 •······· Judge 1: 0 Judge 2: 70 Judge 3: 0 Number of Library "In-House" Publications 20u Judge 1: 28 Judge 2: 0 Judge 3: 0 Number of Book Reviews 20~1 2 3 4 ~.·· . . . . Judge 1:24 Judge 2: 0 Judge 3: 8 0-1 7 9 14-15 Quality of Journal or Newsletter Edited 20ld······· ·········· .... .. ····· .. Judge 1: 48 Judge 2: 0 Judge 3: 23 1 2 3 4 ~ Number of Articles in Refereed Journals 20LL_-: .. / : / .· / .-· / .-· / .- -· ~-··· · · Judge 1: 0 Judge 2: 30 Judge 3: 45 1 2 3 Number of Articles in Non-refereed Journals 101 ...-> ..... ~ Judge 1: 0 Judge 2: 0 Judge 3: 24 1 FIGURE3 Judgment Policies for Research and Publication Subtask ample, a candidate whose activities had been published in refereed journals, or presented at professional meetings, would receive a high rating from judge 2, a 0 rating from judge 1, and a mediocre rat- ing from judge 3. In summary, Social Judgment Analysis revealed substantial differences among the three participants in their evaluations of hypothetical research and publication records and of hypothetical profiles of ten- ure candidates. The analysis revealed more similarities in judgment policies for university service and organizational par- ticipation. Disagreement over what con- stitutes a ''superior'' research/publication record is not surprising; as with faculty status, an increasing emphasis on re- search and publication in academic li- braries is a relatively recent phen.omenon. Conflict over the relative importance of re- search and job performance could also be 354 College & Research Libraries predicted; similar arguments over the pri- macy of classroom teaching or research and publication have been ongoing for many ~ears in other academic depart- ments. DEVELOPMENT OF A CANDIDATE WORKSHEET Research in small group processes has identified a number of factors which con- tribute to interpersonal conflict. These fac- tors include cognitive differences, incon- sistencies in the application of individual judgment policies, and the structure of judgment tasks, as well as self-interest or emotion. In the absence of accurate and mutually understood descriptions of judgment policies, interpersonal differ- ences may be attributed to personal ambi- tion or ignorance. Even when systematic differences in policies are eliminated, con- flict may persist because of the inconsis- tent application of a given policy by differ- ent individuals. Thus, conflict may continue because people disagree about the importance of factors affecting a deci- sion and/ or because individual judgment policies are not mutually understood or consistently applied. 15 Substantial reduction of conflict can be achieved, however, through the external- ization of judgment policies. Externaliza- tion promotes conflict resolution by ena- bling each individual to understand one's policy and to realize the implications of the consistent application of that policy. When individual policy descriptions are mutually shared by members of a group, communication and understanding are enhanced; differences and similarities and potential areas of compromise can be identified. Furthermore, the comparison of policies can be accomplished within a common perceptual framework and through a common vocabulary. Discus- sion turns from the particular case to the characteristics of the task itself. 16 External- ization of judgment policies, or II cognitive feedback," through Social Judgment Analysis has been demonstrated to be helpful in a variety of group problem- solving situations; conflict has been sub- stantially reduced or eliminated. 17 September 1983 The judgment policies presented in this paper reflect substantial disagreements among the three judges concerning the way in which tenure criteria should be used to evaluate candidates. Although the multiple correlation coefficients are gener- ally high, there also appears to be some in- consistency present in the most complex judgment task, research and publication. Based on the research reported above, however, this initial disagreement, likely representative of the faculty as a whole, could be reduced significantly through the externalization of judgment policies through Social Judgment Analysis. When descriptions of individual policies, based on mutually understood criteria, are pro- vided, faculty consensus could emerge on the weights and function forms. Finally, the results of Social Judgment Analysis can serve as the basis for devel- oping a system for evaluating tenure can- didates. It is not suggested that a rating system replace faculty deliberations, but rather that the system serve as the point of departure for faculty discussion of a candi- date. The system would reflect library fac- ulty consensus on the appropriate weights and function forms for the tenure criteria. It would be understood, of course, that the criteria could not be inclu- sive of all possible relevant activities. Dis- cussion for each tenure case could focus first on how well the candidate performed against the group judgment model and would then turn to additional relevant as- pects of the candidate's record or any ex- tenuating circumstances. To demonstrate the way such· a system works, the policies of judge 3 were used to develop an illustrative candidate work- sheet. In figure 4, the performance of a hy- pothetical tenure candidate is given for each criterion. For example, this candi- date's research and publication record in- cludes five presentations at professional meetings, one library "in-house" publica- tion, four to six book reviews, editing a newsletter for a local organization, and four journal articles (two published in ref- ereed journals). Summary of a candidate's qualifications could be completed by the committee responsible for collecting the SUB· TASK 1: PARTICIPATION IN PROFESSIONAL ORGANIZATIONS 1. Local Organizations None Membership Leadership Score 0 !I] 2 4 Rating Weight ; O 2. State Organizations None Membersh~ Leadership Score 0 1 2 4 Rating 0 .3 1.5 2.3 . 3. National Organizations None Membership Leadership Score 0 1 G:J 4 Rating .9 2.8 3.7 SUB-TASK I Rating SUB-TASK II : UNIVERSITY SERVICE 1. External University Service Score Rating None Moderately 1 .8 active 2 1.9 3.1 Extremely active ~ 2. Library Service None Moderately Extremely 0 .8 1.9 2.7 4.1 Score Rating 1 .3 active activ~ 2 5 G:J 1.9 1.1 SUB-TASK II RATING __ 5._2 _ SUB-TASK Ill: RESEARCH AND PUBLICATION RECORD 1. Number of Presentations at Professional Meet~s Score 0 1 2 3 4 5 W Rating Weiqht ; 0 2. Number of Library ·· In-House " Publications Score 0 EJ 2 3 Rating Weight = 0 3. Number of Book Reviews Score 0·1 2·3 14-61 7·9 10-11 t2·13 14·15 Rating 0 .1 L!§J .2 .3 .4 .5 4. Quality of Newsletter or Journal Edited Score 0 ~ 2 3 4 Rating 0 .3 .5 .7 .9 5. Number of Article in efereed Journals Score 0 1 f2l 3 Rating o . 7 L.!2J 2. 7 5 t.2 6. Number of Articles in Non-Refereed Journals Score 1 r2l 3 4 1.4 .15 .3 1.5 Rating .15 LiJ 1 1.4 .4 SUB-TASK Ill RATING--2.-35- The Application of Social Judgment 355 OVERALL TENURE JUDGMENT 1. ParticipatiOn in Protess1onal O rq amz a h on-; Minimal Moderate Exceplio ~al Score 0 1 2 3 5 6 Rating 0 .3 .6 .8 1 1 1 4 1 7 2. University Service Minimal Score 0 Rating 0 .2 Moderate 2 3 4 .3 .6 8 3. Research and Publication Record lead e rship 5 6 .9 1.1 Poor [1jverage Superior Score 0 1 2 3 4 5 6 Rating 0 1.7 3.5 5.3 7.0 8.8 10 .5 4. Job Performance Poor Average Superior Score 0 2 3 rTl 5 6 Rating 0 .7 1.7 2.7 L..iQJ 5.0 6 .7 Educational Credentials MLS Score 0 Rating MLS & Subject Master's 3 4 Weight 0 Doc,l.l!.{ate 51!.J OVERALL TENURE RATING .8 .9 3. 5 4.0 _ _ o __ 9 . 2 FIGURE4 Illustrative Rating Sheet Based on Judge 3's Judgment Policies evidence for each tenure case, with the scores based on the information included in the candidate's vita. A subtask for job performance might also be desirable, to derive more precise measures for the qual- ity of work performed by a candidate. Using weights and function forms de- rived from Social Judgment Analysis, the performance scores for the three subtasks can be converted into ratings. To facilitate the conversion of performance scores into weighted evaluative ratings, the work- sheet shows the corresponding weighted judgment rating directly under each per- formance score. For example, the ratings for research and publication in figure 4 in- corporate the functions and weights for judge 3listed in figure 3. An overall evalu- ation of 2.35 (from a possible 6 points) for research and publication is obtained by summing the weighted ratings of 0 for presentations, 0 for library publications, .15 for book reviews, .3 for editorship, 1.5 for refereed journal articles, and .4 for nonrefereed journal articles. Similar cal- culations result in performance ratings of 2. 7 for organizational participation and 5.2 for university service. The subtask evaluations, together with scores for job performance and educa- tional credentials, serve as the basis for an overall tenure judgment. The relative weights and function forms for the five cri- teria once again provide a precise relation- ship between the scores and the tenure judgment. In the hypothetical case shown in figure 4, the candidate receives an over- all tenure rating of 9.2 out of 20 possible points. Of the 9.2 points, 4.0 are derived from job performance, .8 from organiza- ----1 356 College & Research Libraries tiona! participation, . 9 from university ser- vice, and 3.5 from research/publications. Although the candidate has a relatively strong record in university service, this criterion was not weighted highly by judge 3. Given a weight of .53 for research and publication, the candidate's case is weakened by a below-average research and publication record. The fact that the candidate has a doctorate degree does not contribute to the overilll score, since judge 3 gave a zero weight to educational cre- dentials. Unless the faculty had established an absolute cutoff point for tenure, the score of 9.2 does not in itself indicate a decision. If figure 4 represented a group judgment, · discussion might turn to those accom- plishments, for example, consulting, teaching, not covered by the worksheet. The worksheet could not replace faculty deliberations. However, if based on a fac- ulty consensus on the relative importance of criteria and the relationship of each cri- September 1983 terion to the tenure judgment, such a worksheet could increase the consistency with which the policies of the library are applied to candidates. CONCLUSION · Using the tool of Social Judgment Anal- ysis, tenure policies for three academic li- brarians were derived. These policies show wide discrepancies in the way in which these librarians evaluate hypotheti- cal candidates for tenure. The most strik- ing differences were found among the judgment policies for research and publi- cation records, and for the overall tenure· profiles. More similarities were revealed in the judgment policies for university ser- vice and organizational participation. A candidate rating sheet was also devel- oped, illustrating how the derived policies can be used to rate each candidate. It is the author's contention that such an approach would increase the consistency with which candidates are judged. REFERENCES AND NOTES 1. Dene Kleiman, "Academic Women Show Gains in Combating Sex Discrimination, " New York Times, (15 July 1980), p.1; Edward B. Fiske, "Jailing of a Professor Heightens Fears for Campus Independence," New York Times, (14 Sept. 1980), p.1; Edward B. Fiske, "The Courts in Tenure Cases: Must Faculty Disclose Votes," New York Times, (1 Dec. 1981), sec. 3, p.1. 2. Virgil F. Massman, Faculty Status for Librarians (Metuchen, N.J .: Scarecrow, 1972), p .11-21. 3. Ibid, p.14-15; "Statement on Faculty Status of College and University Librarians," College & Re- search Libraries News, 35 :26 (Feb. 1974); "Standards for Faculty Status for College and University Librarians," College & Research Libraries News, 35:112-13 (May 1974). 4. Martha J. Bailey, "Some Effects of Faculty Status on Supervision in Academic Libraries," College and Research Libraries 37:49-50 (Jan. 1976); Dennis W. Dickinson, "Some Reflections on Participa- tive Management in Libraries," College and Research Libraries 39:257 Quly 1978). 5. Massman, Faculty Status, p.60-63, 200; Beverly J. Toy, ed., "The Role of the Academic Librarian: A Symposium,'' Journal of Academic Librarianship 4:128-38 Quly 1977); Association of College andRe- search Libraries, Systems and Procedures Exchange Center, The Status of Librarians: An Overview, Spec Kit #61 (Washington, D.C.: Feb. 1980), p.2. 6. John Rohrbaugh, "Improving the Quality of Group Judgment : Social Judgment Analysis and the Delphi Technique," Organizational Behavior and Human Performance 24:73-92 (Aug. 1979). 7. See for example, Paul J. Hoffman, ''The Paramorphic Representation of Clinical Judgment,'' Psy- chological Bulletin 57:116-31 (Mar. 1960); Paul Slovic, "Analyzing the Expert Judge: A Descriptive Study of a Stockholder's Decision Processes,'' Journal of Applied Psychology, 53:255-63 (Aug. 1969); Richard L. Cook and Thomas R. Stewart, ''A Comparison of Seven Methods for Obtaining Subjec- tive Descriptions of Judgmental Policy," Organizational Behavior and Human _Performance 13:31-45 (Feb. 1975); Walter Morely Balke, Kenneth R. Hammond, and G. Dale Mayer, "An Alternative Approach to Labor-Management Negotiations," Administrative Science Quarterly 18:331-27 (Sept. 1973). 8. Egon Brunswick, ''Organismic Achievement and Environmental Probability,'' Psychological Re- view 50:255-72 (May 1943); Edward C. Tolman, Purposive Behavior in Animals and Men (New York: Century, 1932); Edward C. Tolman and Egon Brunswick, "The Organism and the Causal Texture of the Environment," Psychological Review 42:43-77 Qan. 1935). I The Application of Social Judgment 357 9. John Rohrbaugh, "Making Decisions about Staffing Standards: An Analytical Approach to Hu- man Resources Planning in Health Administration,'' (manuscript submitted for publication), p. 7. 10. Kenneth R. Hammond, "New Directions in Research on Conflict Resolution," Journal of Social Issues, 21:44-66 (July 1965); Kenneth R. Hammond and Berndt Brehmer, "Quasi-rationality and Distrust: Implications for International Conflict,'' in Human Judgment and Social Interaction, Leon Rappoport and D. A. Summers, eds. (New York: Holt, 1973); Kenneth R. Hammond and others, "Social Judgment Theory," in Human Judgment and Decision Processes; Formal and Mathematical Approaches, Martin F. Kaplan and Steven Schwartz, eds. (New York: Academic, 1975) . 11 . Kenneth R. Hammond "Probabilistic Functioning and the Clinical Method," Psychological Review, 62:255-62 (July 1955); Hoffman, "The Paramorphic Representation." 12. Faculty appointed at the higher ranks are usually considered for tenure in the first year of the term appointment, with tenure available after three years. Such faculty are typically given an initial two-year visiting librarian appointment prior to the term appointment. 13. Generally, multiple Rs between .70 and .80 are regarded as indicative of a statistically adequate judgment model. See Kenneth R. Hammond and others, "Social Judgment Theory: Applications in Policy Formation,'' in Human Judgment and Decision Processes in Applied Settings, Martin F. Kaplan and Steven Schwartz, eds. (New York: Academic, 1977). 14. Massman, Faculty Status, p .60-63. 15. Berndt Brehmer, ''Social Judgment Theory and the Analysis of Interpersonal Conflict,'' Psycholog- ical Bulletin, 83:985-1001 (Nov . 1976). 16. Derick 0. Steinmann and others," Application of Social Judgment Theory in Policy Formulation: An Example," The Journal of Applied Behavioral Science 13:71-72 (Jan.-Feb.-Mar. 1977). 17. See, for example, Leonard Adelman, Thomas R. Stewart, and Kenneth R. Hammond, "A Case History of .the Application of Social Judgment Theory to Policy Formulation, 11 Policy Sciences, 6:137-59 (June 1975); Kenneth R. Hammond and others, Report to the Denver City Council and Mayor Regarding the Choice of Handgun Ammunition for the Denver Police Department, Program of Research on Human Judgment and Social Interaction Report No. 179 (Boulder: Institute of Behavioral Sci- ence, University of Colorado, 1975); John Rohrbaugh, "Improving the Quality of Group Judg- ment: Social Judgment Analysis and the Nominal Group Technique, 11 Organizational Behavior and Human Performance, 28:272-88 (July 1981); Rohrbaugh, 1979; Rohrbaugh, manuscript.