The Influence of Academic Values on Scholarly Publication and Communication Practices UC Berkeley Research and Occasional Papers Series Title The Influence of Academic Values on Scholarly Publication and Communication Practices Permalink https://escholarship.org/uc/item/77j889gf Authors Harley, Diane Earl-Novell, Sarah Arter, Jennifer et al. Publication Date 2006-09-01 eScholarship.org Powered by the California Digital Library University of California https://escholarship.org/uc/item/77j889gf https://escholarship.org/uc/item/77j889gf#author https://escholarship.org http://www.cdlib.org/ Research & Occasional Paper Series: CSHE.13.06 UNIVERSITY OF CALIFORNIA, BERKELEY http://cshe.berkeley.edu/ THE INFLUENCE OF ACADEMIC VALUES ON SCHOLARLY PUBLICATION AND COMMUNICATION PRACTICES September 2006 Diane Harley∗, Sarah Earl-Novell, Jennifer Arter, Shannon Lawrence, and C. Judson King† Center for Studies in Higher Education University of California, Berkeley Copyright 2006 Diane Harley et al., all rights reserved. ABSTRACT This study reports on five disciplinary case studies that explore academic value systems as they influence publishing behavior and attitudes of University of California, Berkeley faculty. The case studies are based on direct interviews with relevant stakeholders— faculty, advancement reviewers, librarians, and editors—in five fields: chemical engineering, anthropology, law and economics, English-language literature, and biostatistics. The results of the study strongly confirm the vital role of peer review in faculty attitudes and actual publishing behavior. There is much more experimentation, however, with regard to means of in-progress communication, where single means of publication and communication are not fixed so deeply in values and tradition as they are for final, archival publication. We conclude that approaches that try to "move" faculty and deeply embedded value systems directly toward new forms of archival, "final" publication are destined largely to failure in the short-term. From our perspective, a more promising route is to (1) examine the needs of scholarly researchers for both final and in- progress communications, and (2) determine how those needs are likely to influence future scenarios in a range of disciplinary areas. Many opportunities and concerns are at play in scholarly communication and publication. These result from capabilities afforded by new technologies, pressures associated with the purchasing power of library budgets,1 marginal operations by university presses,2 and the pricing structures of the publishing industry.3 Many of those involved in supporting new publishing and communication ventures see “the lack of willingness of ∗ Diane Harley, Ph.D. is the co-Principal Investigator of this project, with C. Judson King. She is a senior researcher at the Center for Studies in Higher Education at the University of California, Berkeley and directs the Higher Education in the Digital Age (HEDA) Project. † C. Judson King is former Provost and Senior Vice President – Academic Affairs of the University of California and is Director of the Center for Studies in Higher Education. Harley et al., ACADEMIC VALUES AND SCHOLARLY PUBLICATION 2 the faculty to change” as a key barrier to moving to more cost-effective publishing models in an environment of escalating costs and constrained resources.4 This article summarizes the results of a planning study carried out during the 2005-6 academic year and funded by the Andrew W. Mellon Foundation. In it we describe our assessment of the criteria by which faculty decide when and in what venues to publish or otherwise communicate the results of scholarly research. We were interested in how faculty values relating to advancement and stature in their fields affect these decisions. This was the first step toward making a nuanced and insightful analysis of the roles that universities and faculty do and can play in the resolution of the perceived “crisis in scholarly communication.” Specifically, we developed five disciplinary case studies that are based on direct interviews with relevant stakeholders—faculty, advancement reviewers, librarians, and editors—in five fields. In doing so, we explored academic value systems as they influence publishing behavior and attitudes of a subset of University of California, Berkeley faculty. The larger report is available online.5 Our goal was to provide a preliminary descriptive analysis and understanding of the academic value systems associated with scholarly publication and communication, including means of communication extending beyond archival publication: • Within a discipline. (For example, what do scholars perceive as necessary to make a name for themselves?) It is recognized that there are different needs and value systems for different disciplinary areas, that different disciplines are in different stages of incorporating electronic communication, and that some disciplines, e.g., architecture, have products other than text. • Within a university. (For example, what are the value systems of the academic promotion and advancement processes, as perceived by different actors in those processes?) OVERVIEW OF METHODS The five disciplinary case studies were based on direct interviews with relevant stakeholders, almost all of whom were associated with the UC Berkeley campus. These case studies describe the state of scholarly communication in each of five fields: chemical engineering, anthropology, law and economics, English-language literature, and biostatistics. In the case of law and economics, it was the intersection of these two broad fields that was examined, not the sum.6 We also developed two smaller case studies representing the views of librarians and former Budget Committee members across these five disciplines. These “thickly described” case studies have the potential to enable a more precise identification of the factors associated with academic and disciplinary value systems. More specifically, such case studies should facilitate the identification of the factors that influence attractiveness, viability, and financial sustainability of different methods of scholarly communication for various participants in the publication/communication system, including authors (producers), researchers (consumers), libraries, and publishers. Formal interviews were conducted during the 2005-06 academic year with 49 individuals, 31 of whom were faculty (comprising regular faculty, former and current CSHE Research & Occasional Paper Series Harley et al., ACADEMIC VALUES AND SCHOLARLY PUBLICATION 3 faculty administrators, and recent ex-Budget Committee members). Twenty-two of the faculty interviewed were also editors of scholarly journals or had been so in the recent past. Five librarians were interviewed, as were two campus-level academic administrators. The remaining 11 interviewees were drawn from our steering committee. The basic interview protocol, initially designed for faculty, was modified as required over the course of the project to include questions of particular relevance to each class of stakeholders. As a one-year project, our sample was biased (UC Berkeley only) and relatively small (fewer than 10 informants per discipline), so extensions to other institutions and disciplines should be made with caution. As an external check to the UC Berkeley perspective, we included in each case background research on innovations taking place internationally in the targeted fields. Our research was considerably facilitated by the very structured review process for appointment, promotion, and advancement at the University of California (UC). That process involves formal review at regular intervals, both before and after tenure is awarded. Reviews are initiated in comprehensive written form by the department chair, using material drawn together and submitted by the faculty member. External letters of evaluation are solicited and included for appointments, promotions, and certain critical advancements within the rank of Professor. The package, or “case,” is then reviewed by the dean, by a specially appointed ad hoc committee for promotions, and by the Committee on Academic Personnel (denoted the Committee on Budget and Interdepartmental Relations, or “Budget Committee” for the UC Berkeley campus), who then make a recommendation. That recommendation is followed by the campus administration in nearly all cases.7 (The criteria for advancement are put forward in the University of California Academic Personnel Manual.8) FINDINGS Peer Review Conventional peer review is so central to scholars’ perception of quality that its retention is essentially a sine qua non for any method of archival publication, new or old, to be effective and valued. Peer review is the hallmark of quality that results from external and independent valuation. It also functions as an effective means of winnowing the papers that a researcher needs to examine in the course of his/her research. Peer review was cited as an essential factor when faculty were asked about: (1) their perceptions of both standard and newer forms of publication, (2) disadvantages of newer forms of publication, (3) where one should publish to make a name for oneself in the field (e.g., publish in top flight peer reviewed journals), and, of course, (4) peer review specifically. There is a large tendency for many members of the research community to equate electronic-only publication with lack of peer review, despite the fact that there are many examples to the contrary. Because of the very nature of peer review, this factor holds back even those who are fully aware of the advantages of fully peer-reviewed e-journals, because they know that the individuals reviewing their work for advancement may well not have that awareness. CSHE Research & Occasional Paper Series Harley et al., ACADEMIC VALUES AND SCHOLARLY PUBLICATION 4 It will be important to try to separate the issue of peer review for newer, electronic journals from those issues associated with the fact that most such journals are simply new and not yet well established. To some degree, however, peer review and the means of publication and dissemination can be separated. For example, there are authors whose work is peer reviewed and published in prestigious print journals, but who also retain rights to place the article on their own Web site. As noted by some interviewees, the result is that the work is accessed far more often on the Web site than in the published print journal. Thus, peer review is essential, although there is some worry among interviewees that the quality of peer review may be declining. The result is that it may be easier to rely on the tried-and-true outlets. The locus of peer review has, in some cases, moved out of the institution. Specifically, there is a growing tendency to rely on secondary measures associated with peer review, such as: perceived journal quality, selectivity, and/or stature; the fact that papers are invited; or keynote lectures for conferences. There is reliance, for instance, on university presses and reviewers of journals to evaluate scholarly work. (Even though reviewers for university presses are academic faculty, the editor exerts much more independent judgment than is typical for peer-reviewed journals published by scholarly societies.) In some cases, the Impact Factor9 may also serve as a gauge of quality. Despite the goals and quality of peer review, interviewees mentioned several times that the proliferation of journals has resulted in the possibility of getting almost anything published somewhere, if the author persists in trying to gain acceptance by different journals. The peer review process is more complicated for compound disciplines because many such fields are relatively nascent and therefore result in small, specialized communities of scholarship. Faculty in these interdisciplinary fields often prefer to publish within a single discipline because the most highly respected and recognizable outlets reside there; however, divergent expectations (ranging from quantity to methodology to writing style) and standards (especially with regard to quality) among fields often make it difficult for reviewers in standard fields to judge submissions from compound disciplines. Interdisciplinary publications may address this concern more readily as they become more prestigious. In fields that are joined with law, such as law and economics, the utilization and perception of peer review is particularly complicated, given the prominence of student-edited law reviews. Online Publishing Although online publication may be less of a concern to senior faculty with regard to advancement, they are often hindered in using it by their lack of ability or time. There is also no perceived reward for changing the status quo. Personal desire and interest, however, are often the drivers for participation in newer modes of communication and publication for senior faculty. Publishing in online-only resources is perceived among junior faculty as a possible threat to achieving tenure because online publication may not be counted as much, or even at all, in review. Even when written policy indicates that online publications should not be undervalued in consideration of advancement, actual practice may vary. CSHE Research & Occasional Paper Series Harley et al., ACADEMIC VALUES AND SCHOLARLY PUBLICATION 5 Some interviewees observed that new modes of communication and publication contribute to a proliferation of scholarly material. The result is that it is more difficult for time-pressed faculty to sift through all that is available in their fields. There is the perception that it is easier to get published in newer electronic journals and that they contain material of lesser and dubious quality. There is also a perception that the number of pages publishable in a journal is not restricted by cost for e-journals in the same way that it is for print journals, and thus editors of e-journals are not pressed to be as selective. Crisis/Cost Issues/Open Access Many faculty interviewees believed that UC Berkeley is insulated to a large degree from any crisis in scholarly publishing. The prestige of the institution and the quality of faculty work often enable faculty to publish with the most prestigious journals or presses. For the most part, faculty do not concern themselves with the burden of cost to the institution resulting from the scholarly publication process. These scholars had minimal, if any, understanding of open-access models, although they were somewhat familiar with the “open” concept. We found that scholars are generally receptive to the ideal of making knowledge available for the “public good.” Positive Perceptions Faculty did have a good understanding that the high cost of journals is problematic and faculty in chemical engineering, in particular, viewed open-access models as a possible alternative to commercial presses. Some faculty refuse to publish in particular journals because of their high cost and pricing mechanisms. Senior faculty appeared to be more comfortable with the idea of sharing material at the early stages of work (e.g., preprint servers), as did faculty in chemical engineering, biostatistics, and law and economics in general. Archaeologists already use some open-access Web sites to share field observations. Negative Perceptions The largest concern among scholars was the perception that open-access models had little or no means of quality control, such as peer review. Some faculty in biostatistics, interestingly, equated the high cost of print journals with quality and believed that online open-access models are “cheaper” and therefore might be prone to lower standards. Others expressed fear that scholarly work placed in open-access media could be “stolen,” although faculty with a better understanding of the online publication process saw licensing bodies, such as Creative Commons, as a potential solution. There was also some concern about the ownership of open-access and author-pays journals. Should universities act as repositories and implement some sort of selection process, there could be legal liabilities regarding the acceptance and rejection of work submitted by the institution’s own faculty, who the institution then judges for advancement. Faculty also expressed concern regarding how such repositories would be managed, including how subjects would be organized. CSHE Research & Occasional Paper Series Harley et al., ACADEMIC VALUES AND SCHOLARLY PUBLICATION 6 Author-pays Publishing Models Scholars were generally not aware of author-pays models. Once explained, faculty responses were universally negative. Paying to publish one’s work was perceived as self-promotion and fundamentally in conflict with the peer review process. English- language literature faculty, in particular, equated the author-pays models to vanity presses, while those in the sciences equated it with advertising and therefore believed that any such publication would compromise academic integrity. Many faculty realized that publication costs are an issue and believed that the author-pays model could possibly serve to discriminate against countries, institutions, and faculty with fewer financial resources. In particular, scholars from all fields expressed concern that such a model might exacerbate differences between the sciences and the humanities since funds to cover any charges would likely come from grants. Faculty who were in fields that lacked a sense of urgency in scholarship, especially English-language literature (and one interviewee in biostatistics), viewed the author-pays model as particularly irrelevant. It should be noted that page charges or submission fees are a reality for some disciplines such as the biological sciences and economics respectively. Page charges have now largely disappeared, however, for some scientific disciplines such as chemical engineering. We also note that page charges could have a particularly chilling effect on those who rely on expensive graphics in publications, especially in the humanities. Enhanced Capabilities of Electronic Communication Many faculty interviewed were happy to consume scholarly material afforded by new modes of communication and publication. Day-to-day scholarly practice uses them enormously, but for the last stage of scholarly practice, archival dissemination of scholarly work, scholars rely on traditional publishing formats with few exceptions. There are clear advantages to newer forms of publication that are recognized by a wider circle of scholars than those who have actually used them for publishing their own work. These include the ability to reach a larger audience, ease of access by readers, more rapid publication even when peer reviewed, the ability to search within and across texts, and the opportunity to make use of hyperlinks. Administrators and faculty both cited the fact that new technologies enable innovation in scholarly work. Anthropologists and chemical engineers agreed that moving images and three-dimensional (3D) models are particularly positive attributes. English-language literature faculty noted that technologies enable new ways of conducting scholarly work, most notably manuscript comparison in which single interpretations are no longer necessary because access to multiple interpretations is possible. Faculty, especially chemical engineers, believed that newer technologies have a democratizing effect on scholars outside of North America. The ability to have enough information (e.g., software code, back-end data, etc.) to enable the reproduction of statistical analyses was of particular importance to faculty in biostatistics. Data Storage/Management Needs Data storage and data management needs vary depending on the discipline and even subspecialty. Data produced by scholarly work vary both across and within disciplines, and vary from interpretive text, to visual or motion images, to 3D renderings or computer simulations, to observations whether in numeric or text form. Scholars in some fields also rely upon existing datasets rather than new data. In the sciences, grant monies CSHE Research & Occasional Paper Series Harley et al., ACADEMIC VALUES AND SCHOLARLY PUBLICATION 7 often fund data management and storage. It was noted that funders rarely dictate how data should be preserved. There is little to no institutional support for data management and preservation according to our interviewees. As a result, individual scholars are responsible for maintaining data integrity. Overall, faculty were concerned about the rapid evolution of technologies, which often results in archaic storage devices and thereby loss of work. In some fields with data-rich scholarship, such as biostatistics, there was the concern that not all data can be stored. Some suggested that their department or the university should have policies in place to address this problem. The Budget Committee Interviewees who had served on the Budget Committee with terms ending more than two years ago had not encountered the need to review non-conventional forms of publication and communication, and thus this was not a significant issue during their service. Because of academic specialization, the nine Budget Committee members, in most instances, do not have the disciplinary knowledge necessary to judge the research themselves in cases that they review. Thus there is a heavy reliance on peer review to aid the Budget Committee in its evaluation of scholarly work. As well, lack of peer review is associated—correctly or not—with newer forms of publication. Former Budget Committee members believed that the advancement process should be supportive of non-traditional publishing models, provided that peer review is strongly embedded in the process, and that it should be unprejudiced toward those scholars exploring new modes of publication. Despite faculty perceptions to the contrary, those with Budget Committee experience indicate that there is some degree of flexibility built into the review process. Former Budget Committee members and higher administrators who receive Budget Committee recommendations commented that the committee reflects standards in disciplinary fields and does not mandate appropriate methods, which effectively serves to maintain the status quo. Some explained that if the faculty member or department chair could make the case that a particular publication outlet was sufficiently peer reviewed for quality and well known within a particular subfield, then the Budget Committee would give it appropriate weight. Regardless, faculty are often unwilling to take risks by using newer publishing technologies that they presume may not be recognized by the Budget Committee as reputable and/or prestigious venues. Librarians Librarians appeared to have a much better understanding of available resources and the politics among publishers. They often were more technologically savvy than their faculty counterparts and were well aware of new technologies likely to affect available resources. Unlike many faculty, librarians who were interviewed strongly perceive a crisis in scholarly communication and see the rise in new forms of communication and publication as a positive step—albeit slow and evolutionary. Librarians indicated that they try to educate faculty about the scholarly communication crisis and how faculty might play a larger role. Although new modes of communication are not widely used by CSHE Research & Occasional Paper Series Harley et al., ACADEMIC VALUES AND SCHOLARLY PUBLICATION 8 faculty for presenting their work, librarians believe that open-access and/or author-pays models are viable alternatives to the problem of unsustainable journal costs. Online resources were also viewed largely as advantageous from a consumer perspective for many of the same reasons that faculty provided, e.g., ease of access, speedy dissemination, and so on. Librarians also believed that online technologies enable them to connect faculty and students with better information. Librarians’ main concerns about new modes of publication were along fiscal and technological dimensions, namely the economic sustainability of newer models and the role of the library in that financial equation. Librarians also pointed out the “version” problem for placing scholarly material in repositories. Most problematic for librarians, however, is the increasing reliance by both students, and to some degree, faculty, on search engines such as Google and Yahoo. Publishers and Editors Publishers with whom we made formal contact included the University of California Press, the Berkeley Electronic Press (bepress), the E-Scholarship project of the California Digital Library (CDL), the Public Library of Science (PLoS), Ithaka, the Electronic Publishing Initiative at Columbia (EPIC), and the Stanford Encyclopedia of Philosophy. Formal interviews of principals were conducted for the first four of these. We recognize that this is not a representative group of publishers. Twenty-two of the faculty members whom we interviewed were also editors of journals or had been in the recent past. Perceptions among publishers and editors were tied closely to the mode (print/electronic) of publication, their institutional affiliation and philosophy, and often the disciplinary fields in which they specialized; thus, their opinions often reflected those different viewpoints. Some felt that print publications were ineffective compared to electronic venues in disseminating work in a timely matter, although all recognized the challenges associated with newer forms of publication, regardless of format. Many of the publishers and editors we interviewed were aware of the concern about increasing costs. Publishers concurred that academia is in a transition period with regard to publishing, and they understood the complex interplay of tenure requirements and distribution of publishing choices among faculty. Although scholarship is inherently innovative in both approach and method, and in that way a natural match for newer forms of publication, change is often hindered by institutional requirements and standard practice, such as the perceived necessity of traditional publication for advancement and achieving tenure, and apprehension among scholars that reviewers will not accept newer forms of publication for advancement. Most agreed that use of newer forms of publication has not yet reached a sufficient saturation point to tip the scale and opined that the power to change rests with the university world—for both production and consumption. They recommended incentives for faculty, both in terms of policies (e.g., advancement process) and resources, as well as budgetary and technical support for libraries. Publishers shared with other interviewees the concern about perceptions that equate low cost with low quality for electronic forms of publication. Although all agreed that quality control systems are not in place in most open-access repositories, publishers in general pointed out that electronic journals can and often do use the same review process as traditional print journals. Several also expressed concern about the peer-review process, CSHE Research & Occasional Paper Series Harley et al., ACADEMIC VALUES AND SCHOLARLY PUBLICATION 9 and believed that too much emphasis is placed on outside opinion and prestige rather than a review of actual content quality. One issue for faculty editors is the difficulty in finding reviewers who are qualified, neutral, and objective scholars in a fairly closed academic community. This is compounded by the fact that the increasing quantity of publications requires more scholarly input for the review process, while already overburdened academics have limited time to participate. Editors, in particular, have a difficult time coordinating reviewers’ schedules and available time. REFLECTIONS Lessons Learned and Challenges While our investigations have yielded rich and descriptive case studies that shed light on the current state of scholarly communication, there are limitations to our study. Our small sample, both in the number of participants and in the range of disciplines, makes generalizations at this stage sketchy at best. Furthermore, we focused specifically on one campus in the University of California (UC) public system, UC Berkeley, and our results at this stage are thereby obviously biased. To develop general conclusions applicable to wider populations, future investigations will need to include other campuses and/or institutions. The highly structured advancement system of the University of California has been advantageous to us in many ways in conducting our research. First, we know who the actors are at the several stages of the review process and we were able to talk with them directly. Second, we were able to compare and contrast the views of those in the different steps of the review process—faculty, department chairs, deans, (former) Budget Committee members, and campus-level academic administrators. As well, we can ascertain the views of bodies such as the Budget Committee by those involved in other stages of the review process. Third, at each of the levels of review we had access to persons who have reviewed many advancement cases and who therefore can make informed comparative judgments. This fact gives reviewers the ability to identify the relative importance of different factors, including the medium and nature of the publication vehicle. Fourth, the nature of this review process affords the wherewithal of assessing the degree of importance and roles of peer review and the vehicles for peer review that hold cachet. By talking with reviewers involved with the Budget Committee at different times in recent years, we have been able to make initial inferences of whether and how the values ascribed to new media by that body are changing. The academic values that we have identified in this project may be specific to the most prestigious of universities, where faculty researchers nearly always have their papers accepted for publication and can publish wherever they want. It is also true that the values exercised at these leading universities will likely be emulated throughout the academic community. CSHE Research & Occasional Paper Series Harley et al., ACADEMIC VALUES AND SCHOLARLY PUBLICATION 10 CONCLUSIONS The descriptive case-study approach began to elucidate the ways in which faculty do or do not perceive electronic means and other new capabilities as enhancing (1) the quality, effectiveness, and immediacy of communication of a scholar’s research output to peers and users, (2) the recognition of that research, and (3) the efficiency and effectiveness of progress of scholarship as a whole. The disciplinary case studies also enabled a more precise identification of the factors associated with academic and disciplinary value systems that influence viability and financial sustainability of different methods of scholarly communication for various participants in the publication/communication system, including authors (producers), researchers (consumers), libraries, and publishers. From an examination of the ways in which value systems in five disciplinary areas affect scholarly publication and communication practices, we have reached the following conclusions: • Peer review is the coin of the realm. It is the value system supporting assessment and the perceived quality of research. It is commonly viewed as the primary mechanism through which research quality is nurtured, and through which research is made both effective and efficient. There was also a strong perception that peer review provides an excellent quality filter for the proliferating mass of scholarly information available on the Web. • There is some concern that the locus of peer review has moved out of the institution. This has particular repercussions for academic advancement as increasing reliance is placed on the prestige of publication rather than a review of actual content and quality. This is especially a concern for those scholars in compound disciplines, where peer review can be complicated by differing standards and expectations among fields, and where quality assurance depends upon a small group of specialized academics. • There is presently a somewhat dichotomous situation in which electronic forms of print publications are used heavily, even nearly exclusively, by performers of research in many fields, but perceptions and realities of the reward system keep a strong adherence to conventional, high-stature print publications as the means of record for reporting research and having it evaluated institutionally. This was true of all of the disciplines we examined. In the science fields, although major journals are maintained in print form, electronic replicates are used increasingly for most access and research. • While both are critically important to one’s career, the means of publication and communication for gaining advancement within the institution can differ significantly from those for making one’s name within a discipline. The former depends almost exclusively upon final, fully peer-reviewed archival publication, whereas the latter is more fluid and oriented toward partial results, meetings and information exchanges with other researchers during the course of the research (“in-progress communication”), as well as final, archival publication. CSHE Research & Occasional Paper Series Harley et al., ACADEMIC VALUES AND SCHOLARLY PUBLICATION 11 • Such “in-progress” communication also fulfills needs such as (1) gaining the critical thoughts of others while one’s research is in progress, (2) “staking claim” to one’s activity and accomplishments in an area, and (3) sparking thoughts and new ideas as a product of the discussion.10 • In-progress communication does not substitute for the need for final, archival presentation and dissemination of research results. They serve different purposes and needs. Both are important. • There is much more experimentation with regard to means of in-progress communication, where single means of publication and communication are not fixed so deeply in values and tradition as they are for final, archival publication. • From an institutional standpoint, there are looming questions about how to support faculty in their scholarly practice. Our interviews suggested that (at UC Berkeley, at least) there are currently few, if any, mechanisms or structures that support storing, archiving, and sharing the significant research products of faculty, such as databases, collections of literature, etc., that are created en route to ultimate archival publication. Based on our preliminary research, this is true of other institutions as well, except in a few fields. • Campus-level academic administrators perceive an inevitable but slow evolution toward new forms of publication (particularly in fast-moving scientific disciplines), similar to the shift from print journals to conference proceedings that occurred in computer science in the 1970s. They see this evolution gaining momentum and credibility. Respected scholars, however, will begin using such venues in great numbers only once these venues, and the peer review associated with them, become better established. • According to our interviewees, the Budget Committee has so far rarely needed to address the issue of publication venue. This is because so few new forms of publication are represented in the cases that come through the committee. 11 Former Budget Committee members, however, believed that the committee would be open to new forms provided that they meet the same standards for peer review and quality as traditional forms. Until more cases demanding the evaluation of digital scholarship come before tenure and review committees we foresee that the situation will remain relatively stagnant. • Campus-level academic administrators perceived a distinction between peer review in the discipline and peer review for promotion. While clearly interconnected, administrators maintained that discipline-based peer review cannot stand on its own; the input of immediate colleagues in addition to discipline-based peer review is necessary for promotional consideration. On the other hand, administrators believed that it was possible, in some cases, for local peer review to substitute for discipline- based peer review, for instance in considering the quality of work published in a non- peer-reviewed journal. Results from the project indicate that the values surrounding final archival publication are deep and relatively inflexible in many, if not most, disciplines at research universities. Yet, what scholars value and want will eventually become accepted practice. This is a CSHE Research & Occasional Paper Series Harley et al., ACADEMIC VALUES AND SCHOLARLY PUBLICATION 12 much more realistic way of looking at issues than is devising models and modes of communication because of their cost efficiencies or other non-research criteria and then trying to draw scholars to them. Approaches that attempt to “move” faculty and deeply embedded value systems directly toward new forms of archival, “final” publication are destined largely to failure in the short-term. Thus, it is our opinion that the development of any new models should focus on the needs of scholarly researchers for both final and in-progress communications in order to determine how those needs are likely to influence future scenarios in a range of disciplinary areas. In summary, we suggest that more innovation does and will occur first in in-progress communication than in final archival publication. One can foresee a scenario where useful and effective innovations in in-progress communication will eventually serve as drivers for improvements in final archival publication. It is therefore worthwhile to gain deeper insights into the needs, motives, and new capabilities within in-progress communication as well as for final, archival publication. ACKNOWLEDGEMENTS We would like to thank the Andrew W. Mellon Foundation for generously funding this research. We are also indebted to an unusually active and involved steering committee that has provided invaluable guidance, support, and time. We thank the more than fifty formal and informal interviewees who graciously scheduled time to provide candid opinions and ideas. Irene Perciali participated in preparing background information on publication practices in the five disciplines considered. NOTES 1 Donald W. King, Peter B. Boyce, Carol Hansen Montgomery, and Carol Tenopir. 2003. “Library Economic Metrics: Examples of the Comparison of Electronic and Print Journal Collections and Collection Services.” Library Trends 51(3): 276-300. Also see, Roger C. Schonfeld, Donald W. King, Ann Okerson, and Eileen Gifford Fenton. The Nonsubscription Side of Periodicals: Changes in Library Operations and Costs between Print and Electronic Formats, 2004, http://www.clir.org/pubs/reports/pub127/contents.html (06 December 2006). Also see, Donald J. Waters, “Managing Digital Assets in Higher Education: An Overview of Strategic Issues.” ARL Bimonthly Report 244, February 2006. http://www.arl.org/newsltr/244/assets.html (06 December 2006). 2 Judith Ryan, Idelber Avelar, Jennifer Fleissner, David E. Lashmet, J. Hillis Miller, et al. The Future of Scholarly Publishing from the Ad Hoc Committee on the Future of Scholarly Publishing, 2002, http://www.mla.org/resources/documents/issues_scholarly_pub/repview_future_pub (06 December 2006). Also see, Leigh Estabrook, The Book as the Gold Standard for Tenure and Promotion in the Humanistic Disciplines, 2003, http://lrc.lis.uiuc.edu/reports/CICBook.html (06 December 2006). 3 Theodore C. Bergstrom, “Free Labor for Costly Journals?” Journal of Economic Perspectives 15, no. 3 (2001), http://repositories.cdlib.org/postprints/20 (06 December 2006),183-198. Also see, Aaron S. Edlin and Daniel L. Rubinfeld. 2004. “Exclusion or CSHE Research & Occasional Paper Series http://www.clir.org/pubs/reports/pub127/contents.html http://www.arl.org/newsltr/244/assets.html http://www.mla.org/resources/documents/issues_scholarly_pub/repview_future_pub http://lrc.lis.uiuc.edu/reports/CICBook.html http://repositories.cdlib.org/postprints/20 Harley et al., ACADEMIC VALUES AND SCHOLARLY PUBLICATION 13 Efficient Pricing? The ‘Big Deal’ Bundling of Academic Journals.” Antitrust Law Journal 72, no. 1 (2004), http://works.bepress.com/aaron_edlin/37 (06 December 2006), 119- 157. Also see, Roger Noll and W. Edward Steinmueller. “An Economic Analysis of Scientific Journal Prices: Preliminary Results.” Serials Review 18 (1992):32-37. 4 Deborah Lines Andersen, ed. 2003. Digital Scholarship in the Tenure, Promotion, and Review Process. Armonk, NY: M.E. Sharpe. Also see Nature’s peer review debate at http://www.nature.com/nature/peerreview/debate/index.html (06 December 2006). 5 C. Judson King, Diane Harley, Sarah Earl-Novell, Jennifer Arter, and Shannon Lawrence. Scholarly Communication: Academic Values and Sustainable Models. 27 July 2006, http://cshe.berkeley.edu/publications/publications.php?id=23 (06 December 2006). 6 These particular five fields were selected with the goal of obtaining a diverse array of disciplines and publishing traditions, and taking advantage of the fact that at least one member of our project steering committee had deep knowledge of each of the disciplines selected. 7 Academic Senate Berkeley Division. “Introduction to the Budget Committee.” February 2006, http://academic-senate.berkeley.edu/pdf/Intro_to_BC.pdf (06 December 2006). 8 University of California Office of the President, “Appointment and Promotion, Review and Appraisal Committees,” Academic Personnel Manual, 26 July 2002, section 210, http://www.ucop.edu/acadadv/acadpers/apm/apm-210.pdf (06 December 2006). 9 The so-called Impact Factor is a measure of the citation frequency of papers in journals and is thereby equated by some to the stature and presumably the prestige of the journal. See, e.g., Richard Monastersky, “The Number That’s Devouring Science.” Chronicle of Higher Education. 14 October 2005 http://chronicle.com/weekly/v52/i08/08a01201.htm (06 December 2006). Ibid., “Impact Factors Run into Competition.” http://chronicle.com/weekly/v52/i08/08a01701.htm (06 December 2006). 10 A classic example of in-progress communication from the pre-electronic era is the Gordon Research Conferences, where current research is presented and discussed at length, but there are no written materials, nor are the presentations final or archival. The stimulating environment of these and similar conferences is particularly valuable for generating one’s own research ideas. Another traditional example is faculty invitations to other institutions for visits built around a seminar. 11 The recently released MLA Taskforce report on evaluating scholarship notes that of the departments it surveyed, 40.8 percent at doctoral institutions, 29.3 at master’s institutions, and 39.5 percent at baccalaureate institutions report having “no experience” evaluating digital scholarship. “MLA Task Force on Evaluating Scholarship for Tenure and Promotion.” December 2006. http://www.mla.org/tenure_promotion (10 December 2006). CSHE Research & Occasional Paper Series http://works.bepress.com/aaron_edlin/37 http://www.nature.com/nature/peerreview/debate/index.html http://cshe.berkeley.edu/publications/publications.php?id=23 http://academic-senate.berkeley.edu/pdf/Intro_to_BC.pdf http://www.ucop.edu/acadadv/acadpers/apm/apm-210.pdf http://chronicle.com/weekly/v52/i08/08a01201.htm http://chronicle.com/weekly/v52/i08/08a01701.htm Research & Occasional Paper Series: CSHE.13.06 << /ASCII85EncodePages false /AllowTransparency false /AutoPositionEPSFiles true /AutoRotatePages /All /Binding /Left /CalGrayProfile (Dot Gain 20%) /CalRGBProfile (sRGB IEC61966-2.1) /CalCMYKProfile (U.S. Web Coated \050SWOP\051 v2) /sRGBProfile (sRGB IEC61966-2.1) /CannotEmbedFontPolicy /Warning /CompatibilityLevel 1.4 /CompressObjects /Tags /CompressPages true /ConvertImagesToIndexed true /PassThroughJPEGImages true /CreateJDFFile false /CreateJobTicket false /DefaultRenderingIntent /Default /DetectBlends true /DetectCurves 0.0000 /ColorConversionStrategy /LeaveColorUnchanged /DoThumbnails false /EmbedAllFonts true /EmbedOpenType false /ParseICCProfilesInComments true /EmbedJobOptions true /DSCReportingLevel 0 /EmitDSCWarnings false /EndPage -1 /ImageMemory 1048576 /LockDistillerParams false /MaxSubsetPct 100 /Optimize true /OPM 1 /ParseDSCComments true /ParseDSCCommentsForDocInfo true /PreserveCopyPage true /PreserveDICMYKValues true /PreserveEPSInfo true /PreserveFlatness true /PreserveHalftoneInfo false /PreserveOPIComments false /PreserveOverprintSettings true /StartPage 1 /SubsetFonts true /TransferFunctionInfo /Apply /UCRandBGInfo /Preserve /UsePrologue false /ColorSettingsFile () /AlwaysEmbed [ true ] /NeverEmbed [ true ] /AntiAliasColorImages false /CropColorImages true /ColorImageMinResolution 300 /ColorImageMinResolutionPolicy /OK /DownsampleColorImages true /ColorImageDownsampleType /Bicubic /ColorImageResolution 300 /ColorImageDepth -1 /ColorImageMinDownsampleDepth 1 /ColorImageDownsampleThreshold 1.50000 /EncodeColorImages true /ColorImageFilter /DCTEncode /AutoFilterColorImages true /ColorImageAutoFilterStrategy /JPEG /ColorACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /ColorImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000ColorACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000ColorImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 300 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /GrayImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000GrayACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000GrayImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict << /K -1 >> /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile () /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False /Description << /CHS /CHT /DAN /DEU /ESP /FRA /ITA /JPN /KOR /NLD (Gebruik deze instellingen om Adobe PDF-documenten te maken voor kwaliteitsafdrukken op desktopprinters en proofers. De gemaakte PDF-documenten kunnen worden geopend met Acrobat en Adobe Reader 5.0 en hoger.) /NOR /PTB /SUO /SVE /ENU (Use these settings to create Adobe PDF documents for quality printing on desktop printers and proofers. Created PDF documents can be opened with Acrobat and Adobe Reader 5.0 and later.) >> /Namespace [ (Adobe) (Common) (1.0) ] /OtherNamespaces [ << /AsReaderSpreads false /CropImagesToFrames true /ErrorControl /WarnAndContinue /FlattenerIgnoreSpreadOverrides false /IncludeGuidesGrids false /IncludeNonPrinting false /IncludeSlug false /Namespace [ (Adobe) (InDesign) (4.0) ] /OmitPlacedBitmaps false /OmitPlacedEPS false /OmitPlacedPDF false /SimulateOverprint /Legacy >> << /AddBleedMarks false /AddColorBars false /AddCropMarks false /AddPageInfo false /AddRegMarks false /ConvertColors /NoConversion /DestinationProfileName () /DestinationProfileSelector /NA /Downsample16BitImages true /FlattenerPreset << /PresetSelector /MediumResolution >> /FormElements false /GenerateStructure true /IncludeBookmarks false /IncludeHyperlinks false /IncludeInteractive false /IncludeLayers false /IncludeProfiles true /MultimediaHandling /UseObjectSettings /Namespace [ (Adobe) (CreativeSuite) (2.0) ] /PDFXOutputIntentProfileSelector /NA /PreserveEditing true /UntaggedCMYKHandling /LeaveUntagged /UntaggedRGBHandling /LeaveUntagged /UseDocumentBleed false >> ] >> setdistillerparams << /HWResolution [2400 2400] /PageSize [612.000 792.000] >> setpagedevice