219 Collection Development in the Era of Big Deals Philippe Mongeon, Kyle Siler, Antoine Archambault, Cassidy R. Sugimoto, and Vincent Larivière* Drawing on an original methodology using citations, downloads, and survey data, this paper analyzes journal usage patterns across 28 Canadian universities. Results show that usage levels vary across disciplines and that different academic platforms varied in their importance to different institutions, with for-profit platforms generally exhibiting lower usage. These results suggest economic inefficiencies exist in “big deal” academic journal subscriptions for universities, as most journals in such bundles are seldom or never used. We recommend that universities coordinate resource shar- ing and negotiate strategies with academic journal expenditures based on shared interests and usage trends. Introduction The advent of digital journals in the second half of the 1990s led to several changes in journal ownership, publishing models, and subscription contracts. Prior to the digital age, most schol- arly journals were owned by scientific societies, university presses, and small publishers. The disruptive nature of the internet led to a consolidation of the academic publishing industry, as major players bought smaller publishers, merged (such as Reed-Elsevier and Springer-Nature), and established contracts with scientific societies to publish their journals.1 This concentration of scholarly journals, combined with new digital means for diffusing knowledge, made possible a new model of journal subscription: the “big deal.”2 Until the mid- 1990s, libraries subscribed to individual journals, managed primarily by collection development librarians. Such librarians assessed the needs of students and faculty and evaluated the use of their collections to identify core documents; that is, those that are most useful to their users and should be included in the collection.3 However, with the advent of electronic publishing, pub- lishers started to sell journal bundles to libraries for their “then-current total expenditure on the Philippe Mongeon is Postdoctoral Researcher at the School of Information Management of Dalhousie University, Halifax, Nova Scotia, Canada; email: philippe.mongeon@umontreal.ca. Kyle is Research Fellow at the Université de Montréal, Montréal, Québec, Canada; email: ksiler@gmail.com. Antoine Archambault is Librarian and Head of Technical Services at Service de la bibliothèque, Université du Québec à Trois-Rivières, Trois-Rivières, Québec, Canada; email: antoine.archambault@umontreal.ca. Cassidy R. Sugimoto is Professor at the School of Informatics and Computing, Indiana University Bloomington, Bloomington, Indiana, United States; email: sugimoto@indiana.edu. Vincent Larivière is Professor at École de bibliothéconomie et des sciences de l’information, Université de Montréal, Montréal, Québec, Canada, and Observatoire des sciences et des technologies (OST), Centre interuniversitaire de re- cherche sur la science et la technologie (CIRST), Université du Québec à Montréal, Montréal, Québec, Canada; email: vincent.lariviere@umontreal.ca. © 2021 Philippe Mongeon, Kyle Siler, Antoine Archambault, Cassidy R. Sugimoto, and Vincent Larivière, Attribution-NonCommercial (https://creativecommons.org/licenses/by-nc/4.0/) CC BY-NC. mailto:philippe.mongeon@umontreal.ca mailto:ksiler@gmail.com mailto:antoine.archambault@umontreal.ca mailto:sugimoto@indiana.edu mailto:vincent.lariviere@umontreal.ca https://creativecommons.org/licenses/by-nc/4.0/ 220 College & Research Libraries March 2021 publisher’s offerings, plus an additional 5–15%.”4 These big deals allowed libraries to significantly enlarge their collections at an initially low cost, without running into shelving space issues. Such journal bundles also removed, to a certain extent, the necessity to evaluate journal collections.5 While such deals were, at the time, rightfully considered advantageous to libraries, the situ- ation rapidly changed due to above-inflation cost increases imposed by publishers to big deal subscriptions.6 As reported by the Association of Research Libraries,7 the consolidation of the academic publishing industry ended up being financially disadvantageous for most academic libraries, with an increase of 400 percent in their budget devoted to serials during the 1986–2011 period.8 Moreover, as these bundles contain journals with varying levels of usage, libraries end up paying higher costs for journals that are used.9 On the whole, increases in subscription fees have left libraries with limited flexibility regarding the composition of their collections and reduced investments in other document types, such as books. Given those financial pressures and the emergence of open access publishing alternatives, academic libraries are beginning to question the viability of the big deal model. Many libraries unbundled their journal subscriptions or, more drastically, cancelled subscriptions to all journals from large commercial publishers.10 To make such collection development choices, universities and consortiums need to have data on usage, as well as expertise on how analyze it. While bibliometric methods were at the heart of collection development from its beginning—early bibliometric work generally aimed at informing collection development11—most bibliometric research during the last 30 years has focused on the measurement of scientific activities of researchers rather than on the usage of documents. This is particularly true of the Journal Impact Factor, which was created in the 1970s to help librarians measure journal use12 but became mostly used for research evaluation at the turn of this century.13 Using data retrieved for 28 Canadian universities during the Journal Usage Project of the Canadian Research Knowledge Network (CRKN), this article aims at revisiting the use of bib- liometric indicators and methodologies for collection development in an era of big deals, relying on institutional-level indicators rather than global indicators such as the Journal Impact Factor. It describes a framework and method for assessing the usage of big deals, based on download data, citation data, and survey mentions. The paper first provides a review of the literature on big deals and indicators for collection development, followed by a description of the methods used to retrieve the various data sources and justification for the use of the chosen indicators. It then analyzes the level of concentration of journal usage across institutions and publishers based on the three distinct indicators and measures the overlap of core journals between universities. It concludes with a discussion of the strengths and weaknesses of the method developed, as well as broader implications for science communication in the era of big deals and digital publishing. Literature Review The current state of the academic publishing market—high subscription prices and hetero- geneity in the usage of periodicals—has historical precedents. Libraries have been struggling to keep up with the increases in the number of journals published and in subscription costs for decades. In the 1970s, with the exponential growth of scientific information, many British libraries cancelled subscriptions, as they found that many periodicals were seldom cited.14 Journal cancellations also occurred in the 1990s, with subscription price increases as its leading cause.15 In the United States, these struggles have historically been amplified by the periodical devaluation of the US dollar.16 Collection Development in the Era of Big Deals 221 More contemporarily, CRKN is not the only library consortium that acted against the increasing budgetary pressures brought on by big deals. In fact, this is an issue shared by practically all academic libraries, and several initiatives have been attempted to solve it—with varying degree of success. For instance, the Virtual Library of Virginia (VIVA) cancelled one of their big deal packages, reduced others to only 10 percent of their previous content, and adopted an aggressive, data-based approach to their negotiation with publishers.17 In the United Kingdom, Research Libraries UK (RLUK) developed an alternative to the big deals, called “plan B,” which ultimately did not lead to the opting out of the journal bundles but still helped the consortium to negotiate more advantageous deals with publishers.18 These are just two examples among many others. The Scholarly Publishing and Academic Resources Coalition (SPARC) provides a list of more than 50 recent negotiations and cancellations—go- ing as far back as 2008—on their website.19 Such initiatives are exacerbated by the growth of open access dissemination, which, in its various forms, accounts for about half of the scholarly documents researchers have searched for published in recent years.20 One thing that these negotiations and cancellations have in common is that they require a clear picture of users’ needs and of the extent to which serial collections are being used. The literature thus also contains many accounts of local journal usage evaluations, as well as many proposals of methods that can be used for this endeavor, such as the Journal Impact Factor,21 cited references,22 user ratings and surveys, journal reshelvings, and interlibrary loan (ILL) requests.23 In the digital era, one of the most used indicators is the successful full-text article request (SFTAR, hereafter referred to as “downloads”) reported in the COUNTER Journal Reports.24 These indicators are often combined with journal subscription costs,25 total serials expenditures,26 or full-time employees.27 These indicators all have shortcomings and are better considered as complementary rather than as alternatives to one another. Despite being the object of some criticisms,28 downloads are probably the most relevant usage indicators available for librarians in the digital era.29 References, which have been used for decades, have the advantage of not only showing that a journal was used, but that it was useful.30 Cited references have been widely used (especially in the print era), but they can mostly measure usage (and usefulness) for research purposes and are not capturing other types of use such as teaching.31 This partly explains the moderate correlation, which varies by discipline,32 that previous studies have found between references and downloads.33 Reshelving counts have the obvious limitation that they can only be used to measure usage of print serials, but they tend to underestimate usage when multiple users use the same journal or reshelve it themselves,34 while ILL requests only provide data on us- age of serials to which the library does not subscribe. Despite the various methods and indicators used in serials evaluations, concentration of usage of serials in academic libraries converge toward a Pareto (80/20) distribution.35 Some studies also reported much more concentrated usage with less than 10 percent of journals ac- counting for 80 percent of usage counts,36 and others a less concentrated usage with 30 to 40 percent of journals accounting for 80 percent of downloads.37 Past studies also show that, due to disparities in journal use across different institutions within a consortium, a one-size-fits- all journal list is rarely optimal. The literature also highlights the importance of considering disciplinary differences in journal use to avoid making unfair comparisons between journals.38 For instance, an investigation of the relationship between full-time equivalent (FTE) and journal downloads in the Consortium of Academic Libraries of Catalonia (CBUC) also found 222 College & Research Libraries March 2021 an uneven distribution of journal use between universities and between subject areas.39 Most journal evaluations reported in the literature do take into account the discipline of the journal (or users)40 or focus on a single discipline.41 In summary, our review of the literature suggests that, since all indicators have their own limitations, using a combination of indicators and methods is likely to provide best results. Furthermore, as past studies have highlighted, it is best to adopt an evaluative approach that takes into account the disciplinary differences in knowledge production, dissemination, and use, as well as the particularities of local contexts. As such, the present analysis uses a combi- nation of indicators (downloads, references, and survey mentions), and results are reported at the institutional and disciplinary levels. We also find many small-scale studies of single institutions, mainly in the United States. The present paper thus fills a gap by providing a large-scale, multi-institution analysis of the little explored Canadian context. Methods Data Journal usage comprises multiple dimensions. Different scholars and their institutions may harbor different values and priorities regarding journals from both academic and economic standpoints. For example, a journal might be very important for teaching but much less for research. In contrast, a journal might be of very high importance for researchers but much less for teaching. As universities serve multiple communities (that is to say, faculty, students, and staff) who exhibit different usage patterns,42 indicators of usage should reflect this diversity. Therefore, our analysis draws on three distinct data sources: 1) university-specific download data; 2) citation data (in other words, references made by authors from each university); and 3) faculty and graduate students survey data. Downloads Number of downloads is defined as the number of times documents published in a given jour- nal are downloaded by users from each institution. This indicator is the most comprehensive: it provides the global use of the journal by the university community, irrespective of the user status (whether undergraduate student, faculty or staff member, or other user). Download reports (JR1) were provided for each platform, either by libraries directly, or by CKRN, who retrieved the reports from vendors on their behalf. Downloads cover the 2011–2015 period, include both HTML and PDF formats, and were summed across all platforms for journals that were available through multiple vendors. One limitation of this data source is that it does not cover downloads made to other versions of papers published (such as preprints, postprints, and other green open access version) in each journal or downloads of open access papers made from outside the university domain. References Number of references is defined as the number of times documents published in each journal are referenced by researchers affiliated with each participating institution. This indicator em- phasizes the use of journals by researchers, representing an important interest of faculty and graduate students. This report uses reference data retrieved in Clarivate Analytics’ Web of Science (WOS). We collected all articles published during the 2011–2015 period authored by at least one researcher affiliated with a participating institution (as per the authors’ addresses). Collection Development in the Era of Big Deals 223 It is important to note that, for disciplines associated with Arts and Humanities, references are not as representative of usage as in other disciplines, as researchers in those disciplines are more likely to cite older literature or to cite books.43 Globally, 15,686 distinct journals were referenced in the WOS by researchers from researchers from the 28 universities. One limita- tion of those reference indicators is coverage: data can only be retrieved for journals that are indexed in the WOS as source items. Survey A survey was conducted at 23 of the 28 participating universities.44 For administrative reasons, two institutions conducted their surveys between November 16 and December 19 (Queen’s University and University of Regina), while the surveys for the 21 other institutions were distributed between January 17 and February 27, 2017. Invitation emails were sent directly by institutions to their faculty, graduate students, and postdoctoral fellows—although a few institutions (Brock University, Carleton University, Vancouver Island University) invited only faculty to participate. The survey, performed in English (except at the University of Ottawa, where it was available both in English and in French), included five questions: 1. Please list up to 10 scholarly journals that you consider essential for your research. 2. Please list up to 10 scholarly journals that you consider essential for your teaching. 3. What university do you work for or study at? 4. What is your main status at this institution? (PhD student, postdoctoral fellow, fac- ulty member) 5. What is your main area of study or research? (Natural Sciences and Engineering, Social Sciences and Humanities, or Health Sciences) For questions 1 and 2, journals could be selected from a predefined list of 125,517 jour- nals collectively validated by participating institutions (see section 2.2). Participants could also provide more titles in an open text field when necessary; those were also included in the analysis. For each institution, we summed the number of mentions made to each periodical by respondents. This provides an indication of the perceived importance of the journal by graduate students, postdoctoral fellows, and faculty members of the institution. As men- tioned earlier, while some journals may be considered important by faculty members, others may be more important for students. It was thus decided that all types of users (students, faculty, and others) comprising the university body were equally important and that the journals they mention in the survey should be considered equally important. Therefore, no weighting by type of user was applied. On the whole, 5,579 researchers responded to the survey, of which 5,453 were from the 23 universities participating in this part of the project. The number of participants varied greatly from one institution to another, thus making the results less robust in some cases. For instance, while the analysis includes 529 respondents from Western University, only 29 researchers from Regina answered the survey (see table 1). Breakdown by discipline showed that 45.1 percent of respondents were from the social sciences and humanities, 33.8 percent were from natural sciences and engineering, and 19.8 percent were from health sciences. While these differences can be explained by the differ- ent size of these universities, the fact that the survey was administered by the universities themselves has led to unevenness in how the survey was publicized to each university com- munity and how many reminders were sent. In total, 11,849 distinct journals were mentioned by survey respondents. 224 College & Research Libraries March 2021 TABLE 1 Number of Respondents from Each University, by Status Faculty Member PhD student Fellow Post-doctoral Not Defined Total Institution Band N % N % N % N % Alberta 12 75 23.1% 189 58.2% 50 15.4% 11 325 British Columbia 12 0.0% 0.0% 2 100.0% 2 Brock 8 158 98.1% 0.0% 1 0.6% 2 161 Carleton 10 82 96.5% 0.0% 3 3.5% 85 Concordia 10 1 100.0% 0.0% 0.0% 1 Dalhousie 10 237 68.5% 80 23.1% 18 5.2% 11 346 Lakehead 7 72 72.7% 23 23.2% 4 4.0% 99 MacEwan 6 81 97.6% 1 1.2% 1 1.2% 83 McGill 12 154 35.9% 209 48.7% 65 15.2% 1 429 McMaster University* 11 1 33.3% 1 33.3% 1 33.3% 3 Mount Royal University 7 58 98.3% 1 1.7% 0.0% 59 Queen's University 10 266 61.1% 157 36.1% 12 2.8% 435 Simon Fraser University 10 116 53.7% 81 37.5% 18 8.3% 1 216 Universite de Montreal** 12 1 100.0% 0.0% 0.0% 1 Calgary 11 351 79.8% 61 13.9% 24 5.5% 4 440 Guelph 10 176 56.6% 92 29.6% 42 13.5% 1 311 Lethbridge 8 43 76.8% 9 16.1% 4 7.1% 56 Manitoba 10 1 100.0% 0.0% 0.0% 1 University of New Brunswick 9 101 79.5% 21 16.5% 4 3.1% 1 127 University of Ottawa 11 223 45.1% 241 48.8% 26 5.3% 4 494 University of Regina 8 26 89.7% 3 10.3% 0.0% 29 University of Saskatchewan 10 110 61.5% 44 24.6% 21 11.7% 4 179 University of Victoria 10 165 75.3% 43 19.6% 9 4.1% 2 219 University of Waterloo 11 93 53.8% 40 23.1% 37 21.4% 3 173 University of Windsor 9 140 68.0% 55 26.7% 10 4.9% 1 206 Vancouver Island University 6 43 100.0% 0.0% 0.0% 43 Western University 11 202 38.2% 266 50.3% 59 11.2% 2 529 York University 11 240 58.7% 141 34.5% 22 5.4% 6 409 Not defined 35 29.7% 36 30.5% 5 4.2% 42 118 All institutions 3,251 58.3% 1,794 32.2% 438 7.9% 96 5,579 * Did not participate in the study, but was mentioned by three respondents as their affiliation. ** Did not participate in the survey part of the analysis. Collection Development in the Era of Big Deals 225 Creating a Master Journal List Merging of Datasets The three datasets (that is, downloads, references, and mentions) were merged to create a master journal list. Since journal titles may not only vary between datasets and platforms, but also over time, ISSN and/or EISSN codes were both used to create a first set of unique journals. The remaining titles (those with no ISSN or EISSN) were merged with unique journals on an exact title match basis. Remaining unique titles were considered unique journals. At this step, the list contained 185,336 distinct journal titles. Validation and Correction of Errors Since the cleanliness and quality of the data significantly varied across datasets, the first merging process resulted in many inaccuracies (such as distinct journals merged or unique journals split into multiple entries). For instance, there were several issues with ISSNs and EISSNs provided in both download and reference files—different journals having the same number, as well as massive duplication of journals. A thorough manual verification of the lists was performed to correct as many of these errors as possible. This was done by looking at the list of titles associated with each unique journal and investigating potential errors us- ing UlrichsWeb’s Global Serials Directory45 or by searching online. Journals with similar titles were also investigated to ensure their distinctiveness. Moreover, because selection of journals in WOS is based on a certain level of prominence, we thoroughly investigated this subset of journals to make sure they were properly matched with the download dataset. This step was performed only for journals with at least one article published during the 2010–2015 period. The same investigation was done with the 413 journals mentioned in the survey that had no match in the download or reference datasets. As a result of those validation steps, several hundred false positives and negatives were corrected. This resulted in 125,517 distinct titles. Removal of Nonjournal and Nonacademic Titles The lists of journals included in the different data sources (mostly the institutional down- load data) are not academic journals but other types of documents such as monographs, conference proceedings,46 magazines, newspapers, and websites. We manually verified each of the 125,517 distinct titles using UlrichsWeb to filter out those nonacademic or non- journal titles from the list, resulting in a final list of 47,012 distinct academic journal titles. Assignation of a Discipline As decades of citation analysis have shown, journal usage varies across disciplines (Larivière and Sugimoto, 2019). Thus, each journal’s usage is compared with other journals within the same discipline. We used the National Science Foundation (NSF) journal classification, which contains 14 disciplines divided in 144 specialties based on cognitive and referencing similari- ties (Hamilton, 2003), to classify journals into four broad disciplinary groups: • Arts and Humanities (AH): Arts, Humanities • Social Sciences (SS): Social Sciences, Psychology, Professional Fields, Health • Biomedical (BM): Biomedical Research, Clinical Medicine • Natural Sciences and Engineering (NSE): Engineering and Technology, Chemistry, Biol- ogy, Physics, Mathematics, Earth and Space 226 College & Research Libraries March 2021 WOS-indexed journals were assigned a discipline using the aforementioned NSF classifica- tion through the OST database. To assign a discipline to journals not indexed in the WOS, we used the NSF classification in combination with subject information found on UlrichsWeb’s Global Serials Directory.47 In cases where multiple disciplinary groups fit a single journal, a primary discipline was chosen based on the available journal information. Figure 1 shows the number of distinct journals analyzed for each university and for all 28 universities combined. In general, larger universities—which tend to have medical schools— provided data on a larger set of journals, while smaller universities provided information on a smaller number of journals. Interestingly, the proportion or journals in each broad domain remains roughly the same of all, except for Waterloo, which has a smaller proportion of AH FIGURE 1 Number of Distinct Titles Analyzed, by University and Discipline Identifying Core Journals Collection Development in the Era of Big Deals 227 journals. At the level of the entire dataset (N = 47,012 journals), 15 percent of journals are in AH, 35 percent are in SS, 22 percent are in BM, and 28 percent are in NSE—which is a differ- ent composition than that of WOS, in which SS and AH account for a much lower proportion. FIGURE 3 Thresholds for Number of Downloads, References, and Mentions, by University and Domain FIGURE 2 Example of the Cumulative Distribution of Downloads, References, and Mentions 228 College & Research Libraries March 2021 As shown by Bradford,48 journal use is heavily skewed, with a minority of journals ac- counting for the majority of the usage. Likewise, a large majority of journals account for only a small portion of usage. Three indicators are used to determine the priority status of journals: • Mean number of downloads/year (excluding years without downloads) • Mean number of citations/year • Total number of survey mentions We compiled the cumulative distribution of usage for each institution, indicator, and discipline. This allows us to assess what proportion of journals (that is, 10%, 20%, 30%) that account for a certain proportion of usage. In our analysis, we have chosen 80 percent to be the target in terms of usage for each indicator. Hence, the set of core journals contains all academic periodicals that collectively account for 80 percent of downloads, 80 percent of mentions, or 80 percent of references (see figure 2). Thus, each journal has three chances of being considered a core journal for a given institution: one for each of the type of usage reflected by each indicator. The different sizes of universities and of their levels of participation in the survey influence the absolute values of thresholds for determining core journals (see figure 3). For instance, larger universities have higher thresholds in terms of both number of downloads and number of references, and those are generally higher for BM, followed by NSE, SS, and AH. In terms of number of mentions, given the relatively low number of respondents to the survey when broken down by institution, receiving one mention is generally enough to be considered a core journal for most domains and institutions. Using those empirically determined thresholds, we can assess the proportion of core journals across institutions, disciplines, and publishers. Results General Usage Patterns Figure 4 presents, for each university and domain, the proportion of journals downloaded at least once, cited at least once, or mentioned at least once in the survey. A relatively larger proportion of SS and AH journals have been downloaded at least once than of BM and NSE, suggesting that a more diverse set of journals is used in the former group of disciplines than in the latter. On the other hand—likely because of a lower coverage of their literature in the WOS—a lower proportion of AH and SS journals are cited at least once compared to BM and NSE journals. Unsurprisingly, the proportion of journals mentioned is much lower—generally less than 10 percent for all disciplines and universities, with the exception of a few cases in AH. Figure 5 presents for each institution and domain, the number and proportion of journals that fall into the top 80 percent cumulative proportion of downloads, references, and mentions—the criteria for determining core journals—by university and domain. It shows that, for all universi- ties, a larger number of SS journals are considered to be core, compared to the other domains (left panel). When compiled as a percentage of all journals in that domain and university (right panel), we observe that a larger proportion of journals in SS and AH than BM and NSE are considered to be core journals. More specifically, percentages of core journals by institution are between 13.4 and 43.2 percent in AH, 12.5 and 28.1 percent in SS, 22.8 and 9.7 percent in BM, and 7.6 and 17.8 percent in NSE. This is in part due to the larger number of journals in those disciplines, but it is also influenced by the greater diversity of paradigms and methodologies in those disciplines.49 It also shows that, surprisingly, smaller universities have a higher proportion of core journals, which suggests a more even usage of resources than larger universities, in which a larger body of researchers might work on similar topics, thus concentrating journal usage. Collection Development in the Era of Big Deals 229 FIGURE 4 Proportion of Journals That Have Been Downloaded at Least Once, Received at Least One Citation, or Have Been Mentioned in the Survey, by Institution and Domain FIGURE 5 Number and Percentage of Core Journals, by Domain and Institution 230 College & Research Libraries March 2021 To better understand the overlap between universities’ usage patterns, we compiled, for each journal of the entire collection of all 28 universities combined, the number of universities that consider the title as core (see figure 6). It shows that fewer than 500 titles (484, about 1 percent of the set of journals analyzed here) are considered to be core in all of the 28 universi- ties. This includes large multidisciplinary journals such as Science, Nature, and Proceedings of the National Academy of Sciences, but also more specialized journals such as The Classical Quarterly and national journals such as the Canadian Review of Sociology. A striking feature of figure 6 is the proportion of titles (66%) that are not considered core journals in any of the 28 universities. Use by Platform Figure 7 presents, for the main platforms—that is to say, those that include at least 25 jour- nals for which data was provided by at least 15 universities—the proportion of journals that are considered to be core. Notably, several journals are available on more than one platform. Thus, given that two of our indicators for determining core journals are not platform-specific— journals are referenced and mentioned in the survey without having a clear link to a specific platform—we defined each journal’s main platform as the platform through which it obtained the highest number of downloads.50 Results show that disciplinary platforms from NSE (APS and ACS, among others) have a very high proportion of core journals and that this propor- tion is high for almost all universities. Results also show that, on average, not-for-profits (for example, JSTOR and Canadian Science Publishing), scientific societies, and university-based platforms have a higher proportion of core journals than for-profit ones. It is also worth men- FIGURE 6 Distribution of All Journals (N = 47,012) by Number of Libraries Considering Them as Core Journals Collection Development in the Era of Big Deals 231 tioning strong language effects: universities with French language components and those in Québec (Concordia, McGill, Montréal, Ottawa, and York) have a larger proportion of core journals on mostly-French-language platforms (such as Érudit and CAIRN). Figure 8 presents, for the five major for-profit platforms, the proportion of journals as a function of the number of libraries that consider them to be core. It shows that, for each pub- lisher, a tiny proportion of journals are considered core in all universities. Only 0.4 percent of Wiley journals and 0.2 percent of Elsevier’s are considered core in all 27 libraries, and none of the journals from Sage Publications, Springer, or Taylor & Francis. When considering journals that are core in the majority of libraries (that is, 14 out of 27), these percentages increase to 19.9 percent for Elsevier, 15.7 percent for Wiley, and 12.5 percent for Sage Publications, but they remain particularly low for Springer (3.8%) and Taylor & Francis (2.3%). At the other end of the spectrum, we observe that 63.6 percent of Springer journals, 58.9 percent of Taylor & Francis journals, and 45.8 percent of Elsevier’s journals are not considered to be core in any of the 28 universities. Wiley and Sage Publications fare slightly better, with 35.9 and 32.5 percent of journals that are never core, respectively. FIGURE 7 Proportion of Core Journals, by Institution and Platform (Red denotes high proportion of core journals, grey denotes mild proportion, and blue denotes low proportion) 232 College & Research Libraries March 2021 Discussion and Conclusion Analysis of universities’ usage data provides new insights into the value of journals across multiple domains. Journal usage and core importance were measured liberally, using indicators that emphasize scholarly (in other words, citations), pedagogical (that is to say, downloads), and reputational (that is, survey mentions) dimensions of value. This triangulation of biblio- metric and social science research methods is necessary to ensure that the determination of a core journal upholds both the academic and educational missions of universities. Our results show that, even though Canadian university libraries have access to a great number of scholarly journals through big deals, only a small fraction of those journals—the core collection—are being regularly used. This high usage concentration is, more often than not, much smaller than the expected 20 percent according to Bradford.51 References, downloads, and mentions are complementary measures of journal usage and are helpful in fully capturing the library needs of varying university communities. Furthermore, this multimethod approach mitigates concerns of the disciplinary bias of purely citation-based methods. The importance of these triangulations is emphasized in the results. For example, there are many more jour- nals in SS and AH disciplines that are downloaded rather than referenced. This is, of course, a FIGURE 8 Distribution of Journals from the Five Major For-Profit Platforms, by Number of Universities Considering Them to Be Core Journals Collection Development in the Era of Big Deals 233 function of using the WOS as a key identifier of references, given the known disciplinary and language bias of the database.52 However, as this is the key citation database, it suggests that making decisions purely on citation rates would undervalue important resources in SS and AH. Disciplinary differences were apparent throughout our analyses. Resources in social sci- ences and humanities are distributed more sparsely than those in natural sciences and medi- cal sciences. More specifically, for every institution, there were a larger share of SS and AH journals that were needed to represent all core journals than in any of the other disciplinary domains. Other disciplines were much more concentrated in a fewer number and proportion of journals. This suggests that fewer journals are necessary to fulfill the needs of researchers and students in the natural and biological sciences. Consortia agreements are a useful way for individual libraries to optimize their journal subscription expenses. This article provides a depiction of the degree to which Canadian uni- versities overlap in their core journals. The analysis provides both an indication of nationally important resources, but also could be used to identify ways to share resources. The major- ity of core journals (60%) were shared across all institutions. Considering all 47,012 titles, 66 percent were not considered core journals in any library. Collectively, Canadian institutions should consider how many of these titles should be held at all and if collective arrangements would be more useful for these more peripheral sources. Our analyses also provide indications of the value of certain publishing platforms in vari- ous universities. These data can be used to negotiate with publishers and platforms to ensure that the university is maximizing subscriptions and is not being exploited by suboptimal big deal packages. Among platforms, APS and ACS—both large disciplinary associations—were seen as core publications for nearly all Canadian institutions. Canadian Science Publishing also ranked high among the institutions. The relative value of large for-profit platforms was more variable. Sage Publications and JSTOR have fairly strong value across institutions, whereas only about a third of the content from Wiley and ScienceDirect (Elsevier) were seen as core for the studied institutions. At the lowest end, less than half of the material provided by SpringerLink was listed as a core journal by any of the institutions. This suggests that, at least in the case of some big deals, a complete cancellation of the journal bundle would not have sizable effects on access to relevant resources, especially given the proportion of articles available in open access,53 and that this proportion will continue to increase in years to come.54 Our research has both academic and economic implications. Librarians should be informed about the value of certain collections to have greater agency in negotiations about subscrip- tion prices and to make efficient purchasing decisions. There is also a scientific advantage, as reallocating resources toward the disciplinary emphases of the university should provide a stronger collection for students and staff alike. This analysis brings further empirical evidence that the “big deals” serve commercial interests first and foremost, suggesting that having access to information as a core professional value have turned libraries into lucrative customers for commercial publishers whose business models are focused on selling access to information at the highest possible price. Given the usage concentration and collection overlap at the national level, our findings are in line with the idea advocated by Atkinson that university libraries have much to gain from greater cooperation with collection management and development.55 By joining forces, libraries could reduce financial burdens, increase negotiation power against commercial pub- lishers, and enhance their ability to meet the needs of their communities. 234 College & Research Libraries March 2021 Acknowledgments This research was funded by the Canadian Research Knowledge Network (CRKN) / Réseau canadien de documentation pour la recherche (RCDR), through the 28 institutions participat- ing in the analysis. Contributions to earlier phases of the project were also directly made by the Université de Montréal, Université du Québec à Montréal, Université de Sherbrooke, and Université Laval. Ethics certification was obtained at Université de Montréal (CERAS-2016- 17-130-D), as well as at other institutions participating in the survey (University of Alberta, Brock University, University of Calgary, Carleton University, Dalhousie University, Univer- sity of Guelph, Lakehead University, University of Lethbridge, MacEwan University, McGill University, Mount Royal University, University of Ottawa, Queen’s University, University of Regina, University of Saskatchewan, Simon Fraser University, University of New-Brunswick, Vancouver Island University, University of Victoria, University of Waterloo, University of Windsor, York University). The authors would like to acknowledge fruitful discussions with Richard Dumont and Stéphanie Gagnon (Université de Montréal), Clare Appavoo and Kim Silk (CRKN / RCDR), as well as the contribution of Nicolas Bérubé, Sarah Cameron-Pesant, Fei Shu, Gita Ghiasi, Stefanie Haustein, Adèle Paul-Hus, Diane Marie Plante, Ilya Razykov, Maxime Sainte-Marie, and Niusha Zohoorian for data cleaning and management, and Laurence Dumont for handling the ethics approval process. Notes 1. Vincent Larivière, Stefanie Haustein, and Philippe Mongeon, “The Oligopoly of Academic Publishers in the Digital Era,” PloS one 10, no. 6 (2015): e0127502. 2. Hal R. Varian, “Differential Pricing and Efficiency,” First Monday 1, no. 2 (1996); Theodore C. Bergstrom et al., “Evaluating Big Deal Journal Bundles,” Proceedings of the National Academy of Sciences 111, no. 26 (2014): 9425–30. 3. Thomas E. Nisonger, “Journals in the Core Collection: Definition, Identification, and Applications,” Serials Librarian 51, no. 3/4 (2007): 51–73. 4. Bergstrom et al., “Evaluating Big Deal Journal Bundles,” 9426. 5. Karla Hahn, “The State of the Large Publisher Bundle: Findings from an ARL Member Survey,” ARL Bimonthly Report 245, no. 5 (2006). 6. Richard Poynder, “The Big Deal: Not Price but Cost,” Information Today 28, no. 8 (2011), www.infotoday. com/it/sep11/The-Big-Deal-Not-Price-But-Cost.shtml. 7. Association of Research Libraries, ARL Statistics 2010–11 (Washington, D.C: Association of Research Libraries, 2011), http://www.arl.org/storage/documents/monograph-serial-costs.pdf. 8. ARL Statistics 2010–11. 9. Fei Shu et al., “Is It Such a Big Deal? On the Cost of Journal Use in the Digital Era,” College & Research Libraries 79, no. 6 (2018): 785–98. 10. R. Anderson, “When the Wolf Finally Arrives: Big Deal Cancelations in North American Libraries” (blog post, May 1, 2017), https://scholarlykitchen.sspnet.org/2017/05/01/wolf-finally-arrives-big-deal-cancelations-north- american-libraries/; Karla L. Strieb and Julia C. Blixrud, “Unwrapping the Bundle: An Examination of Research Libraries and the ‘Big Deal’,” portal: Libraries and the Academy 14, no. 4 (2014): 587–615. 11. Edward S. Allen, “Periodicals for Mathematicians,” Science 70, no. 1825 (1929): 592–94; Paul L.K. Gross and Edward M. Gross, “College Libraries and Chemical Education,” Science 66, no. 1713 (1927): 385–89; Herman H. Henkle, “The Periodical Literature of Biochemistry,” Bulletin of the Medical Library Association 27, no. 2 (1938): 139–47. 12. Eugene Garfield, “Introduction I. Journals, References, and Citations,” in Journal Citation Reports®: A Bib- liometric Analysis of References Processed for the 1974 Science Citation Index®, ed. E. Garfield. Science Citation Index, Volume 9, 1975 Annual. 13. Vincent Larivière and Cassidy R. Sugimoto, “The Journal Impact Factor: A Brief History, Critique, and Discussion of Adverse Effects,” in Springer Handbook of Science and Technology Indicators, eds. W. Glänzel et al. http://www.infotoday.com/it/sep11/The-Big-Deal-Not-Price-But-Cost.shtml http://www.infotoday.com/it/sep11/The-Big-Deal-Not-Price-But-Cost.shtml http://www.arl.org/storage/documents/monograph-serial-costs.pdf https://scholarlykitchen.sspnet.org/2017/05/01/wolf-finally-arrives-big-deal-cancelations-north-american-libraries/ https://scholarlykitchen.sspnet.org/2017/05/01/wolf-finally-arrives-big-deal-cancelations-north-american-libraries/ Collection Development in the Era of Big Deals 235 (Cham, Switzerland: Springer International Publishing, 2018), https://arxiv.org/abs/1801.08992. 14. J.A. Urquhart, “Why Libraries Are Cancelling Periodicals and What Can We Do about It,” in The Future of Publishing by Scientific and Technological Societies (Luxembourg: Office for Official Publications of the European Community, 1978), 19–30. 15. Rob Kling and Ewa Callahan, “Electronic Journals, the Internet, and Scholarly Communication,” Annual Review of Information Science and Technology 37, no. 1 (2003): 127–77. 16. Dana L. Roth, “The Serials Crisis Revisited,” Serials Librarian 18, no. 1/2 (August 2, 1990): 123–29. 17. Genya O’Gara and Anne C. Osterman, “Negotiating on Our Terms: Harnessing the Collective Power of the Consortium to Transform the Journal Subscription Model,” Collection Management 44, no. 2/4 (2019): 176–94. 18. Mike McGrath, “Fighting Back against the Big Deals: A Success Story from the UK,” Interlending & Docu- ment Supply 40, no. 4 (2012): 178–86. 19. SPARC, “Big Deal Cancellation Tracking,” https://sparcopen.org/our-work/big-deal-cancellation-tracking/ [accessed 5 January 2020]. 20. Heather Piwowar et al., “The State of OA: A Large-scale Analysis of the Prevalence and Impact of Open Access Articles,” Peer J 6 (2018): e4375. 21. Thomas Nisonger, “Use of Journal Citation Reports for Serials Management in Research Libraries: An Investigation of the Effect of Self-Citation on Journal Rankings in Library and Information Science and Genet- ics,” College & Research Libraries 61 (May 1, 2000). 22. Roth, “The Serials Crisis Revisited,” 123–29; W. Loughner, “Scientific Journal Usage in a Large Univer- sity Library: A Local Citation Analysis,” Serials Librarian 29, no. 3/4 (1996): 79–88; S.E. Edwards, “Making Hard Choices: Using Data to Make Collections Decisions,” Qualitative and Quantitative Methods in Libraries 4 (2015): 43–52; Stephanie H. Wical and R. Todd Vandenbark, “Combining Citation Studies and Usage Statistics to Build a Stronger Collection,” Library Resources & Technical Services 59, no. 1 (January 2015): 33–42. 23. Carole Francq, “Bottoming Out the Bottomless Pit with the Journal Usage/Cost Relational Index,” Technical Services Quarterly 11, no. 4 (June 27, 1994): 13–26; Wayne A. Pedersen, Janet Arcand, and Mark Forbis, “The Big Deal, Interlibrary Loan, and Building the User-Centered Journal Collection: A Case Study,” Serials Review 40, no. 4 (October 2, 2014): 242–50. 24. Terry Bucknell, “Usage Statistics for Big Deals: Supporting Library Decision-Making,” Learned Publishing 21, no. 3 (July 2008): 193–99; Deborah D. Blecic et al., “Deal or No Deal? Evaluating Big Deals and Their Journals,” College & Research Libraries 74, no. 2 (March 2013): 178–93; Cecilia Botero, Steven Carrico, and Michele R. Tennant, “Using Comparative Online Journal Usage Studies to Assess the Big Deal,” Library Resources & Technical Services 52, no. 2 (April 29, 2011): 61–68; Wical and Vanderbark, “Combining Citation Studies and Usage Statistics,” 33–42. 25. Bergstrom et al., “Evaluating Big Deal Journal Bundles,” 9425–30; Virginia Martin et al., “Analyzing Consortial ‘Big Deals’ via a Cost-Per-Cited-Reference (CPCR) Metric,” Serials Review 42, no. 4 (2016): 293–305. 26. Shu et al., “Is It Such a Big Deal?” 785–98. 27. Beatriz Betancourt Hardy, Martha C. Zimmerman, and Laura A. Hanscom, “Cutting without Cursing: A Successful Cancellation Project,” Serials Librarian 71, no. 2 (August 17, 2016): 112–20. 28. Norm Medeiros, “Uses of Necessity or Uses of Convenience? What Usage Statistics Reveal and Conceal About Electronic Resources,” in Usage Statistics of E-Serials, ed. David C. Fowler (New York, NY: Haworth Infor- mation Pres, 2007), 233–43; J.N. Gatten and T. Sanville, “An Orderly Retreat from the Big Deal: Is It Possible for Consortia?” D-Lib Magazine 10, no. 10 (2004); Juan Gorraiz, Christian Gumpenberger, and Christian Schloegl, “Usage versus Citation Behaviours in Four Subject Areas,” Scientometrics 101, no. 2 (November 2014): 1077–95. 29. Medeiros, “Uses of Necessity or Uses of Convenience?” 233–43. 30. Edwards, “Making Hard Choices,” 43–52. 31. Gorraiz et al., “Usage versus Citation Behaviours in Four Subject Areas,” 1077–95. 32. Katherine Chew et al., “E-Journal Metrics for Collection Management: Exploring Disciplinary Usage Dif- ferences in Scopus and Web of Science,” Evidence Based Library and Information Practice 11, no. 2 (2016): 97–120. 33. Mangkhollen Singson, S. Thiyagarajan, and M. Leeladharan, “Relationship between Electronic Journal Downloads and Citations in Library Consortia,” Library Review 65, no. 6/7 (September 5, 2016): 429–44; John D. McDonald, “Understanding Journal Usage: A Statistical Analysis of Citation and Use,” Journal of the American Society for Information Science and Technology 58, no. 1 (January 1, 2007): 39–50. 34. Dorothy Milne and Bill Tiffany, “A Cost-per-Use Method for Evaluating the Cost-Effectiveness of Serials: A Detailed Discussion of Methodology,” Serials Review 17, no. 2 (June 1, 1991): 7–19; Hugh Franklin, “Comparing Quarterly Use Study Results for Marginal Serials at Oregon State University,” Serials Librarian 16, no. 1/2 (June 26, 1989): 109–22. 35. Ebrahim Emrani, Amin Moradi-Salari, and Hamid R. Jamali, “Usage Data, E-Journal Selection, and Ne- gotiations: An Iranian Consortium Experience,” Serials Review 36, no. 2 (June 2010): 86–92; Blecic et al., “Deal or https://arxiv.org/abs/1801.08992 https://sparcopen.org/our-work/big-deal-cancellation-tracking/ 236 College & Research Libraries March 2021 No Deal?” 178–93. 36. Jane Stephens et al., “Citation Behavior of Aerospace Engineering Faculty,” Journal of Academic Librarian- ship 39, no. 6 (November 1, 2013): 451–57. 37. Richard L. Trueswell, “Some Behavioral Patterns of Library Users: The 80/20 Rule,” Wilson Library Bul- letin 43, no. 5 (1969): 458–61; Wical and Vandenbark, “Combining Citation Studies and Usage Statistics,” 33–42; Botero, Carrico, and Tennant, “Using Comparative Online Journal Usage Studies,” 61–68; James Stemper and Janice Jaguszewski, “Usage Statistics for Electronic Journals,” Collection Management 28 (January 1, 2004): 3–22. 38. Blecic et al., “Deal or No Deal?” 178–93. 39. Miquel Termens, “Looking below the Surface: The Use of Electronic Journals by the Members of a Library Consortium,” Library Collections Acquisitions & Technical Services 32, no. 2 (2008): 76–85. 40. Gorraiz et al., “Usage versus Citation Behaviours in Four Subject Areas,” 1077–95. 41. P.M. Davis, “Patterns in Electronic Journal Usage: Challenging the Composition of Geographic Consortia,” College & Research Libraries 63, no. 6 (November 2002): 484–97; Stephens et al., “Citation Behavior of Aerospace Engineering Faculty,” 451–57; Li Zhang, “Citation Analysis for Collection Development: A Study of International Relations Journal Literature,” Library Collections, Acquisitions, & Technical Services 31, no. 3/4 (September 1, 2007): 195–207. 42. Kathleen E. Joswick and Jeanne Koekkoek Stierman, “Perceptions vs. Use: Comparing Faculty Evalua- tions of Journal Titles with Faculty and Student Usage,” Journal of Academic Librarianship 21, no. 6 (1995): 454–58; Alain R. Lamothe, “Factors Influencing the Usage of an Electronic Book Collection: Size of the E-book Collection, the Student Population, and the Faculty Population,” College & Research Libraries 74, no. 1 (2013): 39–59; Vincent Larivière, Cassidy R. Sugimoto, and Pierrette Bergeron, “In Their Own Image? A Comparison of Doctoral Stu- dents’ and Faculty Members’ Referencing Behavior,” Journal of the American Society for Information Science and Technology 64, no. 5 (2013): 1045–54. 43. Cassidy R. Sugimoto and Vincent Larivière, Measuring Research: What Everyone Needs to Know (Oxford, UK: Oxford University Press, 2018). 44. Four universities decided to opt out of the survey part of the project (Concordia University, Memorial University, University of British Columbia, and University of Manitoba), while one university had already per- formed a similar survey in 2015 (Université de Montréal). 45. UlrichsWeb, Global Serials Directory (2019), http://ulrichsweb.serialssolutions.com/login. 46. Despite the fact that conference proceedings are important documents in fields such as computer science, they were removed from the analysis as most of them are not part of publishers’ big deals, are often available in open access, and their publication schedule is different from that of scholarly journals. 47. UlrichsWeb, Global Serials Directory. 48. Samuel C. Bradford, “Sources of Information on Specific Subjects,” Engineering 137 (1934): 85–86. 49. Randall Collins, “Why the Social Sciences Won’t Become High-consensus, Rapid-discovery Science,” Sociological Forum 9, no. 2 (1994): 155–77; George Ritzer, “Sociology: A Multiple Paradigm Science,” American Sociologist (1975): 156–67. 50. Given that Waterloo did not provide platform-level download data, it is excluded from these analyses. 51. Bradford, “Sources of Information on Specific Subjects.” 52. Philippe Mongeon and Adèle Paul-Hus, “The Journal Coverage of Web of Science and Scopus: A Com- parative Analysis,” Scientometrics 106, no. 1 (2016): 213–28. 53. Piwowar, “The State of OA: A Large-scale Analysis of the Prevalence and Impact of Open Access Articles.” 54. Heather Piwowar, Jason Priem, and Richard Orr, “The Future of OA: A Large-scale Analysis Projecting Open Access Publication and Readership,” BioRxiv (2019): 795310. 55. Ross Atkinson, “Six Key Challenges for the Future of Collection Development,” Library Resources & Tech- nical Services 50, no. 4 (2011): 244–51. http://ulrichsweb.serialssolutions.com/login