key: cord-0944664-nid5dza9
authors: Maleki, Ashraf
title: Why does library holding format really matter for book impact assessment?: Modelling the relationship between citations and altmetrics with print and electronic holdings
date: 2021-12-20
journal: Scientometrics
DOI: 10.1007/s11192-021-04239-9
sha: 3c36d8134d6d37a799dbd1b95608c936e49d2c80
doc_id: 944664
cord_uid: nid5dza9

Scholarly books are important outputs in some fields and their many publishing formats seem to introduce opportunities to scrutinize their impact. As there is a growing interest in the publisher-enforced massive collection of ebooks in libraries in the past decade, this study examined how this influences the relationship that library print holdings (LPH), library electronic holdings (LEH) and total library holdings (TLH) have with other metrics. As a follow up study to a previous research on OCLC library holdings, the relationship between library holdings and twelve other metrics including Scopus Citations, Google Books (GB) Citations, Goodreads engagements, and Altmetric indicators were examined for 119,794 Scopus-indexed book titles across 26 fields. Present study confirms the weak correlation levels observed between TLH and other indicators in previous studies and contributes additional evidence that print holdings can moderately reflect research, educational and online impact of books consistently more efficient than eholdings and total holdings across fields and over time, except for Mendeley for which eholdings slightly prevailed. Regression models indicated that along with other dimensions, Google Books Citations frequently best explained LPH (in 14 out of 26 fields), whereas Goodreads User counts were weak, but the best predictor of both LEH and TLH (in 15 fields out of 26), suggesting significant association of eholdings with online uptake of books. Overall, findings suggest that inclusion of eholdings overrides the more impactful counts of print holdings in Total Library Holdings metric and therefore undermines the statistical results, whilst print holdings has both statistically and theoretically promising underlying assumptions for prediction of impact of books and shows greater promise than the general Library Holding metric for book impact assessment. Thus, there is a need for a distinction between print and electronic holding counts to be made, otherwise total library holding data need to be interpreted with caution. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s11192-021-04239-9.

Books are important scholarly outputs in Social Science and Humanities and their many publishing formats (print and various electronic formats) introduce opportunities to investigate their various aspects of impact. As quantifying impact in humanities is important (Nederhof, 2006; Torres-Salinas & Moed, 2009 ) current research concerns with quantifying impact of books across their various publishing formats. Libraries are the major source of data for academic books and most scholars seek to obtain the materials they need from their institutional library rather than a bookstore. Library Holding (LH) refers to the number of libraries that have acquired a certain book title in their collection. A recent study into the WorldCat (a significant source of library holding statistics) has indicated that U.S. libraries (about 63%) are predominant among an overall 15,195 libraries worldwide, of which 38% (5804) account for academic libraries, 29% (4441) public libraries and 33% (4950) other libraries . This suggests that there can be stronger academic theme in WorldCat's library holding counts for scholarly books.

Two different studies in 2009 introduced library holdings as a potentially useful indicator for assessment of book impact, one of them calling it Libcitations short for Library Citations (White et al., 2009) and the other naming it Catalog Inclusions (Torres-Salinas & Moed, 2009 ). It is introduced as an indicator of books' acquisition worth from librarians' point of view who choose titles based on institutional and community requests. As shown in an earlier investigation into publication format of books in libraries (Maleki, 2021) , broad coverage is the major reason for importance of library holding statistics for books in WorldCat's OCLC (Online Computer Library Centre). Work by Torres-Salinas and Moed (2009) and White et al. (2009) have made a significant contribution to the area of book impact assessment. However, ever since they have introduced library holdings as an indicator of book impact, the feature of Work Format in WorldCat, which is one of the central services for Library Management and the underlying factor in calculating holding statistics, has not been investigated (Maleki, 2021) .

One important reason for significance of book format is that academic libraries have shown a growing interest to acquire ebooks in the past decade, particularly in the U.S. (Romano, 2016) . WorldCat provides access to over 50 million records from 729 ebook providers through users' libraries 1 and the number of libraries having access to ebook records is mentioned as eholdings in OCLC API results. Admittedly, there are good reasons for libraries to provide digital version of books; ebooks are key to sustaining library service at such times as current COVID-19 pandemic by automatically removing the need for physical presence of users. However, for about a decade, what we know about library holdings is largely based upon empirical studies that have extensively used Total Library Holdings for book impact assessment which is indirectly representing the sum of counts of library print and electronic holdings. A major problem with this kind of application is that it does not take into account the different methods in which print and electronic books are acquired for libraries.

Libraries may order ebooks based on specific packages from publishers, which means that some books without actual demand can also find their way into holding statistics. However, as it also is not clear how much library print book orders are made based upon real information need, various studies have assessed library book circulation for print (Cabezas-Clavijo et al., 2013) and ebooks (Cox, 2011) or both (e.g. Haugh, 2016) . Considering this, it would be reasonable to raise concerns about how library holding statistics need to be interpreted in terms of impact. Print books constitute a larger proportion of the libraries' costs and space. Library holding counts as well as library loan statistics (Cabezas-Clavijo et al., 2013) are perhaps the most popular methods to track impact of print book collection. Other than inter-library loans, books provided via any acquisition model are meant to become a perpetual part of libraries. Perpetual access translates into higher expenses, but also a more careful selection policy. The major challenge would be with ebooks in libraries as not all types of access to them is perpetual. Majority of academic libraries in the U.S. provide them via subscription (79% of libraries) perhaps to be economical (Kont, 2016) , and then, title by title purchases (75%), demand-driven purchases (49%), upfront purchases (34%), demand-driven short-term loans (18%), ebook approval plans (11%) and combined ebook and print approval plans (10%) (Romano, 2016) . Thus, regardless of demand for all titles, there are substantially larger number of libraries having access to them. It would, therefore, be reasonable to question and investigate the usefulness of ebook holdings presence in library holding counts as an indicator of book impact assessment. This research addresses the questions of whether and to what extent ebooks and print books in libraries are in line with academic, educational, and cultural needs for books in academia and society.

A discussion about the problem of massive collections of ebooks has emerged recently in an analysis of WorldCat Identities (WI) for author level indicator of book holdings, where 398 most prolific book authors in information science were identified with false assignment of book titles and thus an inordinate number of holding counts that originated from ebooks (Torres-Salinas, Arroyo-Machado and Thelwall, 2020) . A previous investigation into print and electronic holdings also showed that the average number of electronic holdings for each title is about 7 times more numerous than print holdings (Maleki, 2021) , showing that a significant part of electronic holding is included in total library holding counts. However, it has not been investigated what sort of implications these differences in provision of the print and electronic books have for impact assessment.

Current research is a follow up study to my previous investigation on comparison of fourteen book metrics (Maleki, 2021) . This research will use 119,794 Scopus-indexed books in 26 fields by libraries across work formats and other metric counts and over time to investigate relationships between these metrics. Correlation test provides initial evidence of potential connection between indicators. The perception behind testing the correlation between library holdings for different publication formats of books and other indicators is to examine the extent to which provision of each publication format represent library mission in covering needs for books. Thus, current research aims to identify the relationship that exists (Sud & Thelwall, 2014) between library holdings of each publication format with formal citations to books and other metrics in order to clarify aspects of impact. The major notion in this research is that because print books and ebooks are acquired by libraries in different methods and to varying extent, it is possible that print and ebook holdings reveal various aspects of impact, hidden from previous statistical correlation studies that used total holding counts. Thus, current research will investigate the difference between library print holdings, library electronic holdings and total library holdings in the empirical relationship they have with various dimensions of book impact including formal citations from Scopus and Google Books as sources of research related impact (Kousha & Thelwall, 2009 Kousha et al., 2011) , syllabus mentions as a source of educational impact , Goodreads captures as an indicator of cultural impact , and altmetric indicators as sources of social media uptake of books (Thelwall et al., 2013) .

As metrics for assessing books are relatively numerous and less known for the aspect of impact they can measure, it would be valuable to learn which metrics perform strongly when statistically explaining LH metrics. Except a few metrics such as syllabus mention that clearly measures educational uptake of books, most assessment metrics such as online social media metrics, however, have a multi-dimensional nature and might be driven for variety of social, educational and scholarly reasons. In this respect, some metrics perform stronger than others in revealing a certain aspect of impact such as Mendeley readerships which might indicate scholarly uptake as well as simple readership (Mohammadi et al., 2016) . Although correlation analysis helps to identify this type of metrics usage, correlation only shows relationship between pairs of metrics and might not help to identify the predominant aspect of metrics usage when they are frequently used for different purposes. In order to take multi-dimensional factors of metrics into accounts and reveal metrics which are best aligned with LH metrics, there is a need for a regression analysis that uses normalized form of all metrics without excluding documents that have zero metric counts. This will help acquiring a universal understanding of metrics strength in predicting each one of LH metrics for different subject fields.

Furthermore, as uptake of books by library holdings metric is comprehensive and substantially larger than any other book metric in a short time after books' first edition publication (Maleki, 2021) , this research also intends to explore the predictability of various aspects of book impact using print and electronic holding counts over time.

The aim of this study is to ascertain the usefulness of book publication format in libraries for book impact assessment. In order to address this goal, correlations between holding counts across print and electronic formats and other metrics are investigated and regression models for predicting metrics with print and electronic book holdings in libraries across disciplines and over time are offered. This research, thus, seeks to address the following questions:

1. Do library print and electronic holdings correlate with other book metrics? 2. Which aspect of impact is best predicted by print, electronic and total library holdings? 3. Is there a significant statistical benefit to use library print holdings instead of library eholdings or total library holdings? 4. To what extent can all book metrics in the study statistically explain Library Holdings across its format? 5. How are the patterns in above questions changed across disciplines and over time?

The relationship between library holdings and many other indicators is already tested in different studies and reviewed in Torres-Salinas and Arroyo-Machado (2020). Previous studies have used it to indicate whether library holding counts can indicate other aspects Kousha and Thelwall (2015) 451 Choice-reviewed books multidisciplinary (2010) Kousha (2011) Zuccala (2015) Kousha (2015) Kousha (2017) Torres-Salinas (2017) Zuccala ( Table 1 ). However, the strongest coefficients with non-WorldCat metrics were seen between library holdings and Goodreads ratings (r = 0.467) and Goodreads reviews (r = 0.448) . Overall, these studies have used the evidence of weak and low moderate coefficients to suggest that library holdings reflect aspects of impact different from both citations and other assessment metrics. Some general patterns could also be followed in the correlation results in terms of subject area and publication date. For instance, library holdings have often shown stronger correlation coefficients in Social Science and Art and Humanities than Medicine and Physical Science with various metrics such as formal citations, Amazon Reviews, and Goodreads Reviews (e.g. Kousha et al., 2011; ) (more in Table 1 ). A few studies have also concerned to apply publication year into the correlation analyses. Older books have often shown stronger correlation coefficient with library holdings than more recent books . One study on books holdings in Association of Research Libraries assessed the correlation between citations and library holdings in fields of History and Literature for books published in 1996-2000 and 2007-2011. In both fields, the later time period showed a weaker relationship (Zuccala & White, 2015) . Kousha et al. (2016) also similarly reported a reduction in strength of correlation coefficients in more recent years across many fields. None of the above-mentioned studies have used publication year of first edition of books instead of the publication date of publishers, although they have reported elimination of duplicate editions.

Some recent studies, have used the number of library holdings across regions and various geographies using entropy method, reporting significant weak Spearman's correlation coefficients with Choice reviews in recommendation, readership and interdisciplinary subject levels. One researcher suggested that WorldCat country distributions with r = 0.959 (p < 0.01) is in strong connection with the Library holdings (Zuccala et al., 2021) . Another study reported the maximum r = 0.407 between distribution of holdings and Choice readerships in Science and Technology books (Zhou & Zhang, 2020b) . In two other studies, they have shown a weak correlation between a compound score based on width and breadth score of books and holding numbers at r = 0.050 (Zhang & Zhou, 2020) and slightly better correlations for reference books at r = 0.209 (Zhou & Zhang, 2020a) , both significant at p = 0.01.

However, correlation results from previous studies need to be considered carefully; firstly, because the relationship has not been examined for library holding counts across book formats; and secondly, due to the differences of examined datasets. The datasets used for correlation analysis sometimes varied in terms of inclusion or exclusion of books with zero metric counts. In other words, some studies used books with at least one of the respective events as mentioned in Table 1 and excluded books that have never been cited, mentioned, or reviewed (e.g., Kousha & Thelwall, 2015; . Exclusion of books with zero metric counts can lead to different correlation coefficients from when they are included (Haustein et al., 2015) . Some studies have examined the correlation analysis only for books meeting certain criteria. For instance, the correlation analyses in were conducted on 997 popular and academic History books at Scopus citations (≥ 4) and Goodreads Ratings (≥ 10) above the 75th percentile across two academic library holdings from ARL (American Association of Research Libraries) and non-ARL (International academic libraries). In the present study, in order to make research results comparable with previous studies, correlation coefficient has been reported for both zeros included and only non-zero datasets of books. A case comparison is also incorporated in the discussion. Furthermore, to indicate the usefulness of work format-based library holdings count, all correlations for print and eholdings are compared with total holdings.

This research uses two datasets from an earlier research (Maleki, 2021) . The research design was planned to scrutinize differences in library print and electronic holdings with each other and with total library holdings in terms of impact. The aim was to examine the relationship that library print and electronic holdings have with citations, and other book-specific indicators. Throughout this paper, the abbreviations LPH, LEH and TLH are used to refer to Library Print Holdings, Library Electronic Holdings and Total Library Holdings, respectively. Book data are drawn from seven main sources: (1) Scopus, (2) OCLC (classify.oclc.org); (3) WorldCat (worldcat.org); (4) Altmetric.com; (5) Open Syllabus Project (opensyllabus.org); (6) Google Books (books.google.com); and (7) Goodreads (goodreads.com). Figure 1 gives an overview of the research process in terms of data and analysis. A complete and a sample dataset are selected to investigate the research questions: Fig. 1 The process of data collection and analysis (Tables 2, 3 , adopted from Maleki, 2021) • Dataset 2 involved all titles in six subject categories (Anthropology, Arts, Business and Economics, Law, Medicine and Political Science) from Dataset 1. This sample is selected for further analyses by: (a) The first edition's publication date of books in WorldCat and (b) Educational impact of books that is manually extracted from opensyllabus.org. The major criteria for selecting the subjects are importance of books in those fields.

Library Print and Electronic Holdings OCLC extended results were collected by submitting ISBNs or title and author name to OCLC API (in January 2020) to extract 'holding' and 'e-holding' of books. The total holding in OCLC is the sum of these two metrics.

Google Books Citations (GB) Google Books offers perhaps the most promising source for identifying book citations to both journal articles (Kousha & Thelwall, 2008) and monographs (Kousha & Thelwall, 2015) . However, as its results are prone to include publication advertisements and other non-relevant documents, it is important to clear the results to get true citation matches. The modification can be done automatically, for example, by the software Webometric Analyst (lexiurl.wlv.uk) in which Google Books citations can be automatically downloaded and analyzed. Here, Webometric Analyst's preproduced protocol to harness Google Books API was used to extract Book-to-Book citations (Kousha & Thelwall, 2015) during December 2019. The least word counts of titles was set to four, in order to reduce, the possibility of false positive retrievals in the search results. The maximum word count in titles was set to six and up to three authors' last names were included in the searches.

Syllabus Mentions (SM) Previous studies have used search engines such as Bing to identify mentions of journal articles (Kousha & Thelwall, 2008) as well as monographs in online syllabi and course reading lists. However, due to the limitations in Bing's free searches, in this research the list of publications in Open Syllabus Project (opensyllabus.org) was used to identify books used for teaching. Open Syllabus Project (OSP) claims to collects list of over 1 million syllabi from about 10 to 15 years ago and mostly from English speaking countries. It also accepts syllabi of volunteer donors and copyright restricted syllabi might not be found in it. This, however, does not undermine the value of this list for studying teaching impact of resources as it might include more nononline syllabi due to receiving them directly from the universities which would, otherwise, remain unidentified with search engine-based exploration of resources. Opensyllabus.org was used to manually aggregate the count of university curricula referring to a book title in March and April 2020. Due to extensive manual checking required, only six fields (Dataset 2) were searched in the Open Syllabus explorer.

Goodreads Engagements The online social networking website of Goodreads has been introduced in previous studies as a suitable source for exploiting informal reader ratings about scholarly books . User text reviews and other Goodreads metrics are also shown to reflect various aspects of impact such as educational, cultural and research value about books and hence they can be considered as a multipurpose source of impact metrics . The counts of users interacting with books in Goodreads, also known in this research as Goodreads Users (GU), star ratings or Goodreads Ratings (GR), Amazon and Goodreads textual comments or Goodreads Text Reviews (GTR) and the mean ratings given to books by users on a five-star assessment or Goodreads Average Rating (GAR) were extracted from Goodreads.com (October 2018) through submitting ISBNs or title and author of books via API. ISBN searches were run using the Webometric Analyst and the title and author searches were submitted via a custom software. In order to deal with numerous pages for some titles in Goodreads, total work engagements for the book rather than page statistics were used.

Altmetric.com Altmetric.com mainly offers mentions of research articles (Adie & Roe, 2013) on various websites and online social networking sites such as Twitter, Facebook, News Outlets, Blogs, and Wikipedia. It also offers mentions of scholarly books in the metrics it covers. For this study, all altmetric indicators were extracted from altmetric.com via API using ISBNs in January 2020. However, only five indicators of Mendeley readers, Twitter unique users, Facebook unique users, Blog Mentions and News Stories are mentioned in this research, as other altmetric.com metrics showed only negligible counts for books. The general usability of the selected metric counts for books was also very limited (Maleki, 2020) , but they are entered into the regression models in order to investigate the possible improvements in the prediction models.

Publication Year of First Edition Given the cumulative nature of library holding counts over time (Maleki, 2021) , the first edition publication date of books were used instead of their republication dates. This mainly is because first edition year of books have a place in the cumulated events either as citations or holding and using it has been shown to normalise the effect that time has on citation accumulation across different fields (Thelwall & Fairclough, 2015) . Publication years are grouped in six consecutive time periods including prior to 2003, 2003-2005, 2006-2008, 2009-2011, 2012-2014 and 2015-2017 in order to include enough books in each publication range to reach statistically meaningful calculations from 40 (2003-2005) for Arts to 3499 (2012-2014) for Business and Economics.

Correlation Analysis Spearman's correlation results between the three library holdingbased metrics (LPH, LEH, and TLH) and other indicators were analysed. Two datasets of books with zero metric counts and without zero counts were examined in a parallel way to produce more robust results for future comparisons. Due to dealing with citation data which often have skewed distribution, Spearman's Correlation coefficients are used.

Prediction Analysis The regression models in present study were designed in two ways (a) to determine the relative ability of library print, electronic and total holdings in predicting citation and altmetric indicators; and (b) to explore the role of metrics in multivariate book assessment models for explaining variances for library print, electronic, and total holdings by other book indicators. Thus, the prediction tasks were supposed to show the extent to which and how library holding metrics, print or electronic, could be explained by other metrics, either combined or alone.

For prediction analysis, the Ordinary Least Squares Regression model was constructed on log-normalized counts of metrics (c) as proposed by Thelwall (2017) . Log-transformed data create normalized distributions suitable for linear regression and produce more realistic results for metrics with many zero metric counts (Thelwall, 2017) , which is apparent in current research for majority of indicators (see Table 1 ). By using this method all books either with zero or non-zero metric counts were included in the models in order to reach a more universal understanding of impact. In order to perform log-transformation, first all books with or without respective metric counts were added one unit (c + 1), and then natural logarithm was performed on them. To test all metrics equally for all 26 fields, Enter model is used which receives all independent variables at once rather than stepwise. Despite using all metrics in models, other combinations of metrics were also examined to aid discussions about the libcitation's capabilities and differences as tools for impact assessment. Subject-level findings on coefficients and R 2 scores for all the eleven metrics are in the Online Appendix. 2 In order to meet the pre-conditions of linear regression, in addition to log-transformation, multicollinearity, condition index and tolerance of metrics are also controlled. In sum, because of strong correlation level between Goodreads Rating and Goodreads Users (r = 0.833, p < 0.001), Goodreads Ratings metric is removed from the regression analyses to prevent collinearity. All other variables met the collinearity requirements (condition index < 8) and were tolerable (VIF < 5) in the regression models.

In response to the first research question, Table 4 indicates the average of significant correlation coefficients across 26 fields. Six Spearman correlation analyses were performed for each field between library holding metrics (LPH, LEH and TLH) and one other metric across its two datasets, (a) zeros included and (b) zeros excluded. The average correlation coefficients between metrics and LPH with zero counts of metrics (−0.009 < r < 0.534) and without zero counts (−0.140 < r < 0.533) were significant weak or moderate and placed at higher ranges than TLH (−0.102 < r < 0.347 and −0.167 < r < 0.366, respectively) and substantially above LEH (−0.050 < r < 0.269 and −0.149 < r < 0.214, respectively). This suggests that TLH, which is the sum of LPH and LEH, depicts correlation coefficients in the middle range below LPH and above LEH. However, it needs to be noted that the average correlation coefficients across fields are used here to only indicate the general differences between LH metrics and other indicators. Thus, these coefficient values should not be used or need to be used cautiously as they are not the actual relationship coefficients as in fieldlevel and the significance of the correlation levels cannot be specified to the average values. The correlation coefficients at subject level along with significance of coefficients need to be found and referenced if necessary from Figshare Online Appendix Tables 1-6. Correlation Coefficients for Zero vs. Non-zero Metric Counts Including books with zero metrics counts have generally increased the average correlation coefficients across fields for all metrics, except for Goodreads-related and online metrics. This increase was seen in the average correlation coefficients of the three holding metrics, however, more noticeably in LPH when comparing the dataset with non-zero to zero-included dataset for Scopus Citations (0.443 and 0.481, respectively), GB Citations (0.385 and 0.431), and Syllabus Mentions (0.349 and 0.534), suggesting that books without respective events can also have relatively lower library print holdings, probably due to prominence of academic library members in WorldCat. However, removing books with zero Goodreads Users increased the median correlations with LPH (0.385-0.533) and TLH (0.342-0.366) but decreased them with LEH (0.269-0.214). This suggests that print holdings of books without Goodreads Users can still be rather high, while books with more library eholdings are enjoying slightly better Goodreads engagement. This pattern is recurrent for Goodreads Ratings and Text Reviews, but not Average Ratings (Table 4) .

Because books also need time to accrue citations, it is important to check the changes in correlation levels over years. In order to cover the answer to the fifth research question, Table 5 shows average significant correlation coefficients between LPH and LEH and other metrics for six fields, over six time periods. Like the overall correlation coefficients, the strength of correlations between LPH and metrics is greater than that of LEH across all years and disciplines, despite occasional inconsistencies in years prior to 2008 in three areas of Anthropology, Arts and Economics and Business due to lower frequency of books (Online Appendix Tables 7 and 8) .

Correlations Trend for LPH Table 5 also suggests that the correlations between LPH and other metrics are significantly moderate or weak and despite a general downward trend, often have undulated over time. Overall, the patterns for different metrics tend to break in 2009-2011 when correlations are reinforced before falling again. The strongest average correlation coefficients across fields were frequently observed in 2003-2005 for the relationship between LPH and Syllabus Mentions at 0.523, GB Citations at 0.418, Goodreads Average Ratings at 0.346, as well as Wikipedia Articles at 0.152. Most of the coefficients between LPH and the altmetric indicators showed a weak peak below 0.3 at different time periods. However, for three indicators of Scopus Citation (0.494), Goodreads Users (0.448), and Goodreads Reviews (0.465), the strongest average correlation coefficients across fields are experienced in 2009-2011. This is mainly because library holding counts' pattern was broken from 2010 onwards (Maleki, 2021) , presumably reflecting the considerable rise in counts of print book holdings contemporary with the same rise in electronic holdings. At disciplinary level, the highest correlation coefficients observed in this research were in [2003] [2004] [2005] between LPH and GB Citations in Arts at 0.664 (n = 33) and LPH and Syllabus Mentions in Political Science at 0.601 (n = 203), both significant at p < 0.001 (see Online Appendix Table 7) .

Correlations Trend for LEH The correlations between LEH and most of the metrics similarly showed a downward trend, but did not significantly surpass 0.3, with the exception of LEH and Goodreads Rating (r = 0.42) in Arts 2003-2005 which is not essential, due to lower book frequency (n = 40). Correlations between LEH and Google Books Citations and Syllabus Mentions were even significant negative in some years in Law and Medicine (see Online Appendix Table 8 ), suggesting that electronic versions of older books with scholarly and educational usage are far less acquired compared to more recent ebooks. Table 6 Average R 2 scores of linear regression models across 26 fields (left) and six sampled fields (right)

There are two independent variables only in the first model (LPH and LEH) and there is only one independent variable in the other three models Bold percentages represent the metric with strongest prediction ratio in the regression model Metrics/models 

In response to the second research question, the relative statistical power of print, electronic and total library holding counts in predicting other metrics was examined. Table 6 shows the average adjusted R square scores of four models across 26 fields: first model represents print holding along with eholding counts as two predictors of the model and the three remaining models represent one variable (Library Total Holdings, Library Print Holdings, or Library Eholdings) as the sole predictors of each metric. This is examined for all fields and the averages are reported for both Datasets 1 and 2.

Comparing LPH Only models with LPH along with LEH Table 6 gives the average R 2 scores of various metrics across 26 fields. It indicates that when LPH and LEH together were entered as the combined predictors of the models (24.5%), the average R 2 scores of Scopus Citations were slightly stronger than that of LPH as the sole predictor of models (21.9%), however substantially stronger than those of only TLH (12.8%) and only LEH predictor models (5.1%). This suggests that despite very weak prediction power of eholdings compared to print holdings, there still is an underlying ability in eholdings to slightly improve the average of prediction scores, when accompanying LPH, particularly for Goodreads Users (17.7% in only LPH vs. 21.3% in LPH and LEH), but very minimally for GB Citations (19.3% vs. 19.6%). The results in field level are offered in the Online Appendix Tables 9-11.

Is LPH Always Stronger than LEH in Predicting Metrics? In response to the third research question, Table 7 gives the average standardized coefficients across fields for the combined LPH-and LEH-predictor models. Since the purpose of prediction task in this research was only to speculate underlying differences between print holdings and eholdings rather than giving a prediction formula, the standardized β is reported as the regression coefficient instead of unstandardized B. Thus, for instance, the strongest average regression coefficient across fields in Table 7 , which relates to Syllabus Mentions (0.480), suggests that by one standard deviation increase in Syllabus Mentions, on average LPH increases by 0.480 times of its own standard deviation across fields. Table 7 suggests that metrics' variance in all models are best explained by LPH, except for Mendeley Readers. The field level results for Mendeley show that in 18 (70%) out of 26 fields, LEH had stronger coefficients. At disciplinary level, library ebook holdings showed a significant advantage for a considerable number of fields in predicting Twitter (16 of 23 fields), Facebook (11 of 22 fields), and Wikipedia (11 of 21 fields) uptake of books, as well. The extended prediction results of each metric-field for Table 7 are given in the  online appendix Tables 12-25 .

Because of a friction in the pattern of library holding counts for books published from 2010 onwards (Maleki, 2021) , the robustness of results produced in the above models is examined by repeating the regressions across the periods of years for six sample fields, illustrated for their averages in Fig. 2 (see field-based results in the online appendix Figs. 1-6) . Results show that majority of the average R 2 scores over years by LEH as the predictor were often below 5%; by TLH were between 5 and 10%; and by LPH were between 10 and 20%. This suggests that for various metrics the best predictions over years are made by LPH, despite broad variations and instability over time.

In order to investigate the underlying differences between LPH, LEH and TLH in terms of the impact they indicate, statistical prominence of aspects of impact in co-presence of different metrics is examined. In response to the fourth research question, we statistically explain various dimensions of LPH, LEH or TLH through regression models using various sets of all other metrics, this time as the independent variables. Table 8 shows average adjusted R 2 scores of models that predict each of LPH, LEH, and TLH, by single or various combined sets of metrics across 26 fields (Syllabus mentions excluded) and six sampled fields where Syllabus Mentions metric is included in the models. In terms of prediction rate, in all models, LPH has significantly better prediction rate than TLH and LEH, except for two single-independent-variable models of Mendeley and Twitter where LEH had very small but still larger R 2 scores (2.8% and 1.3%, respectively) than LPH (0.8% and 0.9%, respectively). This is in line with the correlation results. In presence of all 12 metrics, average R 2 scores show that LPH (44.7%) is 1.8 times better explained than TLH (24.6%), whilst LEH (10%) is as 0.4 times TLH prediction rate (Table 8) .

Single-independent-variable Models On average the best prediction rate by one independent variable was made with Scopus Citations (21.9%) for LPH, whereas the highest for TLH and LEH was seen with Goodreads Users (6.5% and 13%, respectively), which is a persistent pattern between the 26-and six-field datasets.

Two-independent-variable Models The highest average R 2 score for models with two independent variables in 26-field dataset, are for LPH predicted by GB*GU (31.8%); the best models for TLH and LEH consisted of Scopus*GU (19.3% and 8.8%, respectively). The same pattern persists in six-field dataset, suggesting the key role that GB Citations play in LPH prediction models and Scopus Citations in LEH and TLH prediction models.

Three-independent-variable Models The contribution of Syllabus Mentions (SM*GB*GU) to three independent variable models is slightly more substantial (41%) than Scopus Citations (Scopus*GB*GU) (39.3%) for LPH, whereas the reverse is true for TLH (20.1% and 21.5%, respectively) and LEH (6.2% and 8%, respectively) with higher contribution of Scopus Citations.

More than Three Independent Variable Models By increasing the number of variables in the model the adjusted R 2 scores improved with one of the best average scores reached by the four major indicators of Scopus*GB*SM*GU which explained 43.3% of the variance for LPH, 22.5% of the variance for TLH and 8.2% of the variance for LEH. The contributions of GTR and GAR to the models were very minimal (at best 0.4% for TLH). Very minimal improvements (max. of 1.2% for LPH) in the models occurred when six Altmetric indicators, namely Mendeley readers, Wikipedia Citations, Twitter Unique Users, Scopus: Scopus citations; GB: Google books citations; SM: Syllabus mentions; GU: Goodreads users; GR: Goodreads ratings; GTR: Goodreads text reviews; GAR: Goodreads average ratings; Empty cells are not tested because syllabus mentions was not available. Facebook: Facebook users; Wiki: Wikipedia articles; Twitter: Twitter users; Mendeley: Mendeley readers; News: News outlets; Blogs; Blog pages. Bold-Italic R 2 scores indicate the strongest metric or set of metrics in prediction of relevant library holding statistics in each dataset Table 9 Statistically significant linear regression standardized coefficients (β) in six sample fields for TLH

The dependent metric is total library holding. Only Goodreads rating is excluded from the models due to causing collinearity Bold coefficients represent the metric with the strongest prediction rate in the field All R 2 s are significant at p < 0.001; All coefficients are significant at p 

Statistically significant linear regression standardized coefficients (β) in six sample fields for LPH

The dependent metric is library print holding. Only Goodreads rating is excluded from the models due to causing collinearity Bold coefficients represent the metric with the strongest prediction rate in the field All R 2 s are significant at p < 0.001; All coefficients are significant at p 

Statistically significant linear regression standardized coefficients (β) in six sample fields for LEH

The dependent metric is library electronic holding. Only Goodreads rating is excluded from the models due to causing collinearity Bold coefficients represent the metric with the strongest prediction rate in the field All R 2 s are significant at p < 0.001; All coefficients are significant at p Facebook Unique Users, News Posts, and Blog Mentions, were included (Scopus*GB*S M*GU*GTR*GAR*Mendeley*Wiki*Twitter* Face*News*Blog) reaching to 44.7% for LPH, 10% for LEH and 24.6% for TLH. Tables 9, 10 , 11 show the standardized coefficients (β) of the 12 independent variables in the last model for predicting TLH (Table 9) , LPH (Table 10) , and LEH (Table 11) in six sample fields. The major indicators explaining the variances for TLH are Scopus Citations (in 4 out of six fields), and Goodreads Users (2 fields); for LPH are Google Books Citations (4 fields), Scopus Citations (1 field) and Syllabus Mentions (1 field), and for LEH are Goodreads Users (3 fields), Scopus Citations (2 fields) and Mendeley Readers (1 field). Mendeley, Twitter, and Facebook had negative coefficients in LPH prediction models, whilst this is also true for LEH, the coefficients of Mendeley for LEH were more likely to be positive, particularly in Medicine (Table 11 ). The reverse was true for GB Citations whose coefficients in LPH models were always positive and in LEH insignificant or weak negative.

The significant standardized regression coefficients (β) of the models with 11 independent variables (excluding Syllabus Mentions) are given for 26 fields in the Tables 26-28 in Online Appendix. Figure 3 summarizes the results for these syllabus mention-excluded models, in terms of the number of fields which had the strongest standardized regression coefficient for the respective metric. It was found that inclusion of Syllabus Mentions was helpful and important in the LPH prediction models of all fields (particularly in Medicine) (Table 10) , otherwise, in absence of Syllabus Mentions, GB Citations (14 out of 26 fields), Scopus Citations (7 fields), or Goodreads Users (5 fields) would be the strongest prediction factors (Fig. 3) . Majority of the fields in prediction of LEH and TLH had the strongest Goodreads User coefficients (both 15 fields), confirming the above results.

One limitation encountered in the analysis was that Goodreads Rating and Goodreads Users were highly correlated and it was not possible to include them simultaneously in all regression analysis. Therefore, Goodreads Rating were excluded from all models as their inclusion would have interfered with Goodreads Users with negative coefficient and caused Fig. 3 Number of fields with the highest significant standardized coefficient for the metric among 11 variables in the regression model (Scopus*GB*GU*GTR*GAR*Mendeley*Wiki*Twitter*Facebook*News*B log) with LPH, LEH and TLH as the independent/predicted variable collinearity in the regression models, whilst it would not have improved the predictive ability of the models nor incorporated minor improvements.

In answer to the first research question, the relationship between "Total Holdings" and other metrics in previous studies (Table 1) , whilst considering the variations in dataset coverages and sizes, was generally close to the correlations reported in this research, perhaps except for Syllabus Mentions and Wikipedia Article Citations. The correlation between library holdings and Syllabus mentions in Kousha and Thelwall (2015) was slightly weaker (0.121) than current research (0.345), possibly due to the larger size of dataset in current study. However, in contrast, for Wikipedia Article Citations all correlations observed in current research were weaker than the best observation (r = 0.387 in 2010 Arts books) in , possibly because of the differences in methods of data collection.

The average correlations with print holdings, despite much stronger than total holdings in this research, were slightly stronger than that of total holdings in previous studies. For Scopus Citations, all the correlation coefficients with Libcitations in previous studies were very weak (max r = 0.208 in , whereas current research found higher average correlations at medium level for LPH both in zero (r = 0.481) and non-zero datasets (r = 0.443). For Google Books Citations, however, the average correlation in non-zeros datasets (r = 0.385) was only slightly stronger than the highest report in previous studies (r = 0.326 in Kousha & Thelwall, 2015) , suggesting that print holdings have often produced weak correlation results with GB citations, despite some medium level correlations (r > 0.4) at subject level (12 out of 26 field, Table 2 in online appendix). For Goodreads Reviews, the average correlation for LPH at zeros dataset (r = 0.385) was slightly above some previous studies, such as r = 0.346 in , and at non-zero dataset, current research showed stronger coefficients (r = 0.533) than (r = 0.446) (further compared in Table 12 ). This suggests that inclusion or exclusion of books with zero events in different metrics is important in the correlation analysis, since strength of correlations can substantially change. In sum, correlation coefficients observed in this research for LPH might be slightly bigger than in previous reports, but previous studies have tested various fields, datasets and years, making comparisons difficult. Thus, the importance of this research lies in the broad range of subjects, years and fields studied and simultaneous examination of total, print and electronic holdings that made it possible to indicate differences.

In order to compare findings with previous studies, Table 12 offers a case comparison of correlation results for the field of History. had reported the strongest correlation coefficients for library holdings on 997 popular and academic History books cited in journal articles in 2007-2011 with Scopus Citations (≥ 4) and Goodreads Ratings (≥ 10) above the 75th percentile. All of the correlation coefficients in current research between TLH and Scopus Citation, Goodreads Rating or Goodreads User/Reviews were significant weak and below the observation in for Non-ARL Holdings (i.e. Non-American, Research Library Holdings) either in the top quartile or total dataset (Table 12 ). In fact, as the correlation coefficients at the top quartile of citations for current research were lower than total dataset, there is a possibility that the same has happened for the data in for both ARL and non-ARL libraries. The reason for the medium-level correlations at non-ARL libraries, but weak correlation level at ARL libraries in is not apparent, but a possible explanation can be more upload of print collections from non-ARL libraries into WorldCat rather than electronic ones which is more common in ARL or American Research Libraries. This inference needs a distinct investigation, perhaps like Zuccala et al. (2021) and Zhou and Zhang's study Table 12 A case comparison of correlation coefficients for books in the field of history between current research and All correlation coefficients are significant at p < 0.001; except *p < 0.05 (2020b), as they give some evidence to suggest that there is a need to make distinction between the libraries in the US and other parts of the world or at least take into account that geographical factors might also have a place in the distribution of print and electronic holdings in WorldCat. Interestingly, in current research, all correlation coefficients between LPH and Goodreads metrics for History books were at the medium level and, particularly in the dataset without zero counts were above non-ARL Holdings' results, both for Goodreads Ratings (r = 0.488) and Goodreads Reviews/Users (r = 0.550). This suggests that Library Print Holdings is still providing stronger coefficients in relationship to other metric counts than total library holdings. Findings for correlation trends suggest that unlike the results in previous studies, library print holdings can be used to moderately indicate various aspects of impact. But the interesting result is that scholarly research impact shown with Scopus Citations and cultural impact depicted by Goodreads Users and Ratings are probably more sensitive to the uptake of print version of books by libraries in the short-term. Educational curricula and booksourced citations take much more time to take effect and become moderately related to print uptake of books in the long-term but always much more useful than library eholdings. Correspondingly, indicators accrued in long term (such as journal-or book-sourced formal citations, and syllabus mentions) can produce stronger correlation results with print holdings when zero counts are included in the dataset. However, metric counts accumulated online (all Goodreads-sourced metrics and altmetric indicators) do not indicate significant long-term improvement and can only produce stronger correlation with non-zero datasets, suggesting their limitations for assessment of all books.

In answer to the second RQ, various models based on LPH and LEH and a mixture of them were tested indicating that LPH has significantly better prediction results than LEH. The robustness of prediction models was tested over time and with varying mixture of metrics. Figure 2 highlighted a few interesting results. Firstly, the sudden jumps in library holdings, that have occurred for books from 2010 onwards, have not created a better predictability of metrics by eholdings, but have been surprisingly contemporary with a significant upgrade in the predictions made by print holdings particularly for Scopus Citations. This is likely to suggest that the expansion of library print collection have been in line with scholarly usage. It is, however, less known whether this results from libraries prioritizing to order print book when university research staff request for a title. Furthermore, R 2 scores for Goodreads Users in both LPH and LEH predictor models show a sudden jump in 2009-2011, ending in 2015-2017 significantly above that of Scopus Citations (particularly for LPH), suggesting faster accumulation of online users for more recent books. Secondly, GB Citations prediction scores over time (max = 13%) with LPH were significantly lower than the average estimate without time scale (21.5%), suggesting that, despite lower variation within time periods, distinctions across time periods are broader in the extent of bookto-book citations accrued. Likewise, the overall prediction of Syllabus Mentions (23.7%) is stronger than that of most periods except for 2006-2008 (25%) , suggesting that LPH is a better predictor of Syllabus Mentions after a longer time period, perhaps at least four (12%) to seven years (17%) after first publication date. These results are reasonable, because they clearly depict that for more recent books slow accumulation of citations results in weak explanation of variances by LPH; whilst these improve over time for LPH, the predictions with LEH hardly improve and always remain weak.

In answer to the third research question, prediction models suggested that there is a clear difference between LPH, LEH and TLH. LPH mainly reflects conventional aspects of book impact, and particularly depicts the importance of traditional research and educational impact, whereas LEH mainly associates with online engagement of users with books either in Goodreads or Mendeley. Nevertheless, TLH indicates aspects of impact inspired by both LPH and LEH, with varying patterns across fields, but overall balance slightly shifted in favor of online uptake of books, since the frequency of LEH counts is higher than LPH. With regards to Mendeley, results suggested that electronic availability of books significantly associates with their addition to Mendeley by users. The exceptions were Arts, Ethics and Religion, Law, Library Science and Bibliography, Philosophy, Psychology and Political Science, where LPH exceeded LEH (Online Appendix Table 20) , which clearly suggest the lower application of Mendeley for Social Science and Humanities.

As with the fourth research question, along the explored dimensions, it was possible to suggest that major book indicators including Scopus citations, Google Books Citations, Syllabus Mentions, and Goodreads engagements were explicitly best explained by library print holding counts across almost all fields; however, none of the combinations of metrics can create particularly prominent results for predicting electronic holdings, even in the exceptional fields of Mathematics and Engineering and Technology, where there were trivial advantages in favor of library ebook holding counts ( Table 28 in Online Appendix). However, success of electronic collection might be seen in fields that Goodreads, that has a better prediction of online impact, had the highest coefficients in print holding prediction compared to eholdings; these fields are Medicine, Library Science and Bibliographies, Mathematics, Education, and Ethics and Religion, since ebook holdings surpluses the effectiveness of print collection.

Since it was generally overlooked how massive collection of ebooks in libraries affect the library holding counts, this research examined the relationship between library holdings across two formats and other metrics showing that library print holdings are more likely to indicate multiple aspects of impact. Librarians select book formats based on research and educational requests and sometimes financial priorities, however, their administration on e-book selection is more likely to be partial because they are primarily offered in packages by publishers and dealers (e.g. ProQuest or EBSCO). Although sometimes selection process in a library is likely to end up with the choice of an unneeded print book as much as an unused e-book, the extent to which this helps to measure an impactful library collection was less known. Consequently, this research investigated whether and how library holding formats can indicate impact or use from various perspectives.

The present study confirmed correlation levels observed for total library holdings in previous studies and contributed additional evidence that print holdings can moderately reflect research, educational and even online impact of books at higher level than total library holdings. It was also suggested that ebook holdings indicate only weak relationships, below that of total library holdings. As citation-like characteristics were observed in library holding data, it was normally seen that correlation levels for books had increased by age as more library holdings were accumulated. The results showed simultaneous dramatic jumps of library holding counts in 2010 for both print and particularly in sizable quantity for ebooks, while the correlation levels of Scopus Citations and Goodreads engagements only rose significantly with print holdings, and did not change with eholdings. This presumably suggests that print book acquisition is often in demand and effective, but current rate of library eholdings does not statistically imply significant levels of impact or use. This suggests that widespread availability of ebooks in libraries, despite probably in more circulation and use, cannot give enough statistical evidence to measure impact. Furthermore, inclusion of books with zero metric counts produces stronger correlation results with traditional citations and syllabus mentions, but decreases for all online impact indicators. Books without library print book holdings are perhaps less likely to have research and educational usages, while highly acquired print books in libraries may not necessarily indicate online impact and therefore online indicators have limited utility in impact assessment.

In scrutiny of differences between library print and ebook holdings, a broad range of metrics and fields were explored in this research. The overall results suggested that print holdings predominantly indicate traditional aspects of impact such as research and educational impact. Eholdings and total holdings, because of more numerous counts of eholdings than print holdings, are basically more likely to lean toward showing online engagements of users. The prediction of metrics could slightly improve if eholdings accompanied print holdings in the models but not for total holdings. Library print holdings alone were consistently more efficient than eholdings and total holdings in predicting book metric counts across fields and over time, except for Mendeley for which eholdings slightly prevailed. This apparently indicates that library ebooks are more likely to be saved in Mendeley than library print books.

In summary, as previous studies have suggested that library holdings for books are plenty to be considered as the most promising source of scholarly impact assessment (Torres-Salinas et al., 2017) , current research suggests to draw a distinction between Library Print Holding counts and Library Eholding counts in terms of level of impact captured and aspects of impact. Print Holding counts is a promising indicator of book impact as it has both statistically and theoretically good underlying assumptions for prediction of book impact compared to total holdings or the accumulated count of print and electronic holdings that so far has been in use in previous studies. Library eholding statistics show a tendency to indicate online readership statistics as in Mendeley, but the current rate of impact associated with that is also weak. It is important to note that massive collection of ebooks in libraries has undermined the statistical results of predicting impact of books almost ubiquitously across all fields for both eholdings and total library holdings, and correspondingly this research proposed library print holding as a better indicator of book impact assessment instead of total library holdings.

There is a hope that this analysis will contribute evidence about how and where library holding metrics can be used to produce the best statistical results in book impact assessment. It particularly sounds more useful to see a growth in the studies that aim to understand "library ebook usage" as supported by for instance SUSHI and COUNTER protocols to investigate the usefulness of ebooks for scholarly book impact assessment.

The online version contains supplementary material available at https:// doi. org/ 10. 1007/ s11192-021-04239-9.

material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.

Altmetric: Enriching scholarly content with article-level discussion and metrics

Most borrowed is most cited? Library loan statistics as a proxy for monograph selection in citation indexes

Librarians' use of usage statistics for journals and e-books

How do you like your books: Print or digital? An analysis on print and e-book usage at the graduate school of education

Characterizing social media metrics of scholarly papers: The effect of document properties and collaboration patterns

Demand-driven E-book program in Tallinn university of technology library: The first two years of experience with the EBL platform. Slavic and East European Information Resources

Assessing the impact of disciplinary research on teaching: An automatic analysis of online syllabuses

Google book search: Citation analysis for social science and the humanities

An automatic method for extracting citations from google books

An automatic method for assessing the teaching impact of books from online academic syllabi

Are Wikipedia citations important evidence of the impact of scholarly articles and books

Assessing the citation impact of books: The role of google books, google scholar, and scopus

Goodreads reviews to assess the wider impacts of books

Why with bibliometrics the humanities does not need to be the weakest link: Indicators for research evaluation based on citations, library holdings, and productivity measures

OCLC library holdings: Assessing availability of academic books in libraries in print and electronic compared to citations and altmetrics

P-Libcitation vs. E-libcitation?: Libraries' print book holdings resonate with citations and altmetrics but E-book holdings do not. The 2020 online altmetrics workshop

Can Mendeley bookmarks reflect readership? A survey of user motivations

Bibliometric monitoring of research performance in the social sciences and the humanities

Survey of Ebook usage in US academic libraries 2016

Evaluating altmetrics

Web indicators for research evaluation: A practical guide

The influence of time and discipline on the magnitude of correlations between citation counts and quality scores

Do altmetrics work? Twitter and ten other social web services

Library catalog analysis and library holdings counts: Origins, methodological issues and application to the field of Informetrics. Evaluative informetrics: The art of metrics-based research assessment

Library catalog analysis as a tool in studies of social sciences and humanities: An exploratory study of published book titles in economics

PlumX as a potential tool to assess the macroscopic multidimensional impact of books

Exploring WorldCat Identities as an altmetric information source: A library catalog analysis experiment in the field of

Libcitations: A measure for comparative assessment of book publications in the humanities and social sciences

Assessing books' depth and breadth via multi-level mining on tables of contents

Evaluating wider impacts of books via fine-grained mining on citation literatures

Measuring book impact via content-level academic review mining. The Electronic Library

Altmetrics for the humanities: Comparing goodreads reader ratings with citations to history books

Performance-based publisher ratings and the visibility/impact of books: Small fish in a big pond

Correlating Libcitations and Citations in the Humanities with WorldCat and Scopus Data

Acknowledgements Author appreciates Statistical Cybermetrics Research Group (SCRG) for the software Webometric Analyst and Altmetric.com, Goodreads.com and OCLC.org teams for supporting API accesses to data. Author also appreciates anonymous reviewers for their invaluable comments.Funding Open Access funding provided by University of Turku (UTU) including Turku University Central Hospital.

Ashraf Maleki 1 * Ashraf Maleki ashraf.maleki@utu.fi