Research Article
A Comparison of Traditional Book Reviews and
Amazon.com Book Reviews of Fiction Using a Content Analysis Approach
Christy Sich
Research & Instructional Librarian
Western University
London, ON, Canada
Email: csich@uwo.ca
Received: 29 June 2015 Accepted:
14 Feb. 2017
2017 Sich. This is an Open Access article
distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0
International (http://creativecommons.org/licenses/by-nc-sa/4.0/),
which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly attributed, not used for commercial
purposes, and, if transformed, the resulting work is redistributed under the
same or similar license to this one.
Abstract
Objective - This study compared the quality and helpfulness of traditional
book review sources with the online user rating system in Amazon.com in order
to determine if one mode is superior to the other and should be used by library
selectors to assist in making purchasing decisions.
Methods - For this study, 228 reviews of 7 different novels were analyzed
using a content analysis approach. Of these, 127 reviews came from traditional
review sources and 101 reviews were published on Amazon.com.
Results
- Using a checklist developed for this study, a significant
difference in the quality of reviews was discovered. Reviews from traditional
sources scored significantly higher than reviews from Amazon.com. The
researcher also looked at review length. On average, Amazon.com reviews are
shorter than reviews from traditional sources. Review rating—favourable,
unfavourable, or mixed/neutral—also showed a lack of consistency between the
two modes of reviews.
Conclusion
- Although
Amazon.com provides multiple reviews of a book on one convenient site,
traditional sources of professionally written reviews would most likely save
librarians more time in making purchasing decisions, given the higher quality
of the review assessment.
Introduction
Collection development training often promotes the use of book
reviews to assist librarians in choosing fiction for library collections. Book
reviewing as a systematic evaluation of literature first began with the
publication of the Journal des Savants
in 1665 (Boaz, 1958). Traditional book review sources for libraries include
such periodicals as Book Review Digest,
Booklist, Book World, Kirkus Reviews, and Library
Journal. Other traditional book review sources include Saturday Review, Observer,
New York Times Book Review, and The
New Yorker.
In recent years, with the decline of the newspaper industry, the
place for book reviews has come into question. Many stand-alone publications
have folded and book review sections have been amalgamated with other sections
or they have been slashed completely (Ciabattari, 2011). At the same time, with
the rise of the web, independent book reviewers have cropped up in various
online manifestations including Google Books, Goodreads, LibraryThing and
Amazon, just to name a few. These new online platforms are convenient and allow
readers and librarians quick, point-of-need access to reviews.
Not surprisingly, those in the formal book reviewing industry
scorn these types of reader review and ratings systems, criticizing reviews
published on Amazon for being anonymous or sometimes even fraudulent with
claims made that authors seek out friends and family to write glowing reviews
(Harmon, 2004). Pool (2007) argues that “Amazon has created a system that not
only allows but encourages ethical and literary standards far lower than those
we find in print reviewing” (p. 100).
Still, despite the criticism from the book reviewing industry,
Amazon seems like a quick and easy way for librarians to get a sense of a
book’s quality and whether or not it might be worthy of selection. In order to
test the validity of the anecdotal claims about the flaws with Amazon reviews,
and to be able to recommend review sources as tools for collections librarians,
this study aims to measure the quality of reviews found on Amazon and compares
the results to the quality of reviews in traditional publications, using a content
analysis approach. By examining the texts of the reviews, the researcher
assessed the quality and helpfulness of reviews found on the Amazon.com website
to test the claims about the poor quality of Amazon reviews.
In order to determine the quality, the author scored each
individual review using a checklist of 12 categories developed for this study
(see Appendix). The categories of analysis were adapted from John E. Drewry’s Writing Book Reviews (1974), using
elements that make up a helpful review. The 12 categories that the researchers
looked for in each review included character source, character treatment, plot
elements, plot devices, plot summary, theme, setting, four categories of style,
and the inclusion of an evaluative statement about the book. In this study,
quality was determined by looking at the total helpfulness score. Results of
the study will help librarians responsible for collections of fiction decide
whether or not to make use of Amazon as a collections development tool.
Literature Review
The deluge of the number of books published is something that
librarians and readers alike must grapple with. The impetus and context for
this study was formed, in part, through an analysis of some of the literature
on book reviewing. In particular, this research was motivated by the apparent
anxiety of critics, researchers, and writers of reviews who have attempted to
make a case for formal, professional book reviews by condemning the free,
online book review sources for encouraging amateur book reviewing. The
literature reviewed in this study can be organized thematically by the role or
purpose of book reviews, authority and anonymity of reviews—especially in
online formats and the quality of reviews.
Role or Purpose of the Book Review
There has been tension between critics and writers regarding the
purpose of the book review from the early days of periodical publishing. In her
essay, Reviewing (1939), Virginia Woolf unabashedly unleashed her
contempt of book reviews and the critics. Woolf proposed a new system and
questioned the purpose of reviews: “Why bother to write reviews or to read them
or to quote them if in the end the reader must decide the question for
himself?” (p. 12).
Interestingly, when Reviewing was published, Woolf’s
husband contributed his own contradictory commentary, defending the role of
reviews to some degree, by suggesting that with the burgeoning of readers and
the number of books published that the function of the reviewer is “to give to
readers a description of the book and an estimate of its quality in order that
he may know whether or not it is the kind of book which he may want to read”
(Woolf, 1939, p. 29).
Just as with readers needing some guidance on what books to buy
and read, librarians responsible for building large collections of fiction in
both academic libraries and public libraries can also benefit from tools to
help make purchasing decisions, especially when faced with dwindling budgets.
Some articles, including Natowitz and Carlo (1997), Palmer (1991), Burchette
(1992), Greene and Spornick (1995), touch on the impact of reviews on
collection development in libraries as well as on individual purchasing
decisions. In particular, Natowitz and Carlo examined book reviews published in
Choice, Journal of American History, and American Historical
Review to determine the degree to which book reviews provide assessments to
help with acquisition choices. Natowitz and Carlo’s research showed that
different variables involved in book reviewing depend upon the journal the review
is written for, making the case that awareness of different types of reviews in
journals allows librarians to make informed collections decisions.
In reviewing the literature on book reviewing, Blake (1989)
covered the role of the review extensively and noted that there is a connection
between the number of book reviews of a single title and the number of library
collections in which that title can be found. Blake indicated that research on
the reviewing of non-print materials had largely been neglected at that time
and was an area that needed further study. In another study, Palmer (1991)
pointed to the inconsistent role that reviews play in collection development in
libraries by demonstrating that reviews are relied upon heavily by some
libraries and not at all by others.
Pinfold (2007) focused on book reviews as a tool for collections
librarians, explaining that, despite the prevalence of approval plans whereby
libraries automatically receive books based on pre-set parameters rather than
reviews, he heavily relies on reviews to help with selection decisions. Pinfold
also commented on reviews from Amazon, stressing that they should be ignored
because of the lack of peer review.
Authority and Anonymity
As revealed in Pinfold’s paper, a major critique of book reviews
from online platforms such as Amazon is that they lack authority and the peer
review process. Ciabattari (2011) explored this further by discussing the
resurrection of the book review, arguing that just as there is a proliferation
of books being published each year, so too is there a proliferation of
reviewers commenting on books in the online world. Reliance on these reviews then is all the
more difficult because of the amount of reviews available. Ciabattari notes
that “readers can find book news and reviews in formats ranging from a hundred
forty characters to six thousand words and up, online and in print. … Despite
the flood of friendly recommendations coming from Amazon and the social networking
sites, many readers still turn to familiar gatekeepers for curatorial guidance”
(2011, p. 122).
Pool (2007) emphasized the lack of quality control in the free
online world: “In self-published reviews on the Web … critical failings are and
are bound to be exacerbated. … Unscreened, anonymous, and unedited,
self-published reviewers can be—and often are—as biased, uninformed,
ungrammatical, and critically illiterate as they like” (p. 122). To highlight
the uncritical aspect of online reviews further, Harmon (2004) reported on a
glitch that occurred in 2004 when Amazon reviewers’ real names were displayed
for a short time, revealing that some authors had given their own works and
works of friends glowing, five-star reviews.
Quality of Reviews
In a study on the sunny book review, Katz (1985) discusses the
disproportionate number of positive reviews to negative reviews. Katz suggests
that doubling the number of reviews produced with an emphasis on negative
reviews would be more beneficial to librarians.
Some studies have focused on the evaluative aspects of reviews.
Bilhartz (1984) looks at the changes over time by reviewing works about history
and notes that certain periods seemed to produce more critical reviews than
other periods. Authors before 1960 could expect uncritical praise, but by the
sixties, a new breed of reviewers had arrived. They were not so accepting and
who began to write much more critically.
Boaz (1958) similarly points out that the nature of book reviews
changes over time but suggests that this change has to do with an increase of
educated readers. Book reviews were quite scathing in the earlier days of
reviewing, speaking to a small educated few. By the mid-Twentieth Century the
number of educated readers increased and so too did the volume of book
production. At this time, book reviews became more favourable as the book
reviews served increasingly as a means of notifying readers of books and their
contents. It was left to the reader or librarian to make the ultimate decision
about reading or purchasing the work.
In a more recent study, Hartley, Cowan, Deeson, and Thomas (2016)
looked at the quality of individuals' writing to see if any changes occur over
time. The researchers compared the writing styles of 5 different academic
writers of book reviews over a 20-year span to see if there were any changes in
style over time. They used automated measures of spelling, grammar, and
readability found in Microsoft Word 8. They found that, generally, each
reviewer remained relatively consistent over time.
Interestingly, Witucke (1982) compares reviews from several
traditional reviewing publications, examining various aspects, such as coverage
and promptness, and concludes that selectors cannot rely on a single reviewing
publication to get a comprehensive view of a given book.
Liu, Chen, and Chiu (2013) propose a book review recommendation
system that scrapes the web for all available reviews and systematically
presents the reviews to the user in a ranked order based on quality. In order
to evaluate the quality of the reviews for this proposed system, the
researchers discuss two approaches. One was to look at textual features of the
reviews and the other was to look at non-textual features of reviews. Liu and
co-researchers chose to focus their study on non-textual features. These
included book review length, time factor, book rating, and reviewers'
reputation as ways to measure the book review quality. This author would argue
that there are some flawed assumptions built into the features used to score
reviews. For instance, the study gives more credit to lengthier reviews, based
on comments made from an earlier study (Jurca, Garcin, Talwar, & Faltings,
2010) that looked at product reviews and found that lengthier reviews were
perceived by other users as being written by someone with more authority on the
product. It is unclear as to how the length would necessarily contribute to the
quality of a review. Similarly, the timeliness of the review was given a higher
score stating that earlier reviews are more influential. Again, it is unclear
how a review's date can indicate its level of quality.
Aims
This study looks at the textual features of reviews and provides
an alternative approach to assessing the quality of reviews. The purpose is to assess the quality of the
reviews found on Amazon.com and determine whether or not Amazon is an
appropriate tool for librarians, who are responsible for building collections
of fiction to use to inform purchase decisions. The study compares book reviews
posted on Amazon.com with book reviews published in more traditional book
review sources, such as periodicals and newspapers. The author hypothesized a
significant difference between the quality of reviews published in traditional
sources and reviews found on Amazon.com. Therefore, the null hypothesis
is no significant difference between the quality of traditional reviews and the
quality of Amazon reviews. This study
was undertaken with several research questions in mind:
- Is
there a significant difference between the quality of reviews found in
traditional reviews and reviews published on Amazon?
- Is
there a difference in rating agreement across the two types of reviews?
- Does
one type of source have a higher number of favourable, neutral, or
unfavourable reviews than the other?
- Is
the average length of reviews from one type of source an indication of the
quality of the reviews from that type of source?
To this author’s knowledge no studies have yet compared the
quality and helpfulness of traditional book review sources with online user
rating systems such as Amazon.com. A comparison of the two modes of reviews
will help to clarify anecdotal claims that one mode may be superior to the
other. This has the potential of assisting library selectors in identifying
appropriate sources for reviews, saving time, and making the most of library
collections budgets.
Methods
For this study, the author determined that analyzing reviews of
the same titles to look at rating agreement across the different modes of
reviews would be important. As such, randomly selecting reviews from each type
of review source was not possible. Similarly, selecting titles at random was
difficult because often not enough reviews of the randomly selected titles existed
in either the traditional book review sources or on Amazon. Around the time of
this study’s inception, the Lost Man Booker Prize was announced. The purpose of
this special Booker prize was to acknowledge retrospectively novels that had
been published in 1970 and had missed an opportunity to compete when the Booker
Prize changed its rules in 1971 to look at works of fiction in the current year
only. This seemed to be an opportunity to make use of a selected list of books
that would most likely have been reviewed in the traditional review sources.
Two types of reviews were selected for analysis: published reviews
that had been indexed in Book Review Index and Amazon.com reviews that
were written, in order to avoid bias, prior (pre-2008) to the announcement of
the Lost Man Booker Prize. This study used reviews from seven books long-listed
for the Lost Man Booker Prize because they each had six or more reviews from
each type of review source. Table 1 shows the books and number of reviews used
in the study. In total, 127 reviews from traditional published review sources
and 101 reviews from Amazon.com were analyzed.
Table 1
Books Reviewed
|
Title |
Total
number of reviews published in traditional sources |
Total
number of Amazon reviews |
|
Bomber |
22 |
11 |
|
Troubles |
8 |
16 |
|
The
Bay of Noon |
22 |
6 |
|
I’m
the King of the Castle |
8 |
13 |
|
A
Fairly Honourable Defeat |
23 |
15 |
|
Fire
from Heaven |
24 |
34 |
|
The
Driver’s Seat |
20 |
6 |
|
Total |
127 |
101 |
In order to assess the quality of each review, the review’s
helpfulness was scored using a coded checklist developed for this
study that involved 12 categories (see Appendix). The higher the
helpfulness score, the greater the quality of the review, and the more helpful
it is to a librarian making a purchase decision. The categories were developed
and adapted from a list of recommended elements that reviewers should address
when reviewing fiction from John E. Drewry’s Writing Book Reviews (1974).
Written as a guide for review practitioners, Drewry's work was selected for
this study because it was thorough and included a chapter on reviewing fiction.
The goal of Drewry's book was to encourage competent reviewing. Drewry was a
former Dean of the University of Georgia's School of Journalism and not only
wrote reviews for various publications but also taught a course in book
reviewing.
For this study, one point was given to a review for each of the
defined categories that it addressed. The categories included character source
(fictional or historical), character treatment (or character development), plot
elements (how elements are handled, e.g., introduction, suspense, climax,
conclusion), plot devices (e.g., catastrophe, accident, fate, mystery,
sub-plots etc.), plot summary (the plot is described), theme (a comment about
the overall theme is made), setting (a comment is made about the setting or
background), four categories of style (clarity, sentence structure, emotional
qualities, and narrator perspective), and inclusion of a statement of judgment
about the book. The length (word count) of the review and the evaluative
rating—whether the review was favorable, unfavorable or mixed/neutral—were also
documented for analysis. The coders looked for specific words such as
recommended or not recommended to determine a rating.
Best practices in content analysis encourage inter-coder
reliability testing. To follow these principles of best practice and to
establish objectivity in content analysis, two coders were used in the study.
The author coded the entire sample of reviews, and a second coder, a librarian
colleague, was enlisted and trained. The author and second coder met several
times. In two training sessions, they coded together reviews not included in
this study, using the checklist. Where they disagreed, they made changes to the
checklist. They met a third time after independently coding 10 non-study
reviews, and they tweaked the chart again.
The second coder ultimately coded a random sample of 50 of the 228
reviews using the checklist. Initially, the author supplied the coder with 10
reviews from the study and did a preliminary calculation of percent agreement
to make sure that an acceptable level of agreement was being met. At this
point, the coder then coded an additional 40 reviews from the study. A free,
online program called ReCal was used to establish inter-coder reliability by
calculating percent agreement. Cohen’s kappa was selected for this study, as it
takes chance agreement into consideration. According to Lombard, Snyder-Duch,
and Campenella-Brachen (2010), a coefficient of .70 agreement is appropriate
for some studies because some indices, such as Cohen’s kappa, are known to be
more conservative; therefore, lower criteria can be used. In this study,
acceptable levels of agreement were achieved in 10 of the 12 categories (see
Table 2). Additional training may have improved the agreement in the Theme and
Clarity categories.
Results
Quality
The quality of a review was determined by how well a review scored
on the helpfulness checklist (see Table 3). To compare traditional reviews with
Amazon reviews, a two-sample t-test was performed that does not assume equal
variances. The test revealed that the probability of variance is less than
0.05, therefore we can reject the null hypothesis, that there is no significant
difference between the quality of traditional reviews and the quality of Amazon
reviews. The mean score for helpfulness for the traditional reviews (M = 6.57, SD
= 2.399, N = 127) is significantly higher than the scores for the Amazon
reviews (M = 4.81, SD = 1.999, N = 101), using the sample t-test for unequal
variances, t = 5.9, p < 0.0001.
Table 2
Inter-Coder Reliability Testing
|
Elements
of a helpful review |
Cohen’s
kappa |
|
Character source |
.88 |
|
Character treatment |
.89 |
|
Plot elements |
.70 |
|
Plot devices |
.78 |
|
Plot summary |
.92 |
|
Theme |
.60 |
|
Setting |
.87 |
|
Sentence structure |
.80 |
|
Clarity |
.53 |
|
Emotional qualities |
.70 |
|
Narrator perspective |
1.0 |
|
Judgement |
.81 |
Table 3
Review Helpfulness Score by Book
|
Title |
Averagea score
for traditional reviews |
Averagea score
for Amazon reviews |
|
Bomber |
7.0 |
4.8 |
|
Troubles |
6.3 |
6.1 |
|
The
Bay of Noon |
7.2 |
5.7 |
|
I’m
the King of the Castle |
5.0 |
3.4 |
|
A
Fairly Honourable Defeat |
6.2 |
4.5 |
|
Fire
from Heaven |
6.1 |
4.4 |
|
The
Driver’s Seat |
7.3 |
6.5 |
|
Weighted
average score |
6.6 |
4.8 |
Review
Length
A Pearson product-moment correlation coefficient was computed to
assess the relationship between the length of a review and the quality of a
review as measured by the helpfulness score. There was a positive correlation
between the two variables, r = 0.326, n = 228, p = 0. Increases in review
length were correlated with an increase in the helpfulness score. Length of
review was determined by an individual word count. Interestingly, despite the
unlimited space available on its online platform, Amazon reviews tend to be
more succinct on average than traditional reviews. The average word count for
traditional reviews was 394 and the average word count for Amazon reviews was
244. A t-test reveals that this difference is significant (p = 0.0089). There
is a definite tendency toward shorter reviews on Amazon, with well over half
being 300 words or less. The traditional reviews that were analyzed had a
fairly even distribution of varying lengths. Some of the publications
containing the traditional reviews reserved very little room for reviews, while
others had lengthier essays.
Rating
Further research questions that were asked in this study include
the following: Is there a difference in rating across the platforms? Does
Amazon have a higher percentage of positive ratings than the traditional
sources? On average, do the two types of sources agree on rating?
A chi-square test was performed to look at whether a difference
exists between ratings across the two modes. A significant difference was noted
between the number of unfavourable reviews and favourable reviews, depending on
whether the review was from Amazon or a traditional source (see Tables 4 and
5). Traditional sources had more unfavourable reviews and fewer favourable
reviews than expected. The opposite relationship was observed for reviews on
Amazon, which had more favourable reviews and fewer unfavourable reviews than
expected, 
No significant difference existed between the expected number of
unfavourable and neutral/mixed reviews for either modes, 
Also, the sources had no significant difference between the
expected number of favourable and mixed reviews, 
Ranking each book by the highest percentage of favourable views
shows disagreement between the two types of reviews. The Driver’s Seat had
the highest percentage of favourable reviews on Amazon. Conversely, this novel
had the lowest percentage of favourable reviews among the traditional review
sources. Instead, The Bay of Noon received the highest percentage of
favourable reviews in the traditional review sources. Interestingly, the novel
that eventually won the Lost Man Booker Prize, Troubles, was given the
second lowest percentage of favorable reviews by Amazon raters and had only the
fifth highest number of favourable reviews among the traditional review
sources.
Table 4
Amazon Review Ratings
|
Title |
Number of reviews by rating |
Total |
||
|
Favourable |
Mixed/neutral |
Unfavourable |
||
|
A Fairly
Honourable Defeat |
11
(73%) |
3
(20%) |
1
(7%) |
15 |
|
Bomber |
9
(82%) |
2
(18%) |
0
(0%) |
11 |
|
Fire From
Heaven |
26
(76%) |
6
(18%) |
2
(6%) |
34 |
|
I’m the King
of the Castle |
8
(62%) |
4
(31%) |
1
(8%) |
13 |
|
The Bay of
Noon |
5
(83%) |
1
(17%) |
0
(0%) |
6 |
|
The Driver’s
Seat |
6
(100%) |
0
(0%) |
0
(0%) |
6 |
|
Troubles |
12
(75%) |
3
(19%) |
1
(6%) |
16 |
|
Total |
77 |
19 |
5 |
101 |
Table 5
Traditional Review Ratings
|
Title |
Number of reviews by
rating |
Total |
||
|
Favourable |
Mixed/neutral |
Unfavourable |
||
|
A Fairly
Honourable Defeat |
7 (30%) |
9 (39%) |
7 (30%) |
23 |
|
Bomber |
10 (45%) |
6 (27%) |
6 (27%) |
22 |
|
Fire
From Heaven |
16 (67%) |
6 (25%) |
2 (8%) |
24 |
|
I’m
the King of the Castle |
2 (25%) |
5 (63%) |
1 (13%) |
8 |
|
The
Bay of Noon |
17 (77%) |
0 (0%) |
5 (23%) |
22 |
|
The
Driver’s Seat |
4 (20%) |
12 (60%) |
4 (20%) |
20 |
|
Troubles |
2 (25%) |
3 (38%) |
3 (38%) |
8 |
|
Total |
58
|
41 |
28 |
127 |
Discussion
Although Amazon has many benefits as a review source and some
quality reviews do exist, this study shows that the best quality reviews are
found in traditional book review sources. Based on the results of this study,
the null hypothesis that no significant difference exists between the quality
of traditional reviews and the quality of Amazon reviews can be rejected. The
findings of this study reveal that the mean score for helpfulness for
traditional reviews is significantly higher that the score for the Amazon
reviews; therefore, collections librarians would be better equipped for making
purchasing decisions if they avoid Amazon reviews and read reviews found in the
traditional sources, such as those indexed by Book Review Index. Although tracking down these reviews may take
more time, the product is more helpful for making purchasing decisions than
reviews found on Amazon.
Findings in this study also reveal a positive correlation between
length of review with the quality of review. It makes sense that if 12 elements
are needed for a high-quality score, a minimum length is necessary to achieve
this. This study also found that despite having no limits on length in the
online environment, the average word count of the Amazon reviews was
significantly less than the average word count of the reviews from traditional
sources.
One limitation of the study is that the traditional book review
sources were written decades before the reviews on Amazon.com. As Bilhartz
(1984) pointed out, times change, and so do the ways in which writers write
reviews. A follow-up study that compares reviews on Amazon.com to online
sources of contemporary reviews found in proprietary or subscribed sources
would offer additional insight.
Conclusion
Amazon is a tempting source for librarians to use in book
selection largely due to its ease of use and the plethora of reviews that are
often available. With the significant difference in quality that was found in
comparing the Amazon reviews to the available traditional reviews, this
researcher would urge selectors to use Amazon in a limited fashion, and if
used, used in conjunction with more traditional sources. As with other studies,
this research shows that agreement is not consistent even across the
traditional sources; therefore, no single source should be relied upon. This
researcher would recommend that collections librarians seek out traditional
sources of reviews that involve at least an editor in the publication process.
Full text databases that index book reviews from many different traditional
sources are a good alternative to Amazon.com.
Acknowledgments
The
author would like to thank Dan Sich for his assistance coding reviews for the
interrater reliability testing, Ken Meadows for his assistance with the data
analysis of the study, and Lise Doucette for reviewing a draft of this
manuscript.
References
Bilhartz,
T. D. (1984). In 500 words or less: Academic book reviewing in American
history. History Teacher, 17(4),
525–536. Retrieved from https://eric.ed.gov/?id=EJ304820
Blake,
V. L. P. (1989). The role of reviews and reviewing media in the selection
process. Collection Management, 11(1-2), 1–40. http://dx.doi.org/10.1300/J105v11n01_01
Boaz,
M. (1958). Some historical sidelights on book reviewing. In L. C. Merritt
(Ed.), Reviews in library book selection (pp.
179-182). Detroit: Wayne State University Press.
Burchette,
R. B. (1992). An examination of children’s book review media (Masters thesis). University of North
Carolina at Chapel Hill, Chapel Hill.
Ciabattari,
J. (2011). Back from the Dead: The state of book reviewing. Poets &
Writers, 39(5), 121. Retrieved from https://www.pw.org/content/back_from_the_dead_the_state_of_book_reviewing_0?cmnt_all=1
Drewry, J. E. (1974). Writing book reviews. Westport,
Conn.: Greenwood Press.
Greene,
R. J., & Spornick, C. D. (1995). Favorable and unfavorable book reviews: A
quantitative study. The Journal of Academic Librarianship, 21(Journal
Article), 449–453. http://dx.doi.org/10.1016/0099-1333(95)90088-8
Harmon,
A. (2004, Feb 14). Amazon glitch unmasks war of reviewers. The New York
Times. Retrieved from http://www.nytimes.com/2004/02/14/us/amazon-glitch-unmasks-war-of-reviewers.html
Hartley, J.,
Cowan, J., Deeson, C., & Thomas, P. (2016). Book reviews in time. Scientometrics,
109(3), 2123–2128. http://dx.doi.org/10.1007/s11192-016-2114-z
Jurca, R.,
Garcin, F., Talwar, A., & Faltings, B. (2010). Reporting incentives and
biases in online review forums. ACM Transactions on the Web, 4(2),
1–27. https://dx.doi.org/10.1145/1734200.1734202
Katz,
B. (1985). The sunny book review. Technical Services Quarterly, 3(1-2),
17–25. http://dx.doi.org/10.1300/J124v03n01_03
Liu, D. R.,
Chen, W. H., & Chiu, P. H. (2013). Recommending quality book reviews from
heterogeneous websites. Internet Research, 23(1), 27–46. http://dx.doi.org/10.1108/10662241311295764
Lombard, M., Snyder-Duch, J., & Campenella-Brachen, C. (2010).
Practical Resources for Assessing and Reporting Intercoder Reliability in
Content Analysis Research Projects. Retrieved from http://matthewlombard.com/reliability/
Natowitz,
A., & Carlo, P. W. (1997). Evaluating review content for book selection: An
analysis of American history reviews in Choice,
American Historical Review, and Journal of American History. College
& Research Libraries, 58(4), 323–336. http://dx.doi.org/10.5860/crl.58.4.322
Palmer,
J. W. (1991). Fiction selection in Ontario public libraries—How important are
reviews? Public Library Quarterly, 10(4), 39–48. http://dx.doi.org/10.1300/J118v10n04_04
Pinfold,
J. (2007). Reviews: A librarian’s view. African Research & Documentation,
(102), 31–35.
Pool,
G. (2007). Faint praise: The plight of book reviewing in America.
Columbia, Mo.: University of Missouri Press.
Witucke,
V. (1982). The performance of juvenile book review media. Serials Review,
8(1), 49–55. http://dx.doi.org/10.1016/0098-7913(82)90030-2
Woolf, V. (1939). Reviewing. London: Hogarth Press.
Appendix
|
|
Helpfulness Code |
Category Descriptiona |
Score |
|
1 |
CHAR_SOURCE |
Comment
on sources of characters—made up? historical? |
|
|
2 |
CHAR_TREATMENT |
Comment
on treatment of characters, comment on attitude of writer toward his
characters, character development? simple/complex? comment on character
personality |
|
|
3 |
PLOT_ELEMENTS |
Comment
on how (one or more) elements of the plot are handled, e.g., introduction,
conflict, suspense, climax, conclusion (not just mentioned), tension, action |
|
|
4 |
PLOT_DEVICES |
Comment
on plot devices generally used (e.g., catastrophe, accident, fate, mystery,
sub-plots, journey), is plot primary or secondary? (letters, diaries,
flashbacks, etc.) |
|
|
5 |
PLOT_SUMMARY |
Plot
is described or outlined |
|
|
6 |
THEME |
Comment
on overall theme, over-arching ideas in the novel, novel’s message, or
purpose of the novel, e.g., betrayal, love conquers all, good vs. evil |
|
|
7 |
SETTING |
Comment
on setting (historical/local, occupational/institutional, ethereal/esoteric),
background, atmosphere, locale, scenic effects |
|
|
8 |
STYLE_ELEMENTS |
Comment
on elements of style, e.g., words, prose, language used, figures of speech,
sentence structure, paragraphs, tone, allusions, metaphors, symbolism,
pace, aphorisms, truisms |
|
|
9 |
STYLE_INTELLECT |
Comment
on intellectual qualities such as simplicity (e.g., written for children) or
clearness of writing (e.g., triteness, clichés, satire) |
|
|
10 |
STYLE_EMOTIONAL |
Comment
on emotional qualities, e.g., pathos, humour, force, tragedy, pity, horror,
terror, darkness, irony, sarcasm, mysterious, quirky |
|
|
11 |
STYLE_PERSPECTIVE |
Comment
on narration or narrator perspective |
|
|
12 |
JUDGEMENT |
Some
kind of judgement about the book is made—either favorable, neutral/mixed, or
unfavorable |
|
|
13 |
RATING |
Overall
review's rating of the book (favourable, neutral/mixed, or unfavourable) |
aBased on the elements that make up a helpful review from Writing Book Reviews (Drewry,
1974)