Evaluating the Readability of Veterinary Client Handouts from Vetlexicon, Veterinary Partner, and ClientEd

Ann Viera
Veterinary Librarian
Pendergrass Agriculture-Veterinary Medicine Library
University of Tennessee, Knoxville
Knoxville, Tennessee
annviera@utk.edu
ORCID: 0000-0003-3718-4803

Isabella Baxter
Agriculture and Natural Resources Librarian
Pendergrass Agriculture-Veterinary Medicine Library
University of Tennessee, Knoxville
Knoxville, Tennessee
ibaxter1@utk.edu
ORCID: 0000-0003-1788-1871

From January to April 2019, the University of Tennessee Libraries arranged for a trial of Vetlexicon from Vetstream. Vetlexicon offers point-of-care expert veterinary medical opinion service for six species. The trial prompted us to think critically about their collection of 1000 client handouts. An assessment of the readability of the Vetlexicon client handouts would provide an objective measure of the value of the service beyond the sheer number of handouts on offer. We wondered how the handouts in Vetlexicon would compare with others, so we assessed the readability of two additional large collections of veterinary handouts, Veterinary Partner from VIN and ClientEd from Lifelearn (Table 1). Throughout this paper we will use the name of the service providing client handouts to refer to the handout source.

Veterinary clinical information services such as Vetstream, Veterinary Partner, and ClientEd offer veterinarians curated collections of ready-to-use handouts for their clients. Veterinarians and their teams must give clients science-based options combined with the veterinarian’s professional experience, clinical findings, best available evidence, and client preferences (Dean et al. 2017; Stull et al. 2018). To succeed while competing with “Dr. Google” for clients’ attention, veterinarians and their teams require information services that enhance bonds with clients (Lee et al. 2017). A crucial aspect of the veterinarian-client bond is good communication, both verbal and written. While the veterinary profession and veterinary educators have embraced training for verbal communication skills, less attention has been paid to written communication (Royal et al. 2018). Veterinary Technicians in the U.S. have six required communication tasks including “develop and provide client education in a clear and accurate manner at a level the client understands (i.e., oral and written forms, including educational handouts)” (AVMA 2020). Effectively counseling clients based on scientific facts requires assessing their health literacy and communication preferences and providing written communication at a suitable reading level (Boss 2013a, 2013b), which is the focus of our research.

Librarians and veterinarians have investigated how pet health information supports the veterinarian-client bond. Multiple pet species present an information access problem that veterinary team members, medical educators, and librarians deal with daily. Those searching for veterinary information “must have patience for multiple search iterations to capture the majority of the available knowledge” (Alpi et al. 2009). When so much pet health information became available on the Internet, it compounded the problem of client misinformation in veterinary medicine (Mayer 2008).

The effect on animal welfare caused by client misinformation has been previously addressed by veterinarian, author, and continuing education speaker Dr. Nan Boss. Boss urged veterinary professionals to embrace their roles as educators, stating “the best treatment for client misinformation is to offer information before they look elsewhere” (Boss 2003). In that article, Boss suggested offering clients handouts available on CDs from Lifelearn’s ClientEd, the predecessor of one of the handout collections included in this study. Boss also published her own collection of handouts (Boss 2002), practice protocols (Boss 2009), and two editions of a book on educating clients: Educating your clients from A to Z: What to say and how to say it (Boss 1999, 2011). The second edition begins with 16 rules for good communication. Rule Number 13, “Write it Down!” is the most relevant to this study, and the longest. It begins, “Send your clients home with quality, professional easy-to-read materials. They will appreciate your efforts on their behalf. Remember that your best clients are your most educated clients and vice-versa” (Boss 2011).

The value and challenges of educated or informed clients also was recognized by Gorman (2000), Catanzaro (2001), and Gray and Moffett (2010). Veterinarians were urged to regard informed clients as good for both animal welfare and the practice. They were coached to redirect if necessary but to give credit to clients’ research efforts, however misguided, in order to foster a veterinarian-client team approach which would improve compliance with veterinary medical recommendations. Gray and Moffett (2010) stated: “…‘informed’ clients should be a common aim for every consultation.”

Boss was in the vanguard in raising awareness about supplying written materials for veterinary clients, emphasizing the importance of word choice when speaking to a diverse audience ranging from truck drivers to physicians. Boss acknowledged the need to “…quickly ascertain where clients are coming from, so we don’t talk above or beneath their level of understanding” (Boss 2003).

Hare (2005) published an editorial in the Canadian Veterinary Journal designed to increase awareness of health literacy and the importance of using plain language to communicate effectively in all mediums and with all audiences. Shaw and Hunter pushed for adapting a health literacy model from human medicine that could support mutual understanding between clients and veterinarians. The first step in the Health Literacy Model is “Provide information at the clients’ level of comprehension (i.e., functional literacy), and equip clients to discuss (i.e., communicative literacy) and use the information to make decisions (i.e., critical literacy)” (Shaw & Hunter 2017).

Sheats et al. (2019a) noted that human medicine has readability guidelines recommending medical handouts and patient materials and instructions be written at a sixth-grade reading level, and that no such guidelines exist for veterinary medicine. They called for veterinary medicine to establish guidelines, arguing that guidelines would lead to improved animal healthcare.

In addition to veterinary clinicians’ push for increased awareness and adoption of health literacy principles, veterinary librarians have been conducting readability assessments of veterinary documents and information. Murphy (2006) assessed the readability of passages in books and websites on two common animal conditions using the Flesch-Kincaid Readability formula feature in Microsoft Word. Murphy determined that there was limited disease-specific information at an appropriate reading level and noted that the veterinary library’s role is to facilitate access to those quality resources to meet the information needs of veterinary clients. She observed that veterinarians need informed consumers in order to improve compliance with veterinary recommendations, resulting in better animal health. Dorman et al. (2013) used the Flesch-Kincaid Reading Ease and Flesch-Kincaid Grade Level formulas and the Simple Measure of Gobbledygook (SMOG) in an ambitious study of a client brochure-writing assignment by veterinary students in a toxicology course.

Veterinarians have begun conducting readability assessments of materials written for clients as well. Sobolewski et al. (2019) examined consent forms for veterinary clinical trials using three scoring calculators and found none of the forms met readability recommendations. Royal et al. (2018) scored the 10 most often accessed handouts from dvm360.com. Using ReadabilityFormulas.com, they found 9 of 10 handouts scored at or above the 8th grade reading level. Sheats et al. (2019b) used ReadabilityFormulas.com and found all but 1 of 17 equine newsletters and web pages from the American Association of Equine Practitioners were above the 6th grade target. They linked improved readability with client empowerment and satisfaction with veterinary services, and observed that clients with readable information will have no need for Google searches yielding marketing or other materials that are misleading or out-of-date.

Drawing from these studies, our hypothesis was that handouts from Vetlexicon, Veterinary Partner, and ClientEd are written at or above the 8th grade level in the U.S., well above recommended reading level of 6th grade. We adapted methods from the Royal et al. (2018) study.

Handout Selection

A sample of 150 client handouts (50 from each service), were selected and scored for readability. A statistician from University of Tennessee Research Computing Support ran a power analysis, assuming a medium effect size of 0.5 and a power of 0.8, to determine the sample size of 50 handouts from each service, in order to run a two-way analysis of variance (ANOVA) test. We chose to score the 50 most frequently accessed handouts from each service. We sent requests for handouts via email in April 2019 to each of the three services and received the top 50 handouts from Vetlexicon and Veterinary Partner. We did not receive the top 50 for ClientEd, but noticed that handout topics from Vetlexicon and Veterinary Partner were very similar, so we searched ClientEd for the same or similar topics to use as our sample. This tactic was successful, and those 50 handouts became the ClientEd sample set. As we scored the handouts from Vetlexicon and Veterinary Partner lists, we noticed a few duplicates in each list. In July 2019, we sent another request to Vetstream and Veterinary Partner for a list of the next 50 most accessed handouts. From these, we included the next consecutive handout titles, until our sample reached 50 handouts for each service. For both Vetstream and Veterinary Partner, the 50 unique handouts used came from the top 60 most accessed provided by the vendor.

Data Collection and Analysis

This analysis used ReadabilityFormulas.com’s implementation of two readability scoring formulas, the Flesch-Kincaid Grade Level formula (Kincaid et al. 1975) and the Simple Measure of Gobbledygook (SMOG) Index (McLaughlin 1969), to calculate mean-grade-level readability scores. The Flesch-Kincaid Grade Level formula calculates scores based on sentence length and syllables per word.

Grade level_F-K = (0.39 x average sentence length) + (11.8 x average number of syllables per word) - 15.59.

The SMOG Index calculates scores based on the number of polysyllables (words with more than 3 syllables).

Grade level_SMOG = 3 + Square Root ((polysyllable count) x 30/(sentence count)).

Both formulas output their scores as U.S. school grade reading levels, which are straightforward to understand. The Flesch-Kincaid Grade Level formula was selected because it is widely used across readability studies, and the SMOG Index was selected because it works well for health care focused studies (Wang et al. 2013).

Wang et al. (2013) note “there is no clear consensus on how to handle document formatting when calculating reading grade levels.” As a consequence of our using ReadabilityFormulas.com, a freely available online readability calculator used by Royal et al. (2018) the text submission needed to be complete sentences. While the bullet point sections in some handouts consisted of complete sentences, others were comprised of phrases or lists. Similarly, titles, captions, and appendices were omitted from the readability scorer. The entire text of the handout was entered, except for the aforementioned sections. All samples were at least 100 words long.

In consultation with the statistician, we ran a two-way ANOVA test, with one between-subjects factor and one within-subjects factor, using SPSS (version 24). We chose to run a two-way ANOVA because it allowed us to examine not only how the handout source affected the readability scores, but also how the rating systems themselves affected the readability scores. The two-way ANOVA further allowed us to also examine how the source and rating system affect the readability levels together. The readability of the sample of 150 veterinary client handouts was analyzed in a two-way ANOVA test, using an alpha value of .05. The between-subjects factor was client handout source, i.e., Vetlexicon, Veterinary Partner, and ClientEd. The within-subjects factor was the readability rating method, i.e., SMOG and Flesch-Kincaid.

Results from the two-way ANOVA test found significant effects (p≤001) for the rating method main effect, the handout source main effect, and method*handout source (see Table 2). Pairwise comparisons showed a significant mean difference between the SMOG method (M=10.121) and the Flesh-Kincaid method (M=10.745). Pairwise comparisons also indicated a significant mean difference for Vetlexicon (M=9.654) handouts, compared to both ClientEd (M=10.629) and Veterinary Partner (M=11.016) handouts. The difference in means between ClientEd and Veterinary Partner handouts was not significant (see Table 3).

The SMOG method scored handout readability significantly lower than the Flesch-Kincaid rating method across all handout sources, and the mean rating difference among sources is smaller when rating with the SMOG method than with the Flesch-Kincaid method (see Figure 1).

The Appendix provides the titles of the 150 handouts with their scores for the Flesch-Kincaid Grade Level Formula and the SMOG Index, an average of the two readability scores, and notes on whether or not a bullet point section was omitted from the scoring. The handout titles are sorted from lowest to highest average readability score. 113 handouts had an average score that was between the 9th and 11th grade reading levels, 21 handouts scores averaged below the 9th grade reading level, and 16 handouts were scored above a 12th grade level.

Vetlexicon handouts had a mean score of a 9th grade reading level, which was significantly different than the mean rating scores of ClientEd (10th grade level) and Veterinary Partner (11th grade level). In other words, Vetlexicon most often had handouts with lower readability scores than ClientEd and Veterinary Partner. As the mean scores between ClientEd and Veterinary Partner were not significantly different, we cannot say for certain which of those services has more lower scoring handouts. While the mean handout scores help us compare the veterinary information services, they also show that none of the three services have many handouts scoring near the recommended 6th grade level for human medical handouts. This is consistent with Sheats et. al. (2019b), who found that 16 out of the 17 resources they examined were above the 6th grade level and Royal et al. (2018) that 9 of 10 client handouts from dvm360.com were above an 8th grade reading level.

Across the three handout sources, the SMOG index produced a lower score than the Flesch-Kincaid Grade Level formula. Our comparison, however, is not intended to determine which readability formula should be used for evaluating veterinary handouts. Rather, we intended to see whether there was a consistent interaction between rating formulas and handout source. A difference in rating levels is expected, as each uses different calculations to produce a score (Wang et al. 2013).

Consistent with the assertion by Sheats et al. (2019a), we found that handouts containing “unfamiliar anatomic terms, medical jargon, and complex descriptions of disease pathophysiology” had higher readability scores. The 10 highest scoring handouts in this analysis had average scores that ranged from 12.2 to 13.0. Handout topics included Cushing’s syndrome (hyperadrenocorticism) and Addison’s disease (hypoadrenocorticism), upper respiratory infections, marijuana toxicity, a drug handout on fluoxetine, and pancreatitis. All the handouts contained long sentences, many multisyllabic terms, and used medical jargon when describing diseases, methods of diagnosis, and medications. We note that these handouts do represent topics with multisyllabic or names with multiple words and could have been scored higher due to the way the Flesch-Kincaid Grade Level Formula and the SMOG Index include word counts and syllables in rating calculations.

Conversely, the handouts with the lowest readability scores offered practical advice on common topics. The 10 handouts with the lowest readability scored between 7.1 and 8.5. Topics included hypothyroidism, epilepsy, breeding, passing urine, lungworms, kennel cough, torn knee ligaments, Alabama rot, cat flu, and kidney disease. The 10 handouts used short sentences and described treatments and offered descriptions of diseases using plain language. Vetlexicon was the source for 9 of these 10 handouts. These findings align with the findings of Royal et al. (2018) and Sheats et al. (2019b). See Appendix for readability scores for each handout.

Sheats et al. (2019a) are right to encourage the American Veterinary Medical Association (AVMA) to adopt readability guidelines for written communication targeted at the 6th grade level. Attention to readability as part of improving health literacy in veterinary medicine is warranted. Perhaps by coming late to setting health literacy standards, veterinary medicine will be at an advantage. Recently, Ancker et al. (2019) cited a need to get away from the deficit model of health literacy, and suggest examining the “universal effective use of health information” and the roles of all the providers of the information in society, including libraries. They propose a new definition: “Health literacy occurs when a society provides accurate health information and services that people can easily find, understand, and use to inform their decisions and actions.”

Ultimately, the University of Tennessee Libraries did purchase a subscription to Vetlexicon. Vetlexicon handouts have a distinct readability advantage in comparison to Veterinary Partner and ClientEd. However, we cannot recommend Vetlexicon over the other handout collections, as it omits complete information about authors and the date of the last time the information was updated. ClientEd handouts include author name, degrees, any board certification(s) attained, and a copyright date. Veterinary Partner does the same and lists the source of the information, if it is not from VIN, plus two dates: the date published and the date reviewed or revised. As part of the trial of Vetlexicon, we inquired about plans for adding authors and dates on handouts. A Vetstream representative stated that they will be added sometime in the future. The Veterinary Partner attribution information is the most complete and a model for others. Another model for formatting and attribution are Extension publications from U.S. land-grant universities. For example, at the University of Tennessee, Extension Specialist Dr. Jennie Ivey launched a comprehensive website three years ago called UT Horse, which focuses on horse owners, 4-H, and the equine industry in Tennessee (https://ag.tennessee.edu/AnimalScience/UTHorse/Pages/Resources.aspx). Extension publications must meet established readability criteria before publication and present science-based information.

Limitations

One limitation of this study is that we removed the bullet point sections from text entered into the readability calculator. The bullet point sections were often not complete sentences and the readability calculator we used required complete sentences with no formatting in the text. Bullet points make a document easier to read. Removing these sections may have caused some handouts to receive higher scores than they would have with the bullet point sections included. While this is a possibility, our results do not show that higher scoring handouts were more likely to have bullet point sections omitted. When ranked from lowest to highest readability scores, the lower scoring half of handouts included 35 handouts with omitted bullet point sections, and the higher scoring half included 38 handouts with omitted bullet point sections.

Only English language handouts were assessed in this study. There are collections of handouts in other languages. We are aware of 170 handouts in Spanish for veterinary clients in North America included in ClientEd. The AVMA also has a collection of Spanish language materials (https://www.avma.org/PracticeManagement/ClientMaterials/Pages/spanishproducts.aspx). The Canadian Veterinary Association offers handouts in French (https://www.veterinairesaucanada.net/resources/animal-owners). Language barriers and culturally competent care were featured in the October 2019 JAVMA News (Mattson 2019), so offering materials in languages other than English is part of an effort in veterinary medicine to meet the animal health requirements, and health literacy needs and preferences of a larger percentage of clients.

Vetlexicon, Veterinary Partner, and ClientEd are the three largest collections of handouts that provide handouts for veterinary clients in English. This study contributes an analysis of readability levels, finding that most of the handouts from these services score above the recommended 6th grade reading level. Readability needs to be added to the workflows of authors of veterinary information and companies offering handouts. The three ready-to-use handout collections can advantage veterinary teams in educational institutions and private practices striving to offer science-based animal health education, counteract misinformation, and build relationships with clients. More readability assessments should be done in veterinary medicine to meet the health literacy needs of veterinary clients. Librarians who are evaluating clinical information services, or similar resources, should also consider conducting readability assessments of handouts to complement faculty appraisal of the information from the service and to critically examine potential additions to their collections. Conducting readability assessments is straightforward. Readability assessments offer a useful way to improve health literacy by meeting client information needs, with the potential to improve animal health and the quality of veterinary care.

We would like to thank Mike O’Neil, Statistician; Lauren Billman, Head of Subscriptions at Vetstream Ltd.; Lisa Mancuso, Managing Editor Vetstream Ltd.; Ruth Schoonover, VIN content Coordinator; Rhea Morgan, Editor-in-Chief VINcyclopedia; members of the Acquisitions and Continuing Resources Department at University of Tennessee Libraries; colleagues Chris Durman and Sandy Leach; and colleagues at Pendergrass Library.

Alpi KM, Stringer E, Devoe R. & Stoskopf M. 2009. Clinical and research searching on the wild side: exploring the veterinary literature. Journal of the Medical Library Association. 97(3):169-177. DOI: 10.3163/1536-5050.97.3.005.

Ancker J.S., Grossman L.V. & Benda N.C. 2019 Oct. Health literacy 2030: is it time to redefine the term? Journal of General Internal Medicine [Internet]. 1-4. DOI: 10.1007/s11606-019-05472-y.

[AVMA] American Veterinary Medical Association. 2020 Jan. CVTEA Accreditation Policies and Procedures - Appendix I . Available at https://www.avma.org/education/accreditation/programs/cvtea-accreditation-policies-and-procedures-appendix-i.

Boss N. 1999. Educating Your Clients from A to Z: What to Say and How to Say It. Lakewood (CO): AAHA Press.

Boss N. 2002. The Client Education Notebook: Customized Client Education Materials to Use in Your Own Practice. Lincoln (NE): AVLS-PetCom.

Boss N. 2003 Jun/Jul. Clients with bad information. Trends Magazine American Animal Hospital Association. 19:55-56.

Boss N. 2009. How We Do Things Here: Developing and Teaching Office-wide Protocols. American Animal Hospital Association Press.

Boss N. 2011. Educating Your Clients from A to Z: What to Say and How to Say It. 2nd ed. Lakewood (CO): AAHA Press.

Boss N. 2013a. Creating a client-centered hospital. In: Ackerman LJ, editor. Blackwell’s Five-minute Veterinary Practice Management Consult. 2nd ed. Ames (IA): Wiley. p. 70-71.

Boss N. 2013b. Client educations tools. In: Ackerman LJ, editor. Blackwell’s Five-minute Veterinary Practice Management Consult. 2nd ed. Ames (IA): Wiley. p. 362-365.

Catanzaro T.E. 2001. Promoting the Human-Animal Bond in Veterinary Practice. 1st ed. Ames (IA): Iowa State University Press.

Dean R., Brennan M., Baillie, S., Brearly J., Cripps P., Eisler M.C., Ewers R., Handel M., Holmes M., Hudson C., et al. 2017. The challenge of teaching undergraduates evidence-based veterinary medicine. Veterinary Record. 181(11):298-299. DOI: 10.1136/vr.j3441.

Dorman D.C., Alpi K.M. & Chappell K.H. 2013. Subject matter expert and public evaluations of a veterinary toxicology course brochure-writing assignment. Journal of Veterinary Medical Education. 40(1):19-28. DOI: 10.3138/jvme.0912.082R.

Gorman C. 2000. Clients, Pets and Vets: Communication and Management. Newbury (UK): Threshold Press.

Gray C. & Moffett J. 2010. Handbook of Veterinary Communication Skills. Ames (IA): Wiley-Blackwell.

Hare D. 2005. Health literacy. Canadian Veterinary Journal. 46(2):103-104. Available at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2831554/.

Kincaid J.P., Fishburne R.P., Rogers R.L. & Chissom B.S. 1975. Derivation of new readability formulas (automated readability index, fog count, and Flesch reading ease formula) for navy enlisted personnel. Memphis (TN): Naval Air Station. Research Branch Report 8-75.

Lee K., Hoti K., Hughes J.D. & Emmerton L. 2017. Dr. Google is here to stay but health care professionals are still valued: an analysis of health care consumers’ internet navigation support preferences. Journal of Medical Internet Research. 19(6): e210. DOI: 10.2196/jmir.7489.

Mattson K. 2019 Oct. Speak my language: veterinary medicine may face increasing challenges around language barriers [Internet]. JAVMA News. Available at https://www.avma.org/javma-news/2019-11-15/speak-my-language.

Mayer S.H. 2008. A librarian’s guide to providing resources for pet owners. Journal of the American Veterinary Medical Association. 232(10):1464-1467. DOI: 10.2460/javma.232.10.1464.

McLaughlin, G.H. 1969. SMOG grading: a new readability formula. Journal of Reading. 12(8):639-646.

Murphy S.A. 2006. Consumer health information for pet owners. Journal of the Medical Library Association. 94(2):151-158.

Royal K.D., Sheats M.K. & Kedrowicz A.A. 2018. Readability evaluations of veterinary client handouts and implications for patient care. Topics in Companion Animal Medicine. 33(2):58-61. DOI: 10.1053/j.tcam.2018.03.005.

Shaw J.R. & Hunter L.J. 2017. Communicating pet health information to veterinary clients. Clinician’s Brief. 15(5):37-40. Available at http://www.cliniciansbrief.com/article/communicating-pet-health-information-veterinary-clients.

Sheats M.K., Kedrowicz A. & Royal K. 2019a. A call for readability guidelines in veterinary medicine. Journal of the American Veterinary Medical Association. 254(1):46. DOI: 10.2460/javma.254.1.44.

Sheats M.K., Royal K. & Kedrowicz A. 2019b. Using readability software to enhance the health literacy of equine veterinary clients: an analysis of 17 American Association of Equine Practitioners’ newsletters and website articles. Equine Veterinary Journal. 51(4):552-555. DOI: 10.1111/evj.13042.

Sobolewski J., Bryan J.N., Duval D., O’Kell A., Tate D.J., Webb T. & Moore S. 2019. Readability of consent forms in veterinary clinical research. Journal of Veterinary Internal Medicine. 33(2):350-355. DOI: 10.1111/jvim.15462.

Stull J.W., Shelby J.A., Bonnett B.N., Block G., Budsberg S.C., Dean R.S., Dicks M.R., Forsgren B.W., Golab G.C., Hamil J.A., et al. 2018. Barriers and next steps to providing a spectrum of effective health care to companion animals. Journal of the American Veterinary Medical Association. 253(11):1386-1389. DOI: 10.2460/javma.253.11.1386.

Wang L.W., Miller M.J., Schmitt M.R. & Wen F.K. 2013. Assessing readability formula differences with written health information materials: Application, results, and recommendations. Research in Social & Administrative Pharmacy. 9(5):503-516. DOI: 10.1016/j.sapharm.2012.05.009.