Introduction

Testosterone therapy (TTh) is a burgeoning and lucrative aspect of men’s health care. Considering this, TTh is a relatively unknown and misunderstood therapy shrouded in misconception, anecdotes, opaque guidelines, and confusing online information. Enough so that existing literature illustrated gaps that exist in patients’ knowledge of testosterone therapy [1]. With up to 93% of American adults engaging the internet for health care inquiries, in addition to the explosive interest and growth in TTh, there is a significant need to acknowledge as well as better understand the quality and readability of existing TTh content [2].

Multiple studies demonstrated the prevalence of limited health literacy in the US. To combat this, the Plain Writing Act was passed in 2010 requiring federal agencies to use clear communication within health-related materials to increase the public’s understanding [3]. While impactful, this act however only extended within the federal realm and has no foothold within the public or online domain. Given that patients have immediate accessibility to online search engines, patients can research any topic of interest from the comfort and privacy of their own space. This is especially true in the setting of and interest regarding testosterone insufficiency and TTh. The diagnosis of testosterone insufficiency is often riddled with nuances which left un-check can be compounded by the increased reliance on self-discovery and online investigation outside of the classic patient-physician encounter. One landmark study in this exact space published over 5 years ago by McBride et al. illustrated how online sources for TTh were often authored above the reading comprehension of the very patients designed to reach [4].

Our aim with this study was to expound on these results by McBride et al. and evaluate more contemporary readability of a larger assortment of online resources available to patients regarding TTh. Our hypothesis was that after 5 years of acknowledgement of poor readability we would identify improvement with resulting TTh readability.

Materials and methods

We performed two separate Google searches using the terms “Testosterone Therapy” and “Testosterone Replacement”. The first 50 unique search results for each term were included for analysis. Duplicate sites identified within both search results were eliminated. No explicit exclusion criteria were included so that we would best replicate the true 50 first search results patients would encounter. Search links denoted with the “Ad” prefix were included for analysis. Primary text from each link was then copied into a blank Microsoft Word (Microsoft Corp) document where extraneous content was removed such as authorship details, illustrations, and figure captions, website specific content, hyperlinks, and additional resources. The resulting cleaned text for each link was then analyzed using validated English language and readability instruments. Our search process and review occurred over a 3-month time frame, between February and April 2022.

Reading ease was evaluated using the validated Flesch Reading Ease Score (FRES) while general readability was evaluated with five grade-level assessments: the Flesch Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Simple measure of gobbledygook (SMOG), Coleman-Liau Index (CLI) and Automated Readability Index (ARI). FRES is an instrument that evaluates sentence length and word counts and produces a score between 0 and 100 with 0–30 being the most difficult to understand increasing in increments of 10 up to 90–100 which is very easily understood. FKGL, GFI, SMOG, CLI, and ARI are indexes that utilize sentence structure, word length, and syllables to produce an estimated grade level for understanding and have been validated for use in similar academic and patient-focused materials. Word count for each text document was also recorded.

Each unique link was assigned to one of 4 categories for further grouping and analysis based on metrics such as their source, content, and authorship. These categories included “Academic”, “Commercial”, “Institutional”, or “Patient Support”. Two independent reviewers performed this categorization with Cohen Kappa correlation score of 0.851 (near perfect agreement). These categories were subsequently evaluated and compared using one-way analysis through Microsoft Excel (Microsoft Corp). Abbreviated excerpt of resulting content data extraction table is illustrated in Table 1.

Table 1 Abbreviated excerpt of the evaluation of academic sources with readability tools.
Full size table

Results

Of the 100 websites identified and assessed via Google search, only 77 unique sources met inclusion for analysis (Table 2). Twenty-seven (35%) were identified as patient support focused, 21 (27%) academic centric, 18 (23%) from health care institutions and 11 (14%) from commercial interests (Table 2). The average reading ease score was 36.8 (ranging between 25.9–41.6), indicative of difficult-to-read material overall (Table 2). The average grade level required for understanding any included online material was 14 (equivalent to a college sophomore) (Table 3). Academic sources required the highest estimated grade level for understanding at 16 (college senior); commercial, institutional, and patient support sources neared 13 (college freshman) (Table 3). This is an average of 6.5 grade levels above the relative average grade level of reading ability of adults within the U.S. Reading score averages are illustrated in Fig. 1.

Table 2 Breakdown of online sources by category with word count and reading ease score.
Full size table
Table 3 Readability score averages for each source category.
Full size table
Fig. 1: Source category and resultant average grade level readability.
figure 1

This figure represents the average grade level readability for each source category. All four source categories were found to require an average grade level of at least college freshman for understanding.

Full size image

Academic sources, averaging 2671 words per document, utilized 296 and 228% more words compared to commercial (902 words) and both institutional (1172) and patient support sources (1172), respectively (Table 3).

Discussion

Testosterone therapy is an increasingly popular and highly marketable industry amongst male patients, so much so that testing and treatment with TTh has increased fourfold in the last 20 years [5]. This growth in interest is reflected in the steady increase in Google searches for testosterone replacement therapy over the last decade [6]. Therefore, it is essential that providers familiarize themselves with the various TTh resources available in order to tailor their patient counseling and guide them toward credible and coherent PEMs if additional information is needed.

As shown by existing Health Information National Trends Survey (HINTS) data, patients are increasingly relying on the Internet to seek health information [7]. This behavior has inevitably increased during the coronavirus pandemic likely due to a surge in digitalization and information-gathering [8]. In addition, as highlighted by Beck et al., online health information could potentially influence health behaviors, especially in young adults [9]. This highlights the increasing need for quality and readability of online patient education materials. Health literacy is critical for patients, in fact, poor health literacy is associated with worse health outcomes and increased health care costs [10, 11]. Despite this, the information presented to patients online have been widely proven to be above the average health literacy level for Americans [12].

Our study demonstrates similar concerns regarding TTh where all included sources are estimated to be at the college level, with academic sources requiring the highest level of understanding (at or near the level of a college senior). Literature within Urology as well as other medical fields including colorectal surgery, anesthesiology, and ophthalmology have shown PEMs above the advised reading level [13,14,15,16]. McBride’s work in 2016 demonstrated this clearly with search results online [3]. Unfortunately, our results demonstrate that despite advances in medical therapy, and published findings of advanced readability, patient information has not been revised to be more readable for the average patient. This suggests that many patients focused medical materials that we continue to create and disseminate are more advanced than the recommended level of understanding and not as helpful as we may intend [17]. Patients unable to comprehend resources designed for them are therefore left to their own research which we can demonstrate may not be helpful at improving comprehension or understanding of complex medical issues, especially TTh.

To ensure that all web-based PEMs are coherent, straight-forward, and easily accessible would be impossible given the vast number of sources and the dynamic nature of the internet. Even in the academic webspace, this would require a multidisciplinary approach, extensive reviewing, and significant time and dedication. However, this presents an opportunity for improved authorship with audience appropriate documents and information. Editing and or authoring documents in a way so that they are as easily accessible and understandable to a wide audience is likely to bridge the gap of misunderstanding and misinformation best. This can be achieved through multiple strategies including but not limited to the elimination of jargon, use of simple phrasing and reducing overall sentence or paragraph length.

This study has certain limitations despite using multiple different assessment methods. Usage of Google as a primary search engine fails to accurately capture results from patients’ searches on alternative search engines or trusted websites. Google was chosen because it comprises the highest web usage amongst the general population and therefore reasonably illustrates what the majority of the patient population encounters. To objectively evaluate TTh search results, readability tools were used to analyze the text and provide the “grade level” that is needed for someone to engage with the material [18]. These tools are not without fault, as they can only evaluate based on quantitative characteristics of the text, like word count, number of syllables, or sentence length. For example, a shorter word such as a medical jargon may be more difficult to understand than a longer but simpler word. Therefore, a more comprehensive review of readability of patient education materials should consider the medical jargon used as well as account for images and content organization [19]. In addition, readability does not correlate with comprehension.

Conclusions

The most immediate online sources for TTh information grossly exceed the average reading level and understanding of most adults within the U.S. Within the growing realm of TTh, more attention should be taken to publish accurate, accessible, and readable material to improve patient health literacy and empower patients.