Introduction

Male infertility, as described by the World Health Organization, is the “inability of a sexually active, non-contracepting couple to achieve pregnancy in 1 year” in which the male partner can be evaluated through various clinical and laboratory tests [1]. Estimates for the prevalence of male infertility range from 2.5 to 12% in literature [2]. Many studies have investigated possible factors driving regional variations in male fertility rates, including environmental risk factors, nutritional changes and socioeconomic differences [3,4,5]. Although various treatment options for male infertility exist, assisted reproductive technologies (ART) such as in vitro fertilization and intracytoplasmic sperm injection are emerging as popular interventions for couples encountering fertility difficulties [6, 7]. While ART plays an important role in the management of infertility, it is critical that patients are vigilant about the spectrum of diagnostic and therapeutic interventions that are available to them in this setting. Of note, a previous study assessing patient videos focused on infertility found that most of the videos focused exclusively on the positive aspects of ART and failed to describe the true realities and challenges of undergoing treatment [8].

For physicians and non-physicians alike, the internet is an easily accessible tool for gathering health information. In the setting of stigmatized topics such as sexual health, the internet becomes an even more accessible resource that has often been used to drive awareness among the general public [9, 10]. YouTube is the most popular internet video platform, having more than 1 billion users who watch more than a billions hours of video collectively every day [11]. A recent study showed that #MaleInfertility on Twitter, another social media platform, often referenced YouTube Videos (YTV), thereby highlighting that online traffic is driven to YouTube [12]. Although an abundance of health-related videos are uploaded daily to YouTube, the lack of restrictions on what content is made available potentially leads to considerable variability in the quality of information between videos. Given the increasing role of online content in influencing healthcare decision making, physicians should be aware of the quality of information that patients are exposed to prior to presenting to a healthcare practitioner’s office. In this study, YTVs focused on male infertility were evaluated to assess information quality on YouTube and to highlight the presented themes, with the goal of identifying high-quality content that reliably facilitates care.

Methods

On October 8, 2018, the top 50 YTVs were identified after using YouTube’s search algorithm to sort by relevance using the keyword “male infertility”. The YouTube search was carried out anonymously to minimize bias from user profiling techniques. Bias was further mitigated by using private browsing modes for all online searches. In addition, uBlock Origin (Quebec, Canada, https://github.com/gorhill/uBlock), a third-party software that prevents query tracking, was installed. Two investigators extracted YTV features (including duration, date of upload, likes, dislikes, and views), classified creators and assessed YTVs via a standardized checklist (Fig. 1). Major disputes were addressed as discussed by a third reviewer who would also independently review the video. Non-English videos were excluded from analysis.

Fig. 1: Checklist used to assess YTVs.
figure 1

A checklist was created based on AUA guidelines to assess the quality of YTVs uploaded by different sources.

Full size image

The 7 categories used to assess the quality of YTVs were as follows: (1) definition of infertility, (2) importance of history in male infertility, (3) role of physical exams and physical exam findings, (4) diagnostic tests, (5) nonsurgical interventions, (6) surgical interventions, and (7) ART. These categories were adapted from American Urological Association (AUA) guidelines and addressed pathophysiology, evaluation, and management of male infertility [13]. Each video was assessed by two independent raters for including discussion of specific components within each checklist category and given one point for each specific detail discussed. As the video was watched, each reviewer determined whether the YTV contained information relevant to each checklist category. Inter-rater reliability was determined via calculation of the kappa statistic [14]. The density of information was then calculated by dividing the total number of points assigned to the video by the total number of minutes in the video. Content creators producing these videos were classified using definitions employed in prior internet-based research focused on male infertility [12]. A Kruskal–Wallis test was employed to compare the difference in density by content creator.

YTVs were then ranked based on the checklist scores and stratified by content and creator. Grades were assigned based on density of information in the YTV, with a grade of “A” corresponding to the highest information density and grade of “D” corresponding to the lowest information density. Continuous data were reported as means, and the analysis of variance (ANOVA) model was used to compare the data using the R Programming Language 3.5.0 (https://cran.r-project.org/).

Results

A total of 50 YTVs were initially identified via the keyword “male infertility,” and 8 non-English videos were excluded from further analysis. Of the 42 YTVs that were analyzed, 55% (23/42) were created by healthcare organizations and 17% (7/42) were created by healthcare practitioners (HCPs). The remaining 28% (12/42) were uploaded by patients, individual caretakers, and “advocates”, or individuals with no discernible connection to male infertility (Fig. 2).

Fig. 2: YTVs by content creator.
figure 2

Characteristics of male infertility videos uploaded to YouTube are categorized by different content creators.

Full size image

Grades assigned to YTVs based upon aggregated checklist scores were stratified based on YTV features and content creator type (Fig. 3). The kappa statistic in our study was 0.91, indicating strong agreement between raters [15]. The difference in video lengths was statistically significant in comparing different grades; higher grades were associated with shorter duration YTVs (p = 0.0305). Grade A videos were mean ± SD of 2.1 ± 0.7 min long, Grade B videos were 5.9 ± 4.5, Grade C 8.4 ± 4.7, and Grade D 14.5 ± 14.7 min long. Days on YouTube (p = 0.73), number of likes (p = 0.53), number of dislikes (p = 0.74), and total number of views (p = 0.67) were all statistically insignificant between grades. While statistically insignificant, of note is that Grade D YTVs had the highest number of likes and total number of views compared with Grade A, B, and C videos. Of videos created by healthcare organizations and HCPs, 13% and 14%, respectively, were given an A Grade compared with 0% of YTVs created by patients, caretakers, and advocates. The majority of videos (90%) were of low quality regardless of the source. A Kruskal–Wallis test found the difference in information density to be significant (p = 0.00473) between the various content creators, with healthcare organizations having the highest density videos, although the variability in the density of videos was also the highest (Supplementary Fig. 1).

Fig. 3: Content found in YTVs.
figure 3

The distribution of content that can be found in the YTVs as determined by the standardized checklist is shown.

Full size image

Of the 42 assessed YTVs, 31% (13/42) explicitly defined infertility as an inability to conceive after 12 months of unprotected intercourse. Forty-eight percent of YTVs (20/42) indicated that infertility is exclusively a male issue, while 45% (19/42) indicated that both partners were contributors. Of the 42 YTVs, 90% (38/42) discussed how male infertility was assessed, including history taking (25/38), the physical exam (18/38), and diagnostic tests such as semen analysis (38/38). In contrast, only 71% (30/42) of YTVs discussed management of male infertility including nonsurgical interventions (21/30), surgical treatments (14/30), and ART (13/30). Figure 4 provides a detailed breakdown of content covered in the top YTVs as outlined in the checklist.

Fig. 4: Content found in YTVs by creator.
figure 4

The distribution of content is further stratified by the type of content creator.

Full size image

Upon stratifying YTV content by creator type, 30% (7/23) of healthcare organizations discussed ARTs compared with 14% (1/7) of HCPs and 42% (5/12) of non-healthcare-related individuals and patients. More specifically, as shown in Fig. 4, which stratifies the creator type even further within the three categories, none of the videos created by physicians addressed ARTs, although 100% of the physician-created YTVs defined infertility and discussed various diagnostic methods for male infertility. Of the 42 videos that were assessed, patients had created 10 of these. However, only 10% (1/10) of patient-created videos addressed physical exam findings or surgical interventions.

Discussion

The present study highlights that the quality of YTVs on male infertility widely differs even among the top 50 videos as algorithmically prioritized by the YouTube platform. In order to quantitatively assess differences in content between YTVs, a framework to evaluate the quality of YTVs using the density of information was created using AUA guidelines. Scores for each video were then compared against what YTV viewers might see as a marker for quality, such as number of likes, dislikes, views, date uploaded and length of videos. Higher content density videos were more likely to contain information that was considered important in the evaluation and treatment of male infertility according to the AUA male infertility guideline. In our analysis, shorter YTVs were associated with higher content density. In comparison, the total number of days on YouTube, the number of likes and dislikes, and the total number of views had no association with video quality. These metrics are unlikely to identify useful male infertility resources for both healthcare practitioners and patients aiming to identify valuable videos on YouTube. Rather, these typical metrics utilized on the YouTube platform may be misleading, as they are likely informed by users who lack male infertility expertise. There is a need for higher quality content that fully incorporates the AUA guidelines for healthcare providers to safely recommended to patients.

When recommending YTVs to patients as a resource for more information, our analysis did highlight that there were differences in the density of information in the YTVs depending on the content creator, with the most dense YTVs created by healthcare organizations, HCPs, and lastly, individuals. While most videos defined male infertility and discussed possible diagnostic tests, less common topics that were covered in the videos included physical exam findings, surgical interventions and ARTs. More specifically, YTVs created by patients only addressed physical exam findings 17% (2/12) of the time compared with 53% (16/30) of the time for HCPs and organizations. This distinction is notable given the fact that a dedicated physical exam can help identify a range of correctable forms of male infertility [16]. Patients who exclusively employ YTVs created by patients may therefore potentially devalue or overlook the need for a thorough evaluation of the male partner by a specialist in male reproductive medicine.

Patients were also much less likely to discuss surgical treatment options (8%) than nonsurgical options (50%) in their YTVs. This is in comparison to HCPs and organizations who discussed surgical options 43% of the time. Varicocele repair is a cost-effective treatment that is covered by health insurance policies and can improve both natural conception and ART success rates [17]. Yet, despite its success, surgical options were not as often discussed in patient-created YTVs. Given the differences in the quality and quantity of information provided in the top 50 YTVs, additional studies are warranted to determine what drives the popularity of videos so that higher quality content that concurrently appeals to audiences can be developed.

There are certain limitations of our analysis that should be discussed. This study assessed the quality of YTVs on male infertility using an information density calculation. Although this allows for a normalized method of evaluating the content in a video, this approach does potentially penalize longer videos that may discuss each topic in more depth than shorter videos. While we assessed the factual components included in the YTVs, there may be additional emotional and psychological benefits of YTVs that are not reflected in our analysis. Although patient-created YTVs may not fully explore the surgical treatment options of male infertility, they may offer additional benefits to viewers. Given that the videos created by healthcare providers did not have more views nor likes compared with YTVs created by individuals, further studies that explore the emotional and psychological benefits of YTVs may provide insight into how healthcare providers may create content that is more appealing to viewers. One method that healthcare providers and organizations can create content that is both informative and educational, yet still relevant, may be to include a component of patient interviews and testimonials into their videos so that they capture all aspects of male infertility from both a patient and provider perspective.

In addition, given that YouTube is a constantly evolving website, the selection of the top 50 videos at a single time point may not accurately reflect what patients are viewing subsequent to the time of this initial search. There may be other factors that may influence YouTube’s search algorithm, such as location and prior searches that may also determine which videos are shown. As such, the top 50 videos selected identified in this study might not be reflective of what an individual user might encounter when employing the male infertility keyword on YouTube. Finally, non-English language videos were excluded, further decreasing the generalizability of our findings.

Conclusions

Various content creators with differing healthcare backgrounds have adopted YouTube as a platform to discuss male infertility. Given the popularity of YouTube as an educational platform, healthcare professionals should be vigilant of the influence YouTube may have on patient awareness and understanding of the condition. Knowledge gaps identified in YTVs such as limited discussion about the timeline for receiving a male infertility diagnosis can help improve patient counseling and enable practitioners to direct patients to accurate and reliable sources of information for the condition on YouTube. There is much need for high-quality educational videos that address all aspects of male infertility to address the current gaps.