Analysis of sex differences in pre-clinical and clinical data sets

Introduction

As more studies begin to address the role of sex as a biological variable (SABV), it has become increasingly important to understand how to collect and analyze the data so that the presence or absence of sex differences can be assessed. This has led to some concerns about how to conduct statistical analyses. In this brief commentary we do not attempt to review the field of sex differences, but provide a conceptual guide to the statistical analysis of sex differences in research with both animal and human subjects.

Status quo

The use of predominantly male subjects has been documented across myriad research domains—both historically and at present—with the extent of bias varying by subdiscipline [1, 2]. Females have historically been viewed as more variable than males because of the presence of estrous or menstrual cycles in many species, although this belief has been discredited across a wide range of animal models (reviewed in [3, 4]). Noting the potential negative consequences of subject sex bias for women’s health, the National Institutes of Health issued a 1993 act to include women in clinical research, followed by a 2014 policy to encourage balanced use of male and female subjects and tissues by considering SABV during research design and analysis. The mere inclusion of females, however, does not provide insight into the role of sex/gender in physiological, behavioral, and psychological traits, and the majority of studies using male and female subjects fail to report on whether sex differences are present [2, 4].

Types of sex differences

In order to analyze the role of SABV, it is important to understand that there are different types of sex differences, and that these may require different analysis strategies. These categories are not mutually exclusive—more than one type of sex difference may be involved in any given trait. Figure 1 provides a graphical representation of four types of sex differences, as well as guidance on conducting analyses based on the type of sex difference. Specifically, major sex differences that prevent the analysis of males and females on the same scale or metric (Fig. 1a) are qualitative sex differences. Qualitative differences are also present if variables are not related to each other in the same way across the sexes (Fig. 1b). In the case of qualitative sex differences, analyses should be conducted independently by sex and the data reported as though they are two independent experiments. When the average or mean of a dependent variable is different for males and females, these are quantitative sex differences (Fig. 1e) and analyses should be conducted with sex as a factor. There are also sex differences where one aspect of a trait is the same for males and females, but the mechanisms underlying the trait are different, or emerge only under certain conditions: these are latent sex differences (Fig. 1c; also referred to as mechanistic, convergent or divergent). Finally, when the proportions of males and females that exhibit particular traits in response to the independent variable are different, these are population sex differences (Fig. 1d). For latent and population sex differences, the types of analyses will vary based on the data. See the Fig. 1 legend for discussion. For additional examples and discussion of these different types of sex differences refer to [5]. It is important to note that many assessments were developed and validated using exclusively male samples, and when extending these tests to females it is not always clear whether they are operationalizing the same trait in females - a topic which requires further consideration [6].

Fig. 1

Flowchart of questions to ask, possible answers, and suggested analysis aproaches when examining empirical data from male and female subjects. Although informed by the broader literature, responses to questions should be made with respect to the particular data set being analyzed. If a sex difference is expected or present (“Yes”), then the nature of the difference will influence the type of analysis to be conducted. Qualitative differences, in which different traits are examined in the sexes (“A”) or in which the functional form of a relation varies by sex (“B”), require analyses to be conducted separately for males and females; that is, data should be analyzed as if the males and females are from two independent experiments. Quantitative differences, in which males and females show the same pattern of traits, but to different levels, extents, or degrees (“E”), indicate analyses that include sex as a variable of interest (e.g., using sex as a term in a factorial analysis). Latent differences mark sex differences in the mechanisms underlying a trait (“C”), and population differences reflect sex differences in the prevalence of different, non-overlapping traits (“D”). Both can be evaluated in analyses conducted separately by sex, or by including sex as a variable of interest in planned analyses; the decision depends upon the research question, and whether the specific difference of interest is qualitative or quantitative. Thus, a given behavior can reflect multiple types of sex differences. In addition to specifying the statistical significance of sex differences, it is also important to describe their direction and practical importance (e.g., effect size, such as Cohen’s d or percent difference). If it is unclear whether a sex difference is present or not (“Unsure or Unanticipated”), then sex should be included in all assumption checks performed on the data prior to conducting inferential analyses; this may include consideration of all types of differences (“A” thru “E”) with descriptive statistics (e.g., distributions), variance and normality evaluations by sex, and data visualizations (e.g., scatterplots). The sequence and nature of these checks may differ by data set, and may also be done prior to analyzing data in which sex differences are expected. Results can then guide the response to the initial question (i.e., “What can you do to make sure you do not miss an important difference?”). If still unsure whether there are sex differences, then sex can be included as a variable of interest in planned analyses, with the knowledge that analyses will only capture quantitative and some latent and population differences, or analyses may pool subjects across sex. When pooling subjects, effect size should be considered in addition to statistical power, as samples may be too small to detect differences even if they are present (although increases in sample size required to examine main and even interaction effects of sex are modest; see [5]). It may also be important to note when examinations of sex differences are incomplete (e.g., if all types of differences were not studied) or inconclusive (e.g., if power was limited to detect effects).We do not recommend controlling for gender or sex differences as these are non-random differences [8]. Other types of analyses may require examination of individual differences in addition to sex differences [9,10,11,12]. See Table 2 for more information.

Best practices

How should a researcher approach data analysis in light of potential sex differences of various types? While it is beyond the scope of this brief commentary to give prescriptive guidance for all research involving both males and females, we hope to provide some guiding concepts the reader can use when evaluating their data. First, one should consider what is already known about sex differences in the context of the study topic and consider the data that have been obtained (Fig. 1). For a process that only occurs in one sex or varies fundamentally by sex (i.e., qualitative differences), the analysis must proceed separately by sex or only in the sex in which the process occurs (e.g., specific reproductive functions). When sex should be included as a variable in analyses (i.e., if differences are quantitative) descriptions of the effect size or magnitude of any sex differences may help determine the practical significance of the differences, regardless of their statistical significance, and confirmation of the effect is essential [7]. In yet other cases, sex may not appear to be an important contributor to the final results; in these instances, the lack of sex differences should be documented and reported, and consideration should be given to the ability of the data set to identify sex differences (e.g., statistical power). Specific examples of how such analyses might proceed appear in Table 1. In general, we advocate for reporting as much data as feasible for each sex, analyzed in different ways if appropriate. This avoids selecting data that support a hypothesis or defaulting to the assumption that the differences between the sexes are quantitative. Responses to frequently asked questions concerning the analysis of sex differences also appear in Table 2.

Table 1 Sample analysis paths for different research scenarios concerning the study of sex differences

Table 2 Responses to frequently asked questions about the study of sex differences

Conclusions

Our goal has been to lay out a strategy for analysis of data collected from both female and male subjects. We emphasize that investigators should assess and consider the types of sex differences present in the data to guide analyses, and to ensure that meaningful results will be obtained.

Funding and disclosures

No funding sources were used in the creation of this commentary. The remaining authors have nothing to disclose.