Development and validation of a patient-reported measure of psychosocial variables in men with erectile dysfunction (ED) is described. Literature review, focus groups, and medical specialists identified 86 potential items. Redundant, ambiguous, or low item-to-total correlation items were removed. Data from 98 men reporting diagnosed ED and 94 controls assisted in final item selection and psychometric evaluation. Treatment responsiveness was evaluated in 93 men with ED in a 10-week open-label trial of sildenafil citrate (Viagra®). The 14 chosen items resolved into two domains: Sexual Relationship (eight items) and Confidence (six items), the latter comprising Self-Esteem (four items) and Overall Relationship (two items) subscales. The resulting Self-Esteem And Relationship (SEAR) questionnaire demonstrated validity and reliability. The intervention study demonstrated responsiveness to beneficial treatment with significant improvement in scores (P=0.0001). The SEAR questionnaire possesses strong psychometric properties that support its validity and reliability for measuring sexual relationship, confidence, and particularly self-esteem.
An estimated 30 million men in the United States are affected by erectile dysfunction (ED).1 In the Massachusetts Male Aging Study, 52% of the men aged 40–70 y had some degree of ED.1 ED is frequently comorbid with cardiovascular diseases1,2 and other medical conditions such as prostate disease and their associated treatments,3,4,5,6,7,8 diabetes mellitus,1,9 spinal cord injury,10 multiple sclerosis,11 end-stage renal disease,12 and depression.13,14
Sexual dysfunction compromises overall quality of life and is also associated with anxiety and loss of self-esteem.15,16,17,18 Sexual dysfunction can be especially damaging to self-esteem and can contribute to relationship difficulties; it is not surprising to learn that aspects of mental health or psychosocial functioning are diminished in men with ED. Treatment for ED, regardless of the specific therapy, is associated with improvement in sexual, relational, and emotional areas in patients with ED.19 For example, sildenafil citrate (Viagra®) users reported significant improvements in erectile and sexual function that were associated with positive changes in emotional well-being and sexual partner relationships.20,21
Some clinical trials of ED treatments have used generic instruments or questionnaires developed for other conditions.19 Relative to these measures, an ED-specific instrument is expected to be more responsive to changes following beneficial treatment for ED and thus be better suited to measure the clinical relevance of treating ED. To date, three instruments specifically address psychosocial elements related to ED: the Quality of Life in Male Erectile Dysfunction questionnaire (QoL-MED),22 Erectile Dysfunction Effect on Quality of Life instrument (ED-EQoL),23 and the Psychological Impact on Erectile Dysfunction (PIED).24 However, there remains a dearth of adequately validated instruments on components or dimensions of psychosocial functioning and well-being that are specific to ED. Therefore, we developed and validated a patient-reported instrument—the Self-Esteem And Relationship (SEAR) questionnaire—that, in addition to sexual relationships, encompasses related psychosocial factors specific to men with ED. Whereas a previous publication emphasized the treatment responsiveness of the SEAR questionnaire,25 the current article is intended to provide a more complete and detailed account of the psychometric properties of the instrument.
Common practices in the area of scale development were followed.26,27,28,29 In the first phase, an initial set of 86 items or statements concerning feelings and thoughts about ED, along with the corresponding life impact of ED for men with ED, was identified. This process of item generation included a search of the clinical literature on ED and quality-of-life scales, followed by focus groups of 35 men with ED (aged 40–75 y), 34 female partners of these men, and 27 physicians (10 internal medicine, 11 general or family practice, 6 urology) conducted in March 2000.
The objective of the second phase was to reduce the 86 candidate items to a manageable set of relevant items. In December 2000, a field survey of 150 men with ED (aged 18–74 y) was undertaken. The burden on respondents was alleviated by randomly giving each man half of the 86 questions and having responses to the remaining half imputed by the expectation maximization algorithm.30 The expectation maximization algorithm is typically used to compute maximum likelihood estimates with incomplete samples.
Items that were redundant, ambiguous, poorly worded, or had poor measurement properties based on standard procedures (corrected item-to-total correlations (ie, correlation that reflects the relationship of an item with its associated domain score after removing the item from that domain score), exploratory factor analysis, and item-level discriminant validity tests) were removed from further consideration. Response categories were also evaluated for consistency. Refinement and modification resulted in a questionnaire with 14 items (Table 1), which underwent a full psychometric validation.
The psychometric validation followed recommendations from published guidelines.27,28,29,31,32 In February 2001, an observation study was conducted with 98 respondents who reported a clinical diagnosis of ED in the past year (49 were treated with sildenafil at least once, 49 were not treated) and 94 age-matched controls who reported no clinical diagnosis of ED in the past year. The self-administered questionnaire was completed by subjects in private.
A factor analysis with promax rotation, which rotates the factors obliquely to allow for correlations among them, was conducted to identify the number and nature of the domains.33 A scree test was depicted to help determine the number of factors to extract.27,34,35,36 With the scree test, an eigenvalue (ie, the amount of variance that is accounted for by a given factor) associated with each factor is plotted to look for a ‘break’ between the factors with relatively large eigenvalues (factors that are important and retained) and those with small eigenvalues (factors unimportant and not retained). The next series of steps evaluated validity with corrected item-to-total correlations, item-level discriminant validity tests,37,38 convergent validity, divergent validity, and discriminant validity.39 Two generic questionnaires, the 22-item Psychological General Well-Being index (PGWB)40 and the 36-item Medical Outcomes Short-Form (SF-36),41 were included to help assess the convergent and divergent validity of the SEAR questionnaire.
Evidence for convergent validity was based on a Pearson's correlation of 0.40 or higher, consistent with a meaningful correlation.36 Evidence for divergent validity was based on a Pearson's correlation less than 0.30, consistent with less than a medium association.42 Correlations between 0.30 and 0.40 were taken as no evidence to dismiss either convergent validity or divergent validity. Discriminant validity of the SEAR questionnaire was based on a single self-assessment of ED severity (none, mild, moderate, severe) adapted from the Massachusetts Male Aging Study43 and self-reported group affiliation (clinically diagnosed ED or age-matched controls).
The reliability of the SEAR questionnaire was based on internal consistency44 and test–retest reliability with an intraclass correlation coefficient using a two-way random-effects analysis of variance model.45 For test–retest reliability, the assessment was based on the 90 ED subjects with measurements on both test and retest. The retest was taken about 2 weeks after the baseline test.
All questions except negatively worded questions 8 and 11 were scored as 1=almost never/never, 2=a few times (much less than half the time), 3=sometimes (about half the time), 4=most times (much more than half the time), and 5=almost always/always. Questions 8 and 11 were reverse scored, with 5=almost never/never, 4=a few times (much less than half the time), 3=sometimes (about half the time), 2=most times (much more than half the time), and 1=almost always/always. Thus, a higher score signified a more favorable response for all 14 items. The intent of not wording all items positively or all items negatively was to avoid a bias toward agreement, that is, a tendency of respondents to agree with items irrespective of their content.27
Domain (Sexual Relationship, items 1–8; Confidence, items 9–14), subscale (Self-Esteem, items 9–12; Overall Relationship, items 13 and 14), and overall or total scores (items 1–14) were computed by summing their respective items. Each domain score, subscale score, and overall score was transformed onto a 0–100 scale using the following equation:
Higher scores indicated a more favorable response (0=least favorable, 100=most favorable).
The second stage of the psychometric validation investigated responsiveness to change. The SEAR questionnaire was administered at baseline and end of treatment (week 10) in an open-label, multicenter, flexible-dose (initial 50 mg dose adjustable to 100 or 25 mg based on efficacy and tolerability) trial of sildenafil in sildenafil-naïve patients with clinically documented ED. All patients were in a stable single-partner relationship and provided informed consent. A previous report25 highlighted the responsiveness of the instrument to beneficial treatment with sildenafil based on changes from baseline for the entire sample and on the expected moderate correlations with the Erectile Function (EF) domain of the International Index of Erectile Function (IIEF).46 The current report further characterizes the responsiveness of the instrument by providing a descriptive profile of changes from baseline for responders (‘yes’) and nonresponders (‘no’) to two end-of-study global efficacy questions on whether erections and sexual intercourse were improved.
First validation study
In the first validation (observational) study, the 98 subjects who reported ED had a mean (standard deviation (SD)) age of 54.9 (11.4) y. Based on these 98 subjects, the factor analysis with the scree plot depicted a two-factor solution. The first two factors each had an eigenvalue greater than 1.0 and explained more than 10% of the variance in the data. In contrast, the third factor had an eigen-value less than 1.0 and explained less than 10% of the variance, so it and the remaining factors were discarded. Each item loaded onto exactly one factor with a factor loading of at least 0.4 on that factor and less than 0.4 on the other factor; there were no ambiguous items (Table 2).
Items on the first domain covered sexual relationship, whereas items on the second domain represented confidence. Hence, the first domain (items 1–8) was designated Sexual Relationship, and the second domain (items 9–14) was designated Confidence. The estimated correlation between the two domain scores was 0.64, indicating that the two domains were related yet distinct. The Confidence domain was, in turn, defined by items on self-esteem and on overall relationship; therefore, embedded within the Confidence domain were two specific subscales: Self-Esteem (items 9–12) and Overall Relationship (items 13 and 14). Table 3 provides descriptive statistics from 98 men with diagnosed ED who answered the questionnaire.
Each item had a corrected item-to-total correlation greater than 0.4 (Table 2). In terms of item-level discriminant validity tests, based on the item-to-total correlations, 11 of the 14 items were classified as successful in the sense that the correlation of each was significantly higher (P<0.05) with its corresponding domain score (which excludes the item) than with the other domain score (which includes the item). The remaining three items (items 4, 11, and 13) were classified as likely successes in the sense that the correlation of each tended higher with its corresponding domain score than with the other domain score.
Divergent validity (or at least no evidence to dismiss it) on the Sexual Relationship domain of the SEAR questionnaire was predicted and confirmed by its relatively low correlations with all domains on the PGWB and SF-36 (Table 4). For the Confidence domain of the SEAR questionnaire, divergent validity was predicted and confirmed by its relatively low correlations with physical factors of the SF-36 (Physical Functioning, Role-Physical, Bodily Pain, Physical Component Summary), and convergent validity (or at least no evidence to dismiss it) was predicted and confirmed with relatively moderate correlations with the SF-36 Mental Component Summary and Role-Emotional and Mental Health domains, as well as with the PGWB total score and Anxiety, Depressed Mood, Positive Well-Being, and Self-Control domains. No other predictions or hypotheses were made. Findings on item-to-total correlations, convergent validity, and divergent validity for the Confidence domain were reflected in its Self-Esteem subscale and, to a lesser extent, in its Overall Relationship subscale.
Regarding discriminant validity, the SEAR questionnaire discriminated well not only between men reporting diagnosed ED and age-matched controls (Figure 1) but also across levels of self-reported ED severity (Figure 2). Differences between mean scores for the groups with and without diagnosed ED were significant (P=0.0001). Mean scores across levels of ED severity differed significantly (P=0.0001) and, as expected, increased (ie, improved) linearly with decreases in severity (P=0.0001).
Cronbach's α values, a measure of internal consistency reliability, for the Sexual Relationship domain, the Confidence domain, and overall score were 0.91, 0.86, and 0.93, respectively. Cronbach's α values for the Self-Esteem and Overall Relationship subscales of the Confidence domain were 0.82 and 0.76, respectively. The intraclass correlation coefficients, a measure of test–retest reliability (reproducibility), for the Sexual Relationship domain, the Confidence domain, and the overall score were 0.78, 0.71, and 0.79, respectively. Intraclass correlations for the Self-Esteem and the Overall Relationship subscales were 0.72 and 0.57, respectively.
Second validation study
Of the 101 men with clinically documented ED who took sildenafil in the treatment intervention study, 96 (mean (SD) age, 55.5 (11.7) y; mean (SD) ED duration, 4.6 (7.5) y) completed at least one patient-reported assessment and were included in the intent-to-treat analysis. All patient-reported assessments were completed by 93 of the 96 patients. Consistent with changes among the entire sample, similar magnitudes of change were observed among those who reported an improvement in their erections and, separately, among those who reported an improvement in their sexual intercourse (Figure 3). Such magnitudes and direction of effect were lacking among those who reported that their erections did not improve and, separately, among those who reported that their sexual intercourse did not improve (Figure 3).
A previous report25 showed that changes in scores of both SEAR domains and the Self-Esteem subscale correlated moderately (r≥0.40, P=0.0001) with changes in scores on the EF domain of the IIEF; the correlation with the Overall Relationship subscale, although weaker (r=0.35), was nonetheless significant (P=0.0006). Moreover, changes in scores on item 15 of the IIEF (confidence in getting and keeping an erection), an item in the EF domain, correlated moderately with changes in scores on the Confidence domain (confidence in general) of the SEAR questionnaire (r=0.56, P=0.0001).
The development and validation of the SEAR questionnaire represents a substantial step toward establishing an improved measure of the emotional toll that ED can have on men. The analysis presented here reflects our initial expectation, based on the clinical literature and qualitative research from focus groups, that more than one domain would be necessary to quantify how complex traits like self-esteem may be affected by ED. Hence, a multiple-dimension solution was initially sought in the factor analysis. Four criteria were applied to judge interpretability of the results and all were met, including the number of factors retained.35 First, at least three items loaded on each retained factor. Second, the items that loaded on a given factor shared a common conceptual meaning. Third, the items that loaded on different factors measured different conceptual meanings. Finally, the rotated factor pattern demonstrated simple structure in that (1) items that had high pattern loadings (≥∣0.40∣) on one factor also had low loadings on the other factor, and (2) each factor had high loadings for some items and low loadings for the others.
While the factor structure of the SEAR questionnaire is statistically justified, we believe that the structure is consistent with face validity as well. For example, item 6, ‘I felt confident about performing sexually’, belongs to the Sexual Relationship domain because it concerns confidence targeted specifically to a sexual relationship. This domain covers aspects of sexual well-being and functioning, including that of confidence. On the other hand, the Confidence domain covers confidence in general, which pervades a man's overall disposition and relationship; therefore, it is related but not limited to a sexual relationship. Similarly, items having to do with relationship in the Confidence factor are not restricted to a sexual relationship. These items cover the overall relationship, including nonsexual aspects, and hence are best characterized as part of the Confidence factor rather than the Sexual Relationship factor.
The SEAR questionnaire provides a three-level assessment that includes an overall assessment, a domain-specific assessment (Sexual Relationship and Confidence domains), and an assessment of components (Self-Esteem and Overall Relationship subscales of the Confidence domain) of a domain. Many psychological scales contain a domain decomposed into specific subscales, which further aid in targeted interventions.47,48 For example, an emotional functioning domain might include subscales on anxiety and depression, and a cognitive functioning domain might include subscales on short- and long-term memory.
The data from the first validation (observational) study suggests that self-esteem and overall relationship, although conceptually discrete, are intertwined in defining confidence for men with ED. This interdependence is consistent with self-efficacy theory, which suggests confidence in the belief that one can perform a certain task or behave adequately in a given situation.49 For patients with ED, this confidence may manifest itself as behavioral elements in which positive thinking leads to increased self-esteem and a more satisfactory overall relationship.50
The content validity, concreteness, and psychometric evidence of the four items on the Self-Esteem subscale suggest that they adequately and specifically capture self-esteem as it relates to ED. For completeness, we also examined the psychometric characteristics of the two items addressing overall relationship, which help to define (along with self-esteem) the construct of confidence as it relates to ED. Although the data show that the Overall Relationship subscale alone can be a useful measure, its two broad questions and psychometric characteristics suggest that more research is needed to sufficiently and specifically measure an area as intricate as an overall relationship.
The data from the second validation (clinical trial) study indicated that, for samples of men with improved erections and men with improved sexual intercourse, ‘responsiveness’ of the SEAR questionnaire following treatment was generally high and robust. On the other hand, magnitudes of treatment effect were considerably smaller and even negative (deterioration) for samples of men without improved erections and men without improved sexual intercourse. While acknowledging the small sample sizes of nonresponders, we report descriptive differences between responders and nonresponders that are telling and in accord with expectations.
As part of the ongoing validation process, we encourage further evaluation of validity of the SEAR questionnaire with other relationship scales and with treatment satisfaction. The IIEF items focusing on sexual relationship, item 13 on satisfaction with overall sex life and item 14 on satisfaction with sexual relationship, constitute the Overall Satisfaction domain of the IIEF. With respect to SEAR components, a retrospective examination25 of this IIEF domain suggests that it correlated most highly (r=0.71) with the Sexual Relationship domain of the SEAR questionnaire, lending further support for the validity of the SEAR questionnaire.
The 14-item SEAR questionnaire evolved from a rigorous development and validation process. The instrument has an interpretable and multifaceted factor structure. Two domains appeared: Sexual Relationship (eight items) and Confidence (six items). The Confidence domain has two subscales: Self-Esteem (four items) and Overall Relationship (two items). The 14-item SEAR questionnaire possesses strong psychometric properties that support its validity and reliability for measuring sexual relationship, confidence, and self-esteem in men with ED.
This investigation confirms the importance of assessing the effect of ED on psychosocial well-being and functioning in addition to sexual functioning. The SEAR questionnaire acknowledges the multidimensional nature of ED and, in doing so, extends beyond sexual functioning to relational and emotional areas. Such a broad approach can offer more complete and accurate information to the healthcare community regarding the value of treatment for ED. We recommend its use in combination with a measure of sexual function, such as the IIEF, to help with treatment decisions in clinical and research settings.
We thank Sabina Gasper and Karen Macdonald for their helpful contributions during the early stages of instrument development. We also thank Joyce Healey, Janet Matsuura, and Peter Rittenhouse for their editorial assistance. In addition, we are grateful to Caryn Diuguid, Dale Glasser, Vera Stecher, and Michael Sweeney for their valuable contributions and support throughout the project.
Supported by Pfizer Inc.
About this article
BMC Urology (2014)