Introduction

Stroke is the second-leading cause of death and the third-leading cause of combined death and disability in the world1. In China, it is the leading cause of death and disability2. Stroke survivors often suffer from varying degrees of disability3. The 12-month disability rate was 16.6% for stroke survivors in China4. Walking dysfunction occurs in more than 80% of stroke survivors5. Consequently, gait impairments cause difficulties in performing activities of daily living and mobility—Post-stroke gait is characterised by a pronounced clinical presentation of gait asymmetry6,7. Strokes lead to long-lasting disability and have a social impact. The lives of survivors and their families are strongly influenced by these long-term consequences, including physical disability, cognitive disorders, difficulty in concentration or some other severe psychological problems8,9,10.

Stigma is an attribute that discredits a person, reducing them ‘from a whole and usual person to a tainted, discounted one’11. Health-related stigma refers to the stigmatisation of disease12. It is characterised by exclusion, rejection, blame or devaluation that results from experience, perception or reasonable anticipation of an adverse social judgment about a person or group13. In 2002, stigma was divided into two types: self- and public stigma14. Self-stigma is formed by the integration and internalisation of public stigma. When severe, it generates negative emotions such as shame, low self-esteem and even suicide. Public stigma is distinguished from self-stigma as the reaction that the general population has to people with disease, caused by stereotypes, prejudice, and discrimination14.

Goffman mentioned that stigma is a phenomenon that causes social devaluation or social suspicion due to a sign or attribute11. Patients of acquired immune deficiency syndrome (AIDS), mental illness, cancer and chronic diseases suffer public stigma because of the infectiousness, danger, lethality, attribution, and destructive nature of the disease15,16,17,18. For stroke patients, this attribute or sign is the disease itself. Perceptions of stigma have been identified as a concern among stroke patients. Patients sometimes feel avoided by others19. Work exploring stroke-related stigma has focused on stroke patients at different stages of their rehabilitation or looked at stroke as a chronic disease20,21.

There are some tools that examine public stigma, such as the Cancer Stigma Scale17, the AIDS Attitude Scale22, and the Community Attitudes to Mental Illness scale23. However, few studies have explored stroke-related stigma in the non-patient population as extension of perceived stigma sources from stroke patients19,24,25,26, although there are several reasons why this is important. The availability of stroke screening and prevention strategies (e.g. screening high-risk community populations and health education programs) means that people need to consider the possibility of a stroke diagnosis. However, fear of stigmatisation has been identified as a potential barrier to high-risk population screening, and to carry out public health education17.

This study aims to develop a scale measuring stroke-related stigma in non-patient populations. Being able to measure stroke-related public stigma would help identify the extent to which stigma exists, monitor changes in perceptions of stroke as a result of public health education and identify risk factors for more stigmatised beliefs.

Methods

Study design and sample

A cross-sectional descriptive study design was used to test and evaluate the validity and reliability of the public stigma of stroke scale (PSSS). Figure 1 shows the flow chart of instrument development and evaluation. We calculated sample size based on evidence from the literature. According to the calculation recommendation, the sample size should be 10 times the item size27. The sample size for confirmatory factor analysis (CFA) should be at least 20028. Therefore, the sample size of 730 was qualified—380 for exploratory factor analysis (EFA), and 350 for CFA.

Figure 1
figure 1

The instrument development and evaluation. PSSS, public stigma of stroke scale; EFA, exploratory factor analysis; CFA, confirmatory factor analysis.

Full size image

Participants were recruited from communities in China, using online and face-to-face method. Snowballing sampling was used for online data collection, and qualitative study. To ensure quality and completeness of the questionnaire, the duplication of responses was avoided by restricting IP addresses and ensuring that all questions were complete before submission. The inclusion criteria were as follows: (1) aged over 18 years; (2) have heard about stroke; (3) never been diagnosed with stroke; and (4) agreement to participant in the study. Individuals who (1) have been diagnosed with a stigmatised disease, such as cancer, AIDS, mental disease; and (2) experience obstacles in communication, reading, writing, and comprehension, were excluded.

Procedure

Item generation

Multiple steps were utilised to generate scale items. First, a multidimensional concept of stigma theory was employed29, which identified six components of health-related stigma, applicable to varying degrees depending on the illness of interest. The definition of stigma served as a guide for the development of items. The first three components, ‘concealability’ (whether an illness can be hidden from others); ‘aesthetics’ (described as a primitive response by the perceiver towards a non-concealable mark that makes the person less ‘pleasing to the eye’) and ‘disruptiveness’ (whether an illness disrupts usual interactions). These three components are related to stroke patients. Stroke leaves visible legacy symptoms, such as physical impairment and language disability, which are harmful for usual interactions, and makes an individual ‘unpleasing to the eye’. The fourth component is ‘origin’, which related to when and how the disease is believed to have come about. Perceived responsibility is a particularly relevant aspect, because when a patient is believed to have caused their disease, the associated stigma is greater29,30. Stroke patients’ bad habits, such as smoking and drinking problems, make this component relevant to stroke-related stigma. The last two components are ‘peril’ (relates to perceived danger from the stigmatised person; for example, some diseases are considered contagious or dangerous, such as HIV or Mental illnesses) and ‘course’ (refers to changes in the disease over time; for example, chronic and incurable diseases are more stigmatised). These six components highlight the aspects of a disease that may contribute to it being stigmatized.

Second, we performed a comprehensive literature search for health-related stigma, existing scales and relevant items. Traditionally, besides the aspects of a disease that may stigmatized, studies also considering behavioural aspects of stigma such as interpersonal avoidance and social distance, and attitudes towards discrimination such as employment law. Therefore, we developed items basis on previous research into health-related stigma17,20,22,23.

Third, to better structure the appropriate items, community residents (n = 28) were interviewed using semi-structured open-ended interviewing methods. Participants shared their views, experiences and attitudes when they come in contact with stroke patients. The interview outlines were: (a) Have you been in contact with a stroke patient? How were they? How did you feel during these specific experiences? (b) What do you think are the causes of stroke? (c) What do you think about the quality of life of stroke patients? (d) Do you know about the long-term disabilities caused by stroke? What is the impact of these disabilities? (e) How do you feel about stroke patients with disabilities? (f) Do you think stroke patients should be treated as different people? (g) Do you think stroke is caused by the patient’s own mistakes?

Content-based analysis was conducted; two researchers reviewed the transcript and independently coded the main themes to guarantee reliability and dependability. Three themes were generalised in the analysis: unconcealable appearance, stereotypes, and attitude towards stroke patients. The results of content-based analysis support the idea that not only the stigmatizing properties of the disease itself should be considered, but also stigmatizing behaviours and attitudes. To sum up the results of the qualitative study, the multidimensional concept of stigma theory and literature review, subsequently, 49 items were generated across five dimensions: perception of stroke, aesthetics, attributes, communication, and public policy. The first three dimensions were considered as the aspects that may be stigmatized by the disease itself, and the last two dimensions were considered as stigmatizing behaviours and attitudes respectively.

Testing version of PSSS

Expert consultation was adopted to revise the item pool and a pilot study was conducted to certify the testing version of PSSS.

Nursing professors with extensive experience in stroke and stigma, anthropologists, psychologists, and community experts who have experiences in related research, have a title of associate professor or above, were grouped into an expert panel (n = 7). Expert panel members rated each item’s relevance based on three criteria using a four-point Lynn scale (1 = not at all relevant; 2 = slightly relevant; 3 = fairly relevant; and 4 = very relevant)31: the appropriate item attribution; dimensional saturation; and item expression accuracy. The Item content validity index (I-CVI) and the Scale content validity index (S-CVI) were calculated32. For each item, the I-CVI was computed as the number of experts who provided a rating of 3 or 4, divided by the total number of experts, the proportion in agreement about relevance. We used two approaches to calculate S-CVI. First, to require universal agreement among experts, defining the S-CVI as the proportion of items on an instrument that achieved a rating of 3 or 4 by all content experts. Second, computing the I-CVI for each item on the scale, and then calculating the average I-CVI across items. An I-CVI of 0.78 or higher for three or more experts could be considered evidence of good content validity32. The universal agreement of S-CVI should be 0.80 or higher33, and the average S-CVI should be 0.90 or higher34.

Subsequently, in the first round of expert consultation, the CVI was calculated for each item. The results indicate that I-CVI of 12 items was under 0.78; universal S-CVI was 0.43 which should be 0.80 or higher; average S-CVI was 0.86 which should be 0.90 or higher; 1 item was suggested to be deleted due to repeated expression; 2 items (perception of stroke and communication) were suggested to be modified for not being specific enough; and 2 items were suggested to be added in the communication dimension. As for the dimension of attribute only remained 2 items which cannot support a single dimension, then the 2 items were contained in the dimension of perception of stroke based on the semantics. Thus, 38 items across 4 dimensions (perception of stroke, aesthetics, communication, and public policy) were retained. And then, we conducted second round expert consultation which contains 5 experts. The results indicate that I-CVI was 0.8–1; universal S-CVI was 0.92, average S-CVI was 0.98. Therefore, there was no revision for the second round expert consultation.

We conducted a pilot test to assess the test–retest reliability. Data was collected from 52 participants to complete the pilot test twice a week apart35. No items required modification.

Psychometric evaluation

Validity (face, content, construct) and reliability (McDonald’s Coefficient Omega, and test–retest) were evaluated. To assess the face validity, items were checked for wording, order arrangement, degree of understandability and difficulty with 12 community residents. No items required modification. Item analysis was associated with the pooled items among construct validity. In the construct validity process, the pooled potential items were reduced, and factors were extracted using EFA, and confirmed using independent cluster assumption (ICM) of CFA, bifactor-CFA (B-CFA), exploratory structural equation modelling (ESEM) and bifactor-ESEM (B-ESEM).

Measurements

Socio-demographic and health status

Community residents completed a self-administered questionnaire on age, sex, education level, profession, residential area, medical background, health status (such as hypertension, diabetes, heart diseases, cancer, mental illness), awareness about stroke, contact with stroke patients, and to understanding of stroke.

Testing version of PSSS

The PSSS was designed as a five-point Likert scale (1 = completely disagree, 2 = somewhat disagree, 3 = neither agree nor disagree, 4 = somewhat agree, and 5 = completely agree). There were 4 dimensions and 38 items in testing version of PSSS.

Ethical considerations

The study protocol was approved by the Ethics Committee of Guangdong Provincial Hospital of Chinese Medicine. Participation was voluntary, the informed consent was set at the top of the questionnaire, participants anonymously answered the questions that were consistent with their agreement to participate, and agreed to the publication of the results. The investigation conforms with the principles outlined in the Declaration of Helsinki (Br Med J 1964;ii:177).

Data analysis

All data analyses were conducted using SPSS version 24 and Mplus version 836. Descriptive statistics were used to summarise the data.

Reliability

The internal consistency reliability was tested using McDonald’s Coefficient Omega. A score above 0.80 was considered to be a satisfactory measure37. Test–retest reliability was measured using intraclass correlation coefficients. Reliability coefficients above 0.70 were considered satisfactory38.

Construct validity The Kaiser–Meyer–Olkin (KMO) test and Bartlett’s test of sphericity were used to ensure the adequacy of data for EFA. The Bartlett’s test of sphericity must be significant (p < 0.05) and the KMO value must be higher than 0.6039. Parallel analysis and Velicer’s minimum average partial (MAP)40 were employed to determine the number and stability of factor extraction41. A minimum squares with oblimin rotation was conducted42. Items loading at approximately 0.50 on one dimension and not showing cross-loading above 0.45 were retained for analysis in the model43.

ICM-CFA, B-CFA, ESEM, B-ESEM were performed using MLR44,45 to evaluate the fitness on the assumed theoretical dimensions of the PSSS. In ICM-CFA, each indicator is assumed to correspond to a single factor, and B-CFA explicitly accommodates psychometric multidimensionality in the indicators by relaxing the ICM-CFA. Assessing conformations of stratified tissues requires a bifactor model, whereas assessing conceptually adjacent conformations requires an ESEM. However, the bifactor model may express unmodeled cross-loadings through inflated G-factors, whereas the ESEM model may express unmodeled G-factors through inflated cross-loadings46. The goodness of fit criteria range was calculated using adjusted chi-square (χ2/df), comparative fit index (CFI > 0.95), Tucker-Lewis index (TLI > 0.95), root mean square error of approximation (RMSEA ≤ 0.06)47.

Ethical approval and consent to participate

The study protocol was approved by the Ethics Committee of Guangdong Provincial Hospital of Chinese Medicine. Participation was voluntary, the informed consent was set at the top of the questionnaire, participants anonymously answered the questions that were consistent with their agreement to participate, and agreed to the publication of the results.

Results

Demographics and health status

Participants included 730 individuals, over the age of 18, who had not been previously diagnosed with stroke or other well stigmatised diseases. Table 1 shows the participants’ demographic data. The mean age of the participants was 39.23 ± 14.59 years, ranging from 18 to 86 years.

Table 1 Participants’ demographics (N = 730).
Full size table

Construct validity

Exploratory factor analysis

The result of Bartlett’s test of sphericity was significant (χ2 = 8260.598, df = 406, p < 0.001) and the KMO value was 0.92. Parallel analysis and Velicer’s MAP showed that the four-factor model had stability of factor extraction (Supplemental Tables 1 and 2). EFA was undertaken using minimum squares with an oblimin to determine the underlying dimensions of the PSSS. Four factors were extracted, with loadings of 0.4 or greater (Table 2): factor 1, inherent ideology, refers to stigma generated by the stereotype; factor 2, aesthetic feelings, refers to a feeling generated from stroke patients’ appearance; factor 3, avoidance behaviour refers to stigmatized behaviour; and factor 4, policy attitudes refer to stigmatized attitudes. These four extracted factors explained 61.57% of the total variance in stroke-related public stigma. Therefore, 33 items in a four dimensions scale were generated.

Table 2 Exploratory factor analysis with the extraction of four factors (N = 380).
Full size table

Confirmatory factor analysis

ICM-CFA, B-CFA, ESEM and B-ESEM were performed to evaluate the fit on the assumed theoretical dimensions of the PSSS. The fit indices of the models were reported in Table 3. The results showed that ICM-CFA provided an unacceptable level of fitness (CFI < 0.90, TLI < 0.90), and ESEM provided a qualified CFI (0.903) but the TLI was less than 0.90. The fit indices of B-CFA and B-ESEM both showed an acceptable level of fitness but not excellent enough. So we modified the models based on modifications indices. The results suggested that the B-CFA provided the best fit to the data with CFI = 0.957, TLI = 0.950, RMSEA = 0.040. (Fig. 2). We performed 10 correlations between residuals on the theoretical basis, which were item 1 and item 2, item 2 and item 3, As a thinking of stroke is a lifelong disease such as disability, and then it will remind one feel that the stroke patients are unable to work anymore, and then they can no longer take care of themselves anymore; item 16 and item 17, item 17 and item 18, the stiff facial expression and curled up on the bed of stroke patients were disease related experiences and can make others feel uncomfortable, and then they may suffer with urinary tube etc.; item 20 and item 30, item 22 and item 23, item 23 and item 24, item 27 and item 28, when a person is reluctant to engage with a stroke patient, they will also be reluctant to see information about it on media. If a person does not have the patience to communicate with a stroke patient will also try to stay away from the stroke patient, so that in the end they cannot be friends with stroke patients, and the attributes of the family of the stroke patient and the caregiver of the stroke patient are related in that they both have close contact with the stroke patient. item 33 and item 34, item 34 and item 35, public facilities require economic investment, and public benefits and long-term care insurance are related. The parameter estimates from all models were presented in Table 4, and the correlation between factors were shown in supplementals Table 3.

Table 3 Goodness-of-fit statistics of the alternative measurement models.
Full size table
Figure 2
figure 2

Bifactor-CFA. The factor structure of the PSSS (N = 350). Model fit index: MLR chi-square χ2(df) = 706.597 (452); comparative fit index (CFI) = 0.957; Tucker-Lewis index (TLI) = 0.950; root mean square error of approximation (RMSEA) = 0.040. FG, Global factor from a bifactor model; FS1, inherent ideology; FS2, aesthetic feelings; FS3, avoidance behaviour; FS4, policy attitudes.

Full size image
Table 4 Standardized parameter estimates from the alternative measurement models.
Full size table

Reliability

McDonald’s omega coefficient revealed the high reliability of the PSSS (McDonald’s omega = 0.93). The subscales indicated good internal consistency, with McDonald’s omega values ranging from 0.87 to 0.95. Test–retest reliability coefficient of the total scale was 0.721.

Discussion

The present study developed the PSSS, aimed to assess stroke-related public stigma in non-patient populations. Four factors (inherent ideology, aesthetic feelings, avoidance behaviour, policy attitudes) were determined using EFA and B-CFA.

Factor 1, ‘inherent ideology,’ refers to the subjective view of non-patient populations that is generated by the stereotype of stroke patients. We designed items which indicated stroke as a chronic disease, deprivation of working ability and self-care ability, and allowing transposition thinking such as ‘I would feel embarrassed if I had a stroke,’ and ‘A stroke is worse than death.’ Stereotypes are especially efficient means of categorising information about social groups. They are considered ‘social’ because they represent collectively agreed upon notions of groups of persons. They are ‘efficient’ because people can quickly generate impressions and expectations of individuals who belong to a stereotyped group48. This conception is well studied in mental illness stigma14,16. The item ‘Stroke is a lifelong disease (such as disability)’ is similar to the item ‘getting cancer means having to mentally prepare oneself for death’ from the Cancer Stigma Scale17. Stroke survivors have much higher rates of hemiplegia, aphasia, dysphagia, and urinary incontinence compared with other chronic diseases. Cancer reminds of death, whereas stroke reminds of long-term disability.

Factor 2, ‘aesthetic feelings,’ implies public feelings towards visible, external manifestations among stroke patients. This factor stems from the ‘aesthetic’ dimension of Jones multidimensional concept of stigma29, described as a primitive response by the perceiver towards a non-concealable mark that makes the person less ‘pleasing to the eye.’ This is well confirmed in our qualitative analysis. Participants can easily describe external manifestations of stroke patients, such as tilted head, crooked mouth, and even urinary incontinence. The impression of stroke on the public is profound. However, existing tools examining stigma, fail to express this dimension17,22,23. Therefore, items in this dimension were generated based on qualitative data.

Factor 3, ‘avoidance behaviour,’ refers to a typical immediate response to public stigma; it is most prominent in social interaction. Stroke survivors stated that their friends or relatives become estranged after knowing the situation13. This is supported by our qualitative data. Participants indicated that they cannot always interact with patients and worried about causing unnecessary trouble to themselves. This may be explained through Kurzban and Leary’s theory26—human beings possess cognitive adaptations designed to cause them to avoid poor social exchange partners and join cooperative groups. This dimension is included in other assessment tools49,50,51. In the current study, items in this dimension involve a comprehensive social interaction category, including avoidance behaviours such as towards individuals, families, social interactions, and media.

Factor 4, ‘policy attitudes,’ refers to supportive attitudes of the public towards stroke patients. The Cancer Stigma Scale includes this dimension and has satisfactory reliability and validity17. Besides, the item ‘more tax money should be spent on the care and treatment of the mentally ill’ in the benevolence dimension of the Community Attitudes Toward the Mentally Ill scale, also indicated policy attitudes23. This study compiled positive reverse items in this dimension. Since society assumes that older adults suffer from stroke, except for treatment, support for older patients is limited. For example, the society believes that it is normal to retire at an old age; therefore, some insufficient supportive measures are taken for older stroke patients19,24. This dimension involves financial, public, social, and family support.

These four dimensions cover three aspects of stigma measuring: the disease itself, stigma behaviour and stigma attitude. PSSS comprehensively evaluated stroke-related public stigma. Our study developed practical and appropriate items through interviews and expert ratings; items indicated satisfactory content validity. We confirmed the four-factor model using EFA, ICM-CFA, B-CFA, ESEM and B-ESEM. Additionally, the B-CFA showed that the PSSS had a good fit index and confirmed the four-factor model. The B-CFA demonstrated that every item in the PSSS contributed to the domain of the scale; thus, the construct validity was adequate. Regarding reliability, PSSS has satisfactory internal consistency and test–retest reliability, indicating that it is consistent and stable across time.

Limitations

Although our study confirmed that the PSSS is a valid and reliable instrument to measure stroke-related public stigma, some limitations exist. First, the study included a convenient sample, although we do ensure a wide range of age in sampling stage. Second, although we tried to avoid including items that may set social expectations, we could not avoid this situation. Third, the current study assessed validity and reliability in a Chinese context, further studies would be needed to evaluate public stigma of stroke in other contexts. Fourth, the classical test theory has been criticised by scholars of modern measurement theory because of the “inherent flaws” of the mathematical model on which it is based, and there is an urgent need to test the PSSS against modern measurement theory, for example by using Item Response Theory to test the difficulty and discrimination of the PSSS items, which is another important direction for future scale development and application.

Conclusions

Stroke-related stigma could have a negative influence on both, patients, and non-patient populations. Our study demonstrated that the PSSS is an appropriate instrument to assess stroke-related stigma among non-patient populations. The availability of the PSSS would help assess levels of stroke-related public stigma and design interventions to decrease stigma.