Development and psychometric evaluation of public stigma of stroke scale (PSSS)

Stroke patients suffer from public stigma because strokes cause visible disability and heavy social burden. However, existing tools measuring stroke-related stigma do not consider public stigma. The aim of this study was to develop and evaluate a public stigma of stroke scale (PSSS). This cross-sectional study recruited 730 participants, aged above 18 years, with no diagnosis of stroke before. Scale items were generated after reviewing relevant literature and conducting interviews. An expert panel evaluated the validity and reliability of a preliminary scale. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), bifactor CFA (B-CFA), Exploratory structural equation modelling (ESEM), bifactor-ESEM (B-ESEM) were performed to extract factors and evaluate fit on the factor structures. The Omega coefficient was 0.93, and the test–retest reliability coefficient was 0.721. The EFA extracted four factors: inherent ideology, aesthetic feelings, avoidance behaviour, and policy attitudes. These explained 61.57% of the total variance in the data. The four-factor model was confirmed by B-CFA, and met the fitness criteria. The PSSS yields satisfactory psychometric properties and can be used to assess stroke-related public stigma.

Procedure. Item generation. Multiple steps were utilised to generate scale items. First, a multidimensional concept of stigma theory was employed 29 , which identified six components of health-related stigma, applicable to varying degrees depending on the illness of interest. The definition of stigma served as a guide for the development of items. The first three components, 'concealability' (whether an illness can be hidden from others); 'aesthetics' (described as a primitive response by the perceiver towards a non-concealable mark that makes the person less 'pleasing to the eye') and 'disruptiveness' (whether an illness disrupts usual interactions). These three components are related to stroke patients. Stroke leaves visible legacy symptoms, such as physical impairment and language disability, which are harmful for usual interactions, and makes an individual 'unpleasing to the eye' . The fourth component is 'origin' , which related to when and how the disease is believed to have come about. Perceived responsibility is a particularly relevant aspect, because when a patient is believed to have caused their disease, the associated stigma is greater 29,30 . Stroke patients' bad habits, such as smoking and drinking problems, make this component relevant to stroke-related stigma. The last two components are 'peril' (relates to perceived danger from the stigmatised person; for example, some diseases are considered contagious or dangerous, such as HIV or Mental illnesses) and 'course' (refers to changes in the disease over time; for example, chronic and incurable diseases are more stigmatised). These six components highlight the aspects of a disease that may contribute to it being stigmatized.
Second, we performed a comprehensive literature search for health-related stigma, existing scales and relevant items. Traditionally, besides the aspects of a disease that may stigmatized, studies also considering behavioural aspects of stigma such as interpersonal avoidance and social distance, and attitudes towards discrimination such as employment law. Therefore, we developed items basis on previous research into health-related stigma 17,20,22,23 .
Third, to better structure the appropriate items, community residents (n = 28) were interviewed using semistructured open-ended interviewing methods. Participants shared their views, experiences and attitudes when they come in contact with stroke patients. The interview outlines were: (a) Have you been in contact with a stroke patient? How were they? How did you feel during these specific experiences? (b) What  www.nature.com/scientificreports/ Content-based analysis was conducted; two researchers reviewed the transcript and independently coded the main themes to guarantee reliability and dependability. Three themes were generalised in the analysis: unconcealable appearance, stereotypes, and attitude towards stroke patients. The results of content-based analysis support the idea that not only the stigmatizing properties of the disease itself should be considered, but also stigmatizing behaviours and attitudes. To sum up the results of the qualitative study, the multidimensional concept of stigma theory and literature review, subsequently, 49 items were generated across five dimensions: perception of stroke, aesthetics, attributes, communication, and public policy. The first three dimensions were considered as the aspects that may be stigmatized by the disease itself, and the last two dimensions were considered as stigmatizing behaviours and attitudes respectively.
Testing version of PSSS. Expert consultation was adopted to revise the item pool and a pilot study was conducted to certify the testing version of PSSS.
Nursing professors with extensive experience in stroke and stigma, anthropologists, psychologists, and community experts who have experiences in related research, have a title of associate professor or above, were grouped into an expert panel (n = 7). Expert panel members rated each item's relevance based on three criteria using a four-point Lynn scale (1 = not at all relevant; 2 = slightly relevant; 3 = fairly relevant; and 4 = very relevant) 31 : the appropriate item attribution; dimensional saturation; and item expression accuracy. The Item content validity index (I-CVI) and the Scale content validity index (S-CVI) were calculated 32 . For each item, the I-CVI was computed as the number of experts who provided a rating of 3 or 4, divided by the total number of experts, the proportion in agreement about relevance. We used two approaches to calculate S-CVI. First, to require universal agreement among experts, defining the S-CVI as the proportion of items on an instrument that achieved a rating of 3 or 4 by all content experts. Second, computing the I-CVI for each item on the scale, and then calculating the average I-CVI across items. An I-CVI of 0.78 or higher for three or more experts could www.nature.com/scientificreports/ be considered evidence of good content validity 32 . The universal agreement of S-CVI should be 0.80 or higher 33 , and the average S-CVI should be 0.90 or higher 34 . Subsequently, in the first round of expert consultation, the CVI was calculated for each item. The results indicate that I-CVI of 12 items was under 0.78; universal S-CVI was 0.43 which should be 0.80 or higher; average S-CVI was 0.86 which should be 0.90 or higher; 1 item was suggested to be deleted due to repeated expression; 2 items (perception of stroke and communication) were suggested to be modified for not being specific enough; and 2 items were suggested to be added in the communication dimension. As for the dimension of attribute only remained 2 items which cannot support a single dimension, then the 2 items were contained in the dimension of perception of stroke based on the semantics. Thus, 38 items across 4 dimensions (perception of stroke, aesthetics, communication, and public policy) were retained. And then, we conducted second round expert consultation which contains 5 experts. The results indicate that I-CVI was 0.8-1; universal S-CVI was 0.92, average S-CVI was 0.98. Therefore, there was no revision for the second round expert consultation.
We conducted a pilot test to assess the test-retest reliability. Data was collected from 52 participants to complete the pilot test twice a week apart 35 . No items required modification.
Psychometric evaluation. Validity (face, content, construct) and reliability (McDonald's Coefficient Omega, and test-retest) were evaluated. To assess the face validity, items were checked for wording, order arrangement, degree of understandability and difficulty with 12 community residents. No items required modification. Item analysis was associated with the pooled items among construct validity. In the construct validity process, the pooled potential items were reduced, and factors were extracted using EFA, and confirmed using independent cluster assumption (ICM) of CFA, bifactor-CFA (B-CFA), exploratory structural equation modelling (ESEM) and bifactor-ESEM (B-ESEM).
Measurements. Socio-demographic and health status. Community residents completed a selfadministered questionnaire on age, sex, education level, profession, residential area, medical background, health status (such as hypertension, diabetes, heart diseases, cancer, mental illness), awareness about stroke, contact with stroke patients, and to understanding of stroke.
Testing version of PSSS. The PSSS was designed as a five-point Likert scale (1 = completely disagree, 2 = somewhat disagree, 3 = neither agree nor disagree, 4 = somewhat agree, and 5 = completely agree). There were 4 dimensions and 38 items in testing version of PSSS.

Ethical considerations. The study protocol was approved by the Ethics Committee of Guangdong
Provincial Hospital of Chinese Medicine. Participation was voluntary, the informed consent was set at the top of the questionnaire, participants anonymously answered the questions that were consistent with their agreement to participate, and agreed to the publication of the results. The investigation conforms with the principles outlined in the Declaration of Helsinki (Br Med J 1964;ii:177).

Data analysis.
All data analyses were conducted using SPSS version 24 and Mplus version 8 36 . Descriptive statistics were used to summarise the data.
Reliability. The internal consistency reliability was tested using McDonald's Coefficient Omega. A score above 0.80 was considered to be a satisfactory measure 37 . Test-retest reliability was measured using intraclass correlation coefficients. Reliability coefficients above 0.70 were considered satisfactory 38 .
Construct validity The Kaiser-Meyer-Olkin (KMO) test and Bartlett's test of sphericity were used to ensure the adequacy of data for EFA. The Bartlett's test of sphericity must be significant (p < 0.05) and the KMO value must be higher than 0.60 39 . Parallel analysis and Velicer's minimum average partial (MAP) 40 were employed to determine the number and stability of factor extraction 41 . A minimum squares with oblimin rotation was conducted 42 . Items loading at approximately 0.50 on one dimension and not showing cross-loading above 0.45 were retained for analysis in the model 43 .
ICM-CFA, B-CFA, ESEM, B-ESEM were performed using MLR 44,45 to evaluate the fitness on the assumed theoretical dimensions of the PSSS. In ICM-CFA, each indicator is assumed to correspond to a single factor, and B-CFA explicitly accommodates psychometric multidimensionality in the indicators by relaxing the ICM-CFA. Assessing conformations of stratified tissues requires a bifactor model, whereas assessing conceptually adjacent conformations requires an ESEM. However, the bifactor model may express unmodeled cross-loadings through inflated G-factors, whereas the ESEM model may express unmodeled G-factors through inflated crossloadings 46 . The goodness of fit criteria range was calculated using adjusted chi-square (χ 2 /df), comparative fit index (CFI > 0.95), Tucker-Lewis index (TLI > 0.95), root mean square error of approximation (RMSEA ≤ 0.06) 47 .
Ethical approval and consent to participate. The study protocol was approved by the Ethics Committee of Guangdong Provincial Hospital of Chinese Medicine. Participation was voluntary, the informed consent was set at the top of the questionnaire, participants anonymously answered the questions that were consistent with their agreement to participate, and agreed to the publication of the results.  Tables 1 and 2). EFA was undertaken using minimum squares with an oblimin to determine the underlying dimensions of the PSSS. Four factors were extracted, with loadings of 0.4 or greater ( Table 2): factor 1, inherent ideology, refers to stigma generated by the stereotype; factor 2, aesthetic feelings, refers to a feeling generated from stroke patients' appearance; factor 3, avoidance behaviour refers to stigmatized behaviour; and factor 4, policy attitudes refer to stigmatized attitudes. These four extracted factors explained 61.57% of the total variance in stroke-related public stigma. Therefore, 33 items in a four dimensions scale were generated.   www.nature.com/scientificreports/ Confirmatory factor analysis. ICM-CFA, B-CFA, ESEM and B-ESEM were performed to evaluate the fit on the assumed theoretical dimensions of the PSSS. The fit indices of the models were reported in Table 3. The results showed that ICM-CFA provided an unacceptable level of fitness (CFI < 0.90, TLI < 0.90), and ESEM provided a qualified CFI (0.903) but the TLI was less than 0.90. The fit indices of B-CFA and B-ESEM both showed an acceptable level of fitness but not excellent enough. So we modified the models based on modifications indices. The results suggested that the B-CFA provided the best fit to the data with CFI = 0.957, TLI = 0.950, RMSEA = 0.040. (Fig. 2). We performed 10 correlations between residuals on the theoretical basis, which were item 1 and item 2, item 2 and item 3, As a thinking of stroke is a lifelong disease such as disability, and then it will remind one feel that the stroke patients are unable to work anymore, and then they can no longer take care of themselves anymore; item 16 and item 17, item 17 and item 18, the stiff facial expression and curled up on the bed of stroke patients were disease related experiences and can make others feel uncomfortable, and then they may suffer with urinary tube etc.; item 20 and item 30, item 22 and item 23, item 23 and item 24, item 27 and item 28, when a person is reluctant to engage with a stroke patient, they will also be reluctant to see information about it on media. If a person does not have the patience to communicate with a stroke patient will also try to stay away from the stroke patient, so that in the end they cannot be friends with stroke patients, and the attributes of the family of the stroke patient and the caregiver of the stroke patient are related in that they both have close contact with the stroke patient. item 33 and item 34, item 34 and item 35, public facilities require economic investment, and public benefits and long-term care insurance are related. The parameter estimates from all models were presented in Table 4, and the correlation between factors were shown in supplementals Table 3.

Discussion
The present study developed the PSSS, aimed to assess stroke-related public stigma in non-patient populations. Four factors (inherent ideology, aesthetic feelings, avoidance behaviour, policy attitudes) were determined using EFA and B-CFA. Factor 1, 'inherent ideology,' refers to the subjective view of non-patient populations that is generated by the stereotype of stroke patients. We designed items which indicated stroke as a chronic disease, deprivation of working ability and self-care ability, and allowing transposition thinking such as 'I would feel embarrassed if I had a stroke, ' and ' A stroke is worse than death. ' Stereotypes are especially efficient means of categorising information about social groups. They are considered 'social' because they represent collectively agreed upon notions of groups of persons. They are 'efficient' because people can quickly generate impressions and expectations of individuals who belong to a stereotyped group 48 . This conception is well studied in mental illness stigma 14,16 . The item 'Stroke is a lifelong disease (such as disability)' is similar to the item 'getting cancer means having to mentally prepare oneself for death' from the Cancer Stigma Scale 17 . Stroke survivors have much higher rates of hemiplegia, aphasia, dysphagia, and urinary incontinence compared with other chronic diseases. Cancer reminds of death, whereas stroke reminds of long-term disability.
Factor 2, 'aesthetic feelings, ' implies public feelings towards visible, external manifestations among stroke patients. This factor stems from the 'aesthetic' dimension of Jones multidimensional concept of stigma 29 , www.nature.com/scientificreports/ described as a primitive response by the perceiver towards a non-concealable mark that makes the person less 'pleasing to the eye.' This is well confirmed in our qualitative analysis. Participants can easily describe external manifestations of stroke patients, such as tilted head, crooked mouth, and even urinary incontinence. The impression of stroke on the public is profound. However, existing tools examining stigma, fail to express this dimension 17,22,23 . Therefore, items in this dimension were generated based on qualitative data. Factor 3, 'avoidance behaviour, ' refers to a typical immediate response to public stigma; it is most prominent in social interaction. Stroke survivors stated that their friends or relatives become estranged after knowing the situation 13 . This is supported by our qualitative data. Participants indicated that they cannot always interact with patients and worried about causing unnecessary trouble to themselves. This may be explained through Kurzban and Leary's theory 26 -human beings possess cognitive adaptations designed to cause them to avoid poor social exchange partners and join cooperative groups. This dimension is included in other assessment tools [49][50][51] . In the www.nature.com/scientificreports/ current study, items in this dimension involve a comprehensive social interaction category, including avoidance behaviours such as towards individuals, families, social interactions, and media. Factor 4, 'policy attitudes, ' refers to supportive attitudes of the public towards stroke patients. The Cancer Stigma Scale includes this dimension and has satisfactory reliability and validity 17 . Besides, the item 'more tax money should be spent on the care and treatment of the mentally ill' in the benevolence dimension of the Community Attitudes Toward the Mentally Ill scale, also indicated policy attitudes 23 . This study compiled positive reverse items in this dimension. Since society assumes that older adults suffer from stroke, except for treatment, support for older patients is limited. For example, the society believes that it is normal to retire at an old age; therefore, some insufficient supportive measures are taken for older stroke patients 19,24 . This dimension involves financial, public, social, and family support.
These four dimensions cover three aspects of stigma measuring: the disease itself, stigma behaviour and stigma attitude. PSSS comprehensively evaluated stroke-related public stigma. Our study developed practical and appropriate items through interviews and expert ratings; items indicated satisfactory content validity. We confirmed the four-factor model using EFA, ICM-CFA, B-CFA, ESEM and B-ESEM. Additionally, the B-CFA showed that the PSSS had a good fit index and confirmed the four-factor model. The B-CFA demonstrated that every item in the PSSS contributed to the domain of the scale; thus, the construct validity was adequate. Regarding reliability, PSSS has satisfactory internal consistency and test-retest reliability, indicating that it is consistent and stable across time.
Limitations. Although our study confirmed that the PSSS is a valid and reliable instrument to measure stroke-related public stigma, some limitations exist. First, the study included a convenient sample, although we do ensure a wide range of age in sampling stage. Second, although we tried to avoid including items that may set social expectations, we could not avoid this situation. Third, the current study assessed validity and reliability in a Chinese context, further studies would be needed to evaluate public stigma of stroke in other contexts. Fourth, the classical test theory has been criticised by scholars of modern measurement theory because of the "inherent flaws" of the mathematical model on which it is based, and there is an urgent need to test the PSSS against modern measurement theory, for example by using Item Response Theory to test the difficulty and discrimination of the PSSS items, which is another important direction for future scale development and application.

Conclusions
Stroke-related stigma could have a negative influence on both, patients, and non-patient populations. Our study demonstrated that the PSSS is an appropriate instrument to assess stroke-related stigma among non-patient populations. The availability of the PSSS would help assess levels of stroke-related public stigma and design interventions to decrease stigma.

Data availability
All data generated or analysed during this study are included in this published article. The datasets generated and/or analysed during the current study are available from the corresponding author on reasonable request. www.nature.com/scientificreports/