Adaptation, psychometric properties and factor structure of the Spanish Quality in Psychiatric Care-Outpatient Staff (QPC-OPS) instrument

Quality of care is a multidimensional concept that should include the perspectives of all parties involved. There are few instruments with adequate psychometric properties for the assessment of the quality of psychiatric care in community mental health. Quality in Psychiatric Care-Outpatient Staff (QPC-OPS) instrument has adequate psychometric properties to evaluate the quality of psychiatric care from the perspective of professionals. The aim of this study was to validate the Spanish version of the QPC-OPS instrument. The instrument was translated and back-translated, and then was administered to 260 professionals from distinct community mental health services. To assess test–retest reliability, it was re-administered after 7–14 days (n = 157). Confirmatory factor analysis revealed an 8-factor-structure identical to the original version, showing the good fit of the model. The internal consistency coefficient (Cronbach’s alpha) was 0.885. The intraclass correlation coefficient was 0.847 (95% IC 0.790–0.888), which was higher than 0.70 in all factors bar one. The NT394 General Satisfaction Scale was used for analysis of convergent validity showing a rho correlation of 0.31 (p < 0.0001). Results show that the Spanish version of the QPC-OPS instrument is valid and reliable for the assessment of the quality of psychiatric care in the community setting.

, stands out as it is one of the most frequently used instruments in our setting for the evaluation of job satisfaction in various environments, including health care 13 . Professionals have already been described as an essential element in ensuring high standards of quality of care in current approaches such as person-centered care 14 . The combination of the perspectives of professionals and users in the community setting is necessary and provides complementary information of great value 15 . Consequently, the perspective of mental health professionals should be taken into account and used as an additional indicator of care quality in mental health 10,16 . This would help to identify quality improvement strategies 17 and to detect factors that can have a negative impact on the quality of care, such as burnout or work overload among care professionals [18][19][20] . Nevertheless, it has been observed that professionals from distinct disciplines involved in psychiatric care have differing points of view on what characterizes quality of care 4,21,22 .
Separately, there is a lack of comparative studies on the perceptions of quality of care among patients and professionals 23 , mainly due to the absence of standardized assessment instruments with adequate psychometric properties 18 . By way of illustration, a recent systematic review revealed that despite the considerable volume of existing instruments that evaluate the quality of psychiatric care or satisfaction with psychiatric care, their psychometric properties were not very robust and some validation processes involving these instruments were not completely satisfactory 24 .
One instrument that does possess adequate psychometric properties for the community mental health context is the Psychiatric Out-Patient Experiences Questionnaire (POPEQ). However, it only examines the user's perspective 25 .
The only instrument found to have adequate psychometric properties applicable in the community mental health context, and which assesses the quality of psychiatric care from the perspective of both professionals and users, is the Quality in Psychiatric Care 26 in its two versions: Quality in Psychiatric Care-Outpatient (QPC-OP) and Quality in Psychiatric Care-Outpatient Staff (QPC-OPS). The latter version has recently been validated in Norwegian 27 .
This instrument forms part of the family of instruments Quality in Psychiatric Care (QPC) that assess the quality of psychiatric care through its multiple versions in the hospital setting (Quality in Psychiatric Care-Inpatients-QPC-IP) 28 , the community (Quality in Psychiatric Care-Outpatient-QPC-OP) 26 and forensic (Quality in Psychiatric Care-Forensic In-patient QPC-FIP) 29 . The definition of care used to create the instrument was developed through a phenomenographic study 9 , it was evaluated for face validity in a pilot study and empirically tested in a sample of patients admitted to psychiatric hospital units in Sweden 30 .
This study is part of a wider international project to adapt the QPC-OPS instrument in a number of countries, test the psychometric properties and dimensional equivalence of the different versions according to language, and describe and compare the quality of psychiatric care in the community setting in these countries. In this context, the aim of the study was to adapt the QPC-OPS instrument into Spanish and analyze its reliability and validity.

Methods
Design. The study was conducted in two phases. In the first phase, the translation and adaptation of the QPC-OPS instrument into Spanish was carried out. In the second phase, the psychometric properties of the Spanish version of the QPC-OPS instrument were analyzed.
Participants and study setting (sample size). The sample consisted of 260 professionals from different disciplines who work in community mental health services (nursing, psychiatry, social education, case management, social work, administration and occupational therapy), who were actively working in a community service at the time of the study (Outpatient Mental Health Center, Day Hospital, Labor Reintegration Service, Community Rehabilitation Service) and who participate voluntarily. Having less than one year's experience in the area of mental health was established as an exclusion criterion. Non-probability convenience sampling was used.
Data collection was carried out between February, 2019 and February, 2020. Calculation of the sample size was based on internal consistency, temporal stability and construct validity. Estimation of internal consistency was performed following the recommendations of Streiner et al., who considered that between 5 and 20 individuals should be included for each instrument item 31 . In this study, we agreed to include a minimum of five individuals for each item.
To analyze temporal stability, it was estimated that a minimum of 61 professionals would be needed to detect an intraclass correlation coefficient (ICC) around 0.70 between two administrations of the instrument, assuming a confidence level of 95% and a power of 80% in a bilateral comparison 32 .
For construct validity, it was established that the minimum number of subjects necessary would be 250 33 .
Variables and sources of information. As indicated, the QPC-OPS instrument assesses the quality of psychiatric care in the community setting from the perspective of the professionals working there. It consists of a total of 30 items distributed across 8 factors as follows: encounter (6 items), participationempowerment (3 items), participation-information (5 items), discharge (3 items), support (4 items), environment (3 items), next of kin (2 items), and accessibility (4 items). Each item begins with the words "I feel that…" and is scored on a Likert-type scale with four response options ranging from 1 (totally disagree) to 4 (totally agree) with a "not applicable" option for each if considered necessary. A global and per-factor score can be obtained; the maximum global score is 120 points and the minimum 30 points so that a high score on each factor or globally indicates a good perception of the quality of psychiatric care on the part of the professionals. Conversely, a low score justifies the need for an intervention designed to bring about improvements in the areas identified.
The General Satisfaction scale NTP 394 created by Warr et al. 11  www.nature.com/scientificreports/ that ranged between 0.85 and 0.88 and an Intraclass Correlation Coefficient (ICC) of 0.63. This scale assesses intrinsic and extrinsic aspects of working conditions and consists of 15 items. Each item is evaluated through an ordinal scale that ranges from 1 (very dissatisfied) to 7 (very satisfied). The total score is obtained by summing the items, producing a final score between 15 and 105. In addition, data were collected on other variables related to sociodemographic and labor characteristics of the sample: age, sex, nationality, professional category, service where currently working and number of years in the service.
Procedure. The translation and back-translation process was carried out following the Standards for Educational and Psychological Testing 34 .
First, the original version was translated into Spanish by two independent native-speaker translators who had no knowledge of the instrument or the aims of the study. A group of experts comprising nurses, psychiatrists and psychologists reviewed both translations and reached agreement on the first version of the instrument in Spanish. Subsequently, the Spanish version was back-translated into the original language to confirm that the Spanish translation corresponded to the original version in Swedish. In addition, the original authors of the QPC-OPS examined the back-translation and compared it with the original version, finding no discrepancies requiring modification. This version was then pilot tested in 30 professionals with the aim of assessing item comprehension and clarity, as well as the time needed to administer it. Following the debriefing, it was not considered necessary to make any changes to either format of content.
Statistical analysis. Construct validity. Construct validity was analyzed through confirmatory factor analysis (CFA) with estimated parameters using the method of least squares generalized with a polychoric correlation matrix. This method has the same properties as the maximum likelihood estimation, even though the criteria were less strict than the normal ones. It is mainly used to measure ordinal items 35 .
The following fit indices were calculated to determine the overall fit of the model: Bentler Bonnet Normed Fit Index (BBNFI), Bentler Bonnet Non-Normed Fit Index (BBNNFI), the Goodness-of-Fit Index (GFI), the Adjusted Goodness-of-Fit Index (AGFI), the Comparative Fit Index (CFI), the Root Mean Square Error of Approximation (RMSEA), the chi-squared goodness-of-fit test and the ratio between chi-squared and the degrees of freedom (χ 2 /df). The criteria for a good fit were an X 2 /df ratio < to 3 and BBNFI, BBNNFI, GFI, AGFI and CFI close to 0.90 [36][37][38] , and a RMSEA value lower than 0.08 39,40 .
Convergent validity. Convergent validity was analyzed using the Spearman correlation coefficient with the NTP 394 General Satisfaction Scale.
As a complementary method, an analysis of the Spearman correlation was conducted between the QPC-OPS questionnaire factors to assess convergent validity based on the hypothesis that the correlation between each factor and the general instrument should be higher than the correlations between the factors 41 .
Reliability. The item analyses included calculation of item means, standard deviations and corrected item-total correlation.
Cronbach's alpha was used to assess the internal consistency of the instrument globally and for each of the factors. Levels of reliability above 0.70 were considered adequate 31 .
Temporal stability or test-retest was evaluated after 7-14 days through the ICC in a sample of 157 professionals. The values of this coefficient range between 0 and 1. A value greater than or equal to 0.70 was considered an indicator of good agreement 31 . Composite reliability was also calculated.
The SPSS Statistics program version 26 was used for analyses, along with EQS program version 6.2 for the confirmatory factor analysis (CFA) 42 .
Ethics declarations. First, in order to be able to carry out the study, permission to translate and adapt the QPC-OPS instrument into Spanish was sought from the original authors.
This study was approved by the research ethics committee at Fundación Sant Joan de Déu, under code CEIC PIC-83-16. All research was performed in accordance with relevant guidelines and regulations. All participants and their guardians were informed about the aim of the study and gave verbal and written consent to voluntarily and anonymously take part.

Results
Participant characteristics. A total of 260 professionals participated, of whom 26.2% were men and 73.8% were women. Some 95.8% of participants had Spanish nationality and the remaining 4.2% were of different nationalities. The sample consisted of a variety of professional categories: nursing (29.61%), psychiatry (20.38%), psychology (16.53%), social education (8.84%), case management (7.7%), social work (7.30%), administration (5%) and occupational therapy (4.61%). The mean age was 40 ± 10.3 years, while the mean of years worked in community service was 8.68 ± 7.70 years.
Construct validity. Confirmatory factor analysis (CFA). Confirmatory factor analysis was used to verify the internal structure of the instrument, in which an 8-factor model identical to the structure of the original instrument was proposed. Table 1 shows the fit of the model. All indices showed a reasonable fit.
All item saturations were equal to or greater than 0.50 with the exception of items 2 (0. The hypothesis was confirmed in the analysis of the correlations between the factors and the general instrument, with the strongest correlations found between the majority of factors and the general instrument. Factors 1 (Encounter) and 3 (Participation-Information) showed the strongest correlations with the total instrument (rho = 0.764 y rho = 0.755, respectively), while Factor 6 (Environment) had the weakest correlation (rho = 0.348). Table 2 shows the correlations of all the factors with the total instrument score.
Reliability. Cronbach's alpha internal consistency coefficient for the whole instrument was 0.885, reaching values above 0.70 in five of the eight factors (Table 3). For the factors F4.Discharge, F6.Environment and F7.Next of Kin, Cronbach's alpha values of 0.462, 0.537 and 0.440 were obtained, respectively. Internal consistency was not notably improved by excluding any of the items.
The mean item value ranged from 2.58 to 3.82 and the standard deviation ranged from 0.44 to 0.92. Only three items showed an item-total correlation under 0.20 (item 2: "Security in the center is high", item 4: "Patients can feel safe along with other patients in the waiting room" and item 9: "Patients are not bothered by other patients in the waiting room") ( Table 4).
Appendix 1 shows inter-correlations between individual items for each QPC-OPS sub-scale. In factor 6 (Environment) the correlation between items 2 and 9 was 0.130.
The ICC analysis demonstrated that test-retest reliability was 0.847 (95% CI 0.790-0.888; n = 157), and this value was higher than 0.70 in all instrument factors, except D5 with a value of 0.681 (Table 3). Composite reliability was 0.957 for the whole instrument, reaching values higher than 0.70 in five of the eight factors. For the factors F4.Discharge, F6.Environment and F7.Next of Kin, composite reliability values of 0.473, 0.621 and 0.488 were obtained, respectively (Table 3).

Discussion
The aim of this study was to adapt the Quality in Psychiatric Care Outpatient Staff (QPC-OPS) instrument to Spanish and analyze its reliability and validity. This instrument allows assessment of the quality of community psychiatric care from the professionals' perspective so that strong points and areas for improvements in care can be detected. On the other hand, it is a useful instrument for the evaluation of interventions focused on improving the quality of psychiatric care.
The adaptation process of the QPC-OPS involved a translation and back-translation to obtain the Spanish version. Other studies in the QPC family 27,43-48 produced versions in other languages using a similar process. Results in this phase were satisfactory and there were no difficulties found with respect to the comprehension and administration of the instrument.
Results show that, at the psychometric level, the values for construct validity (Confirmatory factor analysis), internal consistency and temporal stability (test-retest) were adequate.
The confirmatory analysis performed that the Spanish version, like the Norwegian version 27 , has the same 8 factors on quality of care as the original Swedish QPC-OP 26 version and no modification of any item was required.
For the analysis of reliability, internal consistency was analyzed using Cronbach's alpha. A Cronbach's alpha of 0.885 was obtained for the whole instrument and the factors in general showed values greater than 0.70, which is considered an adequate value according to Nunnally and Berstein 49 . The global Cronbach's alpha is somewhat lower than the original version of the instrument 26 and other versions of the QPC family 28,29,43,44,48 , although higher than the Norwegian community version 27   Temporal or test-retest stability was also analysed in the study. This value was not calculated in the original Swedish QPC-OP version 26 , the other original versions of the QPC family 28,29 or in the translated versions 44,46,47 , with the exception of the Spanish hospital versions of the QPC-IPS and QPC-IP 43,48 . The ICC for the whole instrument and for each of the factors was above 0.70 except in the case of F5.Support (0.681), which in general is considered good agreement 31,32 . For the Spanish QPC-IPS and QPC-IP, the ICC for the whole instrument was 0.91 and the factors had values above 0.70 except for the factor F6.Discharge 43,48 . www.nature.com/scientificreports/ Convergent validity of the Spanish QPC-OPS in our study was calculated through the Spearman rho correlation coefficient with the NTP 394 General Satisfaction Scale. The correlation obtained was positive 50 , showing that the greater the job satisfaction, the higher the perception of the quality of psychiatric care provided. Of the studies related to the family of QPC instruments, this is the third study in which convergent validity is calculated, the first being the Spanish hospital version of the QPC-IPS 43 and the second the Spanish hospital version of the QPC-IP 48 . The Spanish QPC-IPS calculated convergent validity using the same NTP 394 scale, obtaining a higher value (rho = 0.58); considered a moderate positive correlation. The Spanish QPC-IP calculated convergent validity using a 10-point numerical satisfaction scale, obtaining also a higher value (rho: 0.49); considered a moderate positive correlation.
At the same time, the correlation was analyzed between the total instrument score and each of its factors, showing that the correlation was higher between each factor and the general instrument than the correlations between the factors. This confirms the Fayer and Machin hypothesis 41 .
The limitations of this study arise from the lack of uniformity in the professional categories as each one has differing numbers of professionals and the concept of "community mental health professionals" is very wide, encompassing many health workers. However, this reflects the reality of the composition of Spanish community care teams. A further limitation is that this instrument was adapted in the community context so it should only be used in this population profile or a similar one. Another limitation that should be highlighted is that the NTP scale presented a low correlation with the total QPC-OPS score. This may be due to the fact that it is not the most appropriate gold standard for the assessment of the quality of care. Nevertheless, it is a widely-used instrument in our setting. Future research should take into account that there is a need for a more suitable instrument to function as the gold standard.
Finally, it should be emphasized that it was not possible to assess sensitivity to change or predictive validity given the cross-sectional design of the study. These limitations should be taken into account in the design of future studies.

Conclusions
The Spanish version of the QPC-OPS instrument is a simple, useful tool for the measurement of various aspects of the quality of community psychiatric care from the perspective of mental health professionals. Its 8-factor structure and psychometric properties are consistent and in agreement with the original version, allowing the instrument to be used to measure the quality of community psychiatric care from the perspective of professionals in the Spanish-speaking population. The results of these measurements can be used to increase the capacity to assess the quality of services provided.

Data availability
The data that support the findings of this study are available upon reasonable request from the corresponding author. The data are not publicly available due to privacy and ethical restrictions. The data was taken from our own study.