Article | Open | Published:

Comparison of focused cognitive training and portable “brain-games” on functional outcomes for vocational rehabilitation participants

Abstract

Cognitive remediation performed in a cognitive laboratory was compared with a sham control using portable brain games to study effects on vocational, neurocognitive, and functional outcomes for participants with psychotic disorders in vocational rehabilitation (VR). Seventy-seven participants (61% schizophrenia, 39% other psychosis) in transitional (45.5%) or supported employment (54.5%) were randomly assigned to 6 months of portable cognitive-games (CG) or cognitive remediation (CR) plus a weekly goal-setting group, and evaluated during training, post-training and at 12 months. Overall rates of employment did not differ significantly at 12-month follow-up; however, VR + CG attained employment more rapidly during training. A significant time by condition interaction favored VR + CR on Quality of Life Total Score and Instrumental Functioning over 12 months. Neurocognitive outcomes favored VR + CR, particularly on attention. Training hours related significantly to neurocognitive improvement regardless of condition. No differences were found in training adherence despite portability for VR + CG. Results indicate that VR + CR had significantly greater effect than VR + CG on neurocognition and community functioning, but not on employment outcome. Job attainment rates during the training period revealed a potential advantage for portable training raising new questions concerning how cognitive remediation can be most effectively integrated with VR.

Introduction

Recent reviews of cognitive remediation (CR) for people with schizophrenia have found moderate effect sizes on neurocognitive outcomes, and there has been growing evidence for generalized benefits to social role functioning when paired with rehabilitation interventions. McGurk et al.1 performed a meta-analysis of 26 RCTs and found effect sizes of 0.41 for cognitive outcomes and 0.36 for psychosocial functioning; Wykes et al.2 performed a meta-analysis of 40 studies and reported effect sizes of 0.45 for cognitive improvement and 0.42 for social functioning; and Katsumi et al.3 recently did an updated literature review and reported similar findings.

Our research group (Wexler & Bell4,5) and others (e.g. McGurk et al.6; Nuechterlein et al.7) have focused on CR as an augmentation to vocational rehabilitation (VR), reporting favorable employment outcomes. A meta-analysis of 9 studies from 5 different countries8 found that participants receiving CR showed 20% higher employment rate, worked more days and earned US $959 more than those who did not receive CR. They concluded that CR plus vocational services enhanced productivity and quality of life. Recently, Lystad et al.9 reported on a complex study in Norway that compared CR with Cognitive Behavior Therapy (CBT) for participants with schizophrenia spectrum disorders in VR programs over two years. They found that both groups improved on several neurocognitive domains, though results favored CR. Vocational outcomes did not differ by condition, although cognitive improvement in CR uniquely predicted hours worked. As efforts to integrate CR into clinical and rehabilitation activities continue, it is important to learn as much as possible about its effectiveness across various rehabilitation programs in everyday practice. Wykes and Spaulding10 in a seminal paper argue that we need to know about the clinical significance of CR, its impact on functioning and what features of the intervention are most important and for whom. The present report describes a study designed with some of these aims in mind. Data collection was begun in 2009 and completed in 2015. To our knowledge, it is the first study conducted on individuals with schizophrenia-spectrum illness designed to compare CR in the context of a range of vocational rehabilitation services with an active sham control comprised of portable “brain-games.” The primary aim was to compare a scientifically validated focused cognitive training procedure (Brain Fitness and Insight by Posit Science) performed in a cognitive training laboratory (Cog Lab) to non-specific “brain games” (Brain Age-2) that would serve as a rigorous sham control. We believed that Brain Age-2 was an excellent choice as a sham control because it had face valid exercises of executive function and provided user-friendly feedback in the form of a “Brain Age” score, that indicated progress toward “younger” (better) cognitive performance. However, we believed it was a sham because unlike the targeted training of Posit Science’s auditory (Brain Fitness) and visual (Insight) exercises, the Brain Age-2 had no training that addressed primary sensory processing and had very limited within-exercise adaptations to performance. No CR clinical trial had used such a face valid comparator at the time this study was proposed. Recently, a review of 18 commercial CR programs concluded that Posit Science had by far the strongest scientific evidence of effectiveness, that Brain Age-2 was in the second tier of evidence, while almost all others had either poor evidence or none at all11. Thus, it appears that Brain-Age 2 has some scientific support for cognitive benefits, at least in healthy adults12,13. In addition, Brain Age-2 was on a portable, hand-held device (Nintendo DS), which allowed us to ask a second question regarding the impact of portability on adherence. While most previous studies including our own supported the benefits of targeted CR performed on site14, Brain Age-2 had the advantage that participants could use it outside of laboratory hours, and thus might train much more. Moreover, because portable training allows users to train at their convenience, it was less likely to conflict with job searching or establishing a new work schedule. At the time of this study, there were no tablet ready versions of Posit Science’s targeted CR exercises. Had there been, we might have focused our study on comparing portable training with Cog Lab training. Although we couldn’t do that comparison, it was still an important question to test whether portability would lead to more training adherence. For these reasons, we hypothesized that (1) scientifically supported CR training would be more effective in improving neurocognitive function than a face valid, sham control, and that these neurocognitive benefits would generalize to superior vocational outcomes and community functioning; and (2) portability of computer training would lead to more overall training activity. Results Sample Characteristics Baseline demographic, illness and employment characteristics (Table 1) showed no significant differences between conditions, except for a greater number of participants with schizophrenia and schizoaffective disorder (when combined) for the Posit Science group (72% vs 50%, p < 0.05). Most of the participants were receiving SSDI, SSI or VA service-connected benefits (89.6%) and hadn’t worked on average in almost 11 years. Average earnings in the last 30 days was about$25.00. At baseline 30 (39%) had not begun their vocational rehabilitation service at intake but were in process; 24 (31%) had just begun working in Incentive Therapy; 15 (19.5%) were in SE though none were currently working; 6 (8%) had begun working in Compensated Work Therapy at the VA, and 2 (2.6%) were working casual jobs a few hours a week. All participants were eventually enrolled in some form of VR with slightly more than half in SE (54.5%). There was no difference between conditions in type of VR services used (Table 1).

The 13 participants from the CMHC sample did not differ on PANSS scores at baseline, but they did differ significantly on QoL scores (VA mean = 67.9 (17.07); CMHC mean = 54.92 (11.45); t (75) = 2.62, p < 0.01). However, there was no significant difference between the two conditions regarding what site they came from (Chi Sq = 0.74, p = ns). The 13 participants from CMHC represents only 16.8% of the whole sample, and when site was added as a covariate in subsequent analyses, it had no effect on findings.

Retention and Adherence

Retention was somewhat better for VR + CR at 6- (87% vs 74%) and 12- month follow-up (74% vs 71%) but not statistically different (Consort Flow diagram in Supplementary Material). Adherence to treatment did not differ by condition (F(1,74) = 1.19, p = 0.29). A few participants spent a lot of time on Brain Age-2, influencing a higher mean number of training hours in the VR + CG condition but with a very large standard deviation and range (VR + CG M = 63.5, SD = 65.12, Range 0–193.00). When categorized into 3 levels of training (less than 14 hours; 14 to 60 hours; more than 60 hours), the conditions also did not differ significantly (Χ2(2) = 3.87, p = 0.15). While both conditions showed generally good attendance at the weekly goal-setting group, VR + CR participation (mean = 19.27 (7.95) groups out of 26 weeks) was significantly greater than that of VR + CG (mean = 14.78 (9.90), t(72) = 2.15, p < 0.05).

Employment Outcome

Conditions did not differ by competitive employment rate at 12-month follow-up (VR + CG = 9/38, VR + CR = 9/36; Χ2(1) = 0.02, p = 0.89). However, non-parametric mixed-effects model of employment attainment at 2, 4, 6 and 12 months revealed significant condition (Χ2(1). = 4.73, p < 0.03) and time (Χ2(3) = 13.94, p < 0.01) effects favoring VR + CG, but no time-by-condition interaction (Χ2(1) = 4.25, p = 0.24). This analysis indicated that VR + CG led to significantly earlier job attainment during the 6-month active training period, but those receiving VR + CR caught up over the 6-month follow-up period (Figure 1).

Hours engaged in training was not significantly associated with employment attainment (b = 0.003, se = 0.005, p = 0.591, OR = 1.003). Non-parametric analysis revealed significant differences in employment hours and earnings at 6 months favoring VR + CG (Table 2), but no difference at 12 months. Mixed-models analyses of employment hours using all time points (2 months, 4 months, 6 months, 12 months), showed a condition effect favoring VR + CG for employment hours (df = 1, ATS = 6.21, p < 0.013), but no time or time-by-condition interaction. There was a significant decline in overall productive hours across groups by the end of 12 months (time effect: df = 2.62, ATS = 5.56, p < 0.001), but no time-by-condition interaction (Supplements S2 & S3.) The decline in hours over time was mostly due to the 6-month transitional work program, which ended around the 6-month assessment. Since most recipients of this service didn’t get employed, their productive hours declined sharply by 12 months. Of note, 41.2% of participants receiving Supported Employment (SE) attained employment over 12 months compared with 16.7% of those in transitional employment a significant difference (Χ2(1) = 3.94, p < 0.05).

Neurocognitive Outcomes (Table 3)

Mixed-models indicated no main effect for condition on baseline neurocognitive measures (Tables 3 and 4). A significant time-by-condition interaction in the Attention domain (p = 0.034) indicated significant increases in Attention over 12 months (p = 0.002) in VR + CR, and no appreciable change from baseline in VR + CG (p > 0.50; Supplement S4). No other between-condition effects were detected on neurocognitive outcomes.

Additional analyses support the relationship between cognitive training activity and cognitive outcomes across conditions and time. Hours of cognitive training by time predicted change in Working Memory (p = 0.010) and Processing Speed (trend level; p = 0.059) over 12-month. There were also significant time effects, unrelated to hours of training, for Attention (p = 0.002) and Visual Learning (p = 0.019), and for Global Cognitive Index (p = 0.006). Follow-up analysis of simple slopes indicated that improvements in Working Memory were obtained for individuals who trained 40 hours or more across the treatment period. Three-way interactions between condition, training hours, and time were non-significant.

Quality of Life (QLS) Outcomes (Table 3)

Mixed-models indicated no difference in baseline between conditions (Tables 3 and 5). Significant time-by-condition interactions for Total QLS (p < 0.02) and Instrumental Function (p < 0.01; Supplement S5) indicated that these QLS measures remained stable over time in VR + CR (p < 0.50) but declined significantly for VR + CG (QLS Total p < 0.01 and Instrumental Function p < 0.001). Training hours were associated with Instrumental Function ratings at the trend level (p = 0.053).

Discussion

To our knowledge this is the first report comparing focused cognitive remediation in a “Cog Lab” to a sham control of cognitive games on a mobile device in the context of vocational services. Employment outcomes by 12 months in this chronically unemployed and disabled psychiatric sample do not indicate that either condition was superior. There may have been some advantage to having a portable device during the 6-month training phase as suggested by more rapid rate of employment and more hours of competitive work in the VR + CG condition. Furthermore, an upshift in employment in VR + CR following completion of cognitive training, reaching levels equivalent to VR + CG by 12 months, supports the interpretation that participants in this condition only fully engaged in job searching after completing the 6-month curriculum of cognitive training. Since hours spent training did not differ by condition and appeared unrelated to attaining employment by 12 months, it remains possible that coming into the “Cog Lab” rather than using a portable training system may have had an opportunity cost in terms of employment.

In a previous study that compared enhanced SE with 12-month cognitive training along with a social information processing group to enhanced SE alone5, we found that although the training condition led to better vocational outcomes at two-year follow-up, those in the “Cog Lab” took longer to obtain a job. The findings of overall superior employment outcome with cognitive training in that study may have been observed due to a longer treatment and follow-up period, which provided more time for the benefits of cognitive training to generalize to vocational function. Moreover, the enhancement of SE included an initial period of paid on-the-job training so that almost everyone began work within 3 months and had the benefit of synergy between cognitive training and work activity. McGurk and colleagues1,6 have similarly reported better vocational outcomes using cognitive training but all participants were engaged in SE. Unsurprisingly, in our study, what made the greatest difference in employment was whether participants were enrolled in SE. Those in SE had superior employment rates to those in transitional work programs, regardless of cognitive training condition.

Regarding neurocognitive outcomes, conditions did not differ on MATRICs global score, but VR + CR was superior to VR + CG in the Attention domain. This suggests that despite little overall improvement by either training method, there may have been some specific advantage for the Posit Science training over Brain Age-2 in this domain. We have long maintained that cognitive training produces the greatest benefits when performed in the context of other activating rehabilitation activities. Although, some participants obtained work in our study, most were looking for work or only working a few hours in transitional employment and were otherwise inactive. Cognitive training may be most effective when there are opportunities to generalize and reinforce its lessons through other rehabilitation activity. Nevertheless, we did detect a relationship between amount of training across condition and improvement on MATRICS Working Memory score, suggesting a dose response relationship regardless of the training method. Such findings are unlikely related to practice-effects because the period between assessments is large (6 months) and the MATRICs assessments were selected because they were ones that showed only small practice- effects at much shorter intervals.

Findings for community function as measured by QLS suggest that participants in the VR + CR experienced functional benefits that extended beyond the active intervention period, while those in VR + CG showed consistent decline from baseline over active and follow-up periods. Everyone began the study when they were beginning some type of vocational service, but after initial efforts it appears that the VR + CG group began to be less engaged in productive roles, despite a few more individuals obtaining employment. It may be that the systematic training that led to improvements in attention encouraged VR + CR participants to stay engaged in productive activity, even though it took them longer to get employed. It is also possible that a regular schedule evolved around “Cog Lab” training that created new opportunities to engage in community activity. Participating in more groups than VR + CR may also have increased their capacity for engagement, especially since goal setting included job searching and other forms of productive activity (such as helping family members). Although we do not know the mechanism for the relationship between VR + CR and QLS findings, it is a signal that cognitive remediation with goal-setting groups may be useful in augmenting rehabilitation efforts.

We hypothesized that a mobile device might lead to more training; however, both conditions showed good engagement in cognitive training with no significant difference in overall training hours. A few participants did use their portable device a great deal more than those who were limited to training during lab hours, but this was uncommon. There was, however, a significant difference in group attendance, suggesting that VR + CR participants may have been more likely to attend group since they were already attending “Cog Lab.” It is also possible that since VR + CG participants obtained employment sooner, going to groups conflicted with work activity. Now that many cognitive training programs including Posit Science’s Brain HQ are available on tablets, the option of portability might provide both the benefits of more focused training and times of training more compatible with individual schedules. It remains unknown, however, whether the assistance of a “Cog Lab” monitor and the added social experience of being with others in groups and in the “Cog Lab” offers unique benefits.

Limitations to these conclusions include the lack of a no-treatment control condition, the powerful disincentives to competitive employment in a sample where 90% are receiving disability payments, and the relatively small size of the sample. Only about half of the participants received SE, and since SE had the greatest impact on vocational outcomes its effects may have overshadowed other influences.

While we observed an advantage of laboratory-based over portable device training on cognitive outcomes, these gains did not appear to translate immediately to work outcome. Indeed, given that “Cog Lab” attendance may have impeded job seeking or attainment, the advantages of cognitive gains may come at the cost to other opportunities. Therefore, for those who identify work as a primary goal, we might recommend starting the job search first and offering cognitive training once the client is stable on the job and a need for cognitive remediation has been identified based on job performance or other client goals. Consistent with the supported employment model, the principles of “place then train” could apply appropriately to cognitive remediation interventions targeting vocational outcome.

Method

Participants

Seventy-seven participants including 64 Veterans at VA Connecticut Healthcare System (VACHS) and 13 non-veterans at Connecticut Mental Health Center (CMHC) with chronic and disabling psychotic illness (61% schizophrenia, 39% other psychosis) in transitional work services (45.5%) or supported employment (54.5) were included in the study. This was originally intended to be for Veterans only, but the study was modified in its third year to include non-Veterans from CMHC to meet recruitment goals. Inclusion criteria included willingness to engage in vocational rehabilitation services of some kind and presence of disabling psychotic illness. Additionally, vocational programs had some admission requirements including demonstrated need for services, referral from clinicians, and an interest in work rehabilitation. All participants were unemployed, though a few were earning money through informal work. Co-morbid substance use disorder was an exclusion with less than 30 days of abstinence. Known neurological disease, uncorrected sensory impairment, intellectual deficiency or premorbid IQ estimate under 70 were additional exclusion criteria.

Measures

Clinical, Quality of Life and Vocational Assessments

Structured Clinical Interview for DSM IV (SCID)15 was used for diagnostic evaluation at baseline. Positive and Negative Syndrome Scale (PANSS)16 was used for symptom assessment at baseline and quality of life was measured using Quality of Life Scale interviews (QLS)17. QLS has five subscales: Intrapsychic, Interpersonal, Instrumental Function and Objects and Activities. Our raters achieve good to excellent inter-rater reliabilities on all subscales on both instruments18,19. QLS was administered at baseline, 2-month, 4-month, 6-month and 12-month follow-ups. Vocational data was recorded at the same observation points and included job attainment, competitive hours worked and money earned, transitional hours worked and money earned, and volunteer work. These data were collected from the vocational program records which were available for Incentive Work Therapy (an in-house work placement that pays half minimum wage and has reduced expectations), Compensated Work Therapy (paid work placements through contracts) and SE. The very few participants who worked independently outside of SE reported their earnings at 6 and 12-month follow-up. Informal income was included in competitive earnings. Types of jobs obtained and reasons for termination were also recorded for qualitative purposes, but are not included in this report.

Neurocognitive Assessments

Premorbid intellectual functioning was assessed at baseline using the Wechsler Test of Adult Reading20. The MATRICS battery21 was used to assess changes in cognitive performance over time from baseline to 6-month and 12-month follow-up. Performance was evaluated based on age- and gender-corrected standard scores in domains of Processing Speed, Attention/Vigilance, Working Memory, Verbal Learning, Visual Learning, Reasoning and Problem Solving, and a global neurocognitive composite of these scores.

Procedures

All procedures were approved by the Institutional Review Boards at VA VACHS and Yale and were carried out in accordance with the relevant guidelines and regulations. Recruitment was done by clinician referral and by publicly displayed flyers. Following informed consent, participants had baseline assessments and if found eligible were randomized in blocks of 6 (3 to Cognitive Games (VR + CG) and 3 to laboratory-based Cognitive Remediation (VR + CR) in a 1:1 ratio by a statistician unrelated to the study. Blocks of 3 were used to ensure nearly equal numbers in both conditions while keeping assessors blind to condition. VR services were unrelated to randomization; individuals had free access to all available VR services. Trained master’s level or Ph.D. research staff blind to condition performed assessments. Training began in the “Cog Lab” located at the Learning Based Recovery Center (LBRC) at VACHS, but VR + CG participants were given the option of subsequently training at home following 3 sessions of instruction about how to use the portable devices. Goal-setting groups occurred weekly at the LBRC.

Vocational Rehabilitation (VR) Programs

VACHS and CMHC had a range of work services that included transitional programs and Individual Placement and Support22 SE programs. IPS programs at both institutions were independently evaluated for fidelity and were in the acceptable range. The transitional programs included paid placement in work therapy at VACHS or at a psychosocial clubhouse with staff support and guidance for job seeking.

Statistical Analyses

Employment outcomes were highly skewed and were analyzed using the nonparametric approach for repeated measures24. Data were ranked, then fitted using a mixed-effects model with an unstructured variance-covariance matrix and p-values adjusted for ANOVA-type statistics (ATS). Predictors included condition (VR + CG vs. VR + CR), time, and group-by-time. Employment rates were analyzed across time using a generalized linear model with a logit link function and random subject effects and included the same predictors as above. A logistic regression analysis tested differences between conditions in employment status at study end. Number of training hours was tested as a covariate.

Cognitive and QLS outcomes were analyzed using mixed-effects models that account for the correlation of repeated measures within participants while allowing for inclusion of cases with incomplete or missing data. Models included a random intercept and had condition, time, and group-by-time as predictors. MATRICS measures were collected at 3 time points (baseline, 6, and 12 months). QLS and employment, volunteer and transitional work hours and earnings were collected at 5 time points (baseline, 2, 4, 6, and 12 months). All tests-were two tailed with alpha set at 0.05.

Additional information

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

1. 1.

McGurk, S. R., Mueser, K. T., Feldman, K., Wolfe, R. & Pascaris, A. Cognitive training for supported employment: 2-3 year outcomes of a randomized controlled trial. Am J Psychiat 164(3), 437–441 (2007).

2. 2.

Wykes, T., Huddy, V., Cellard, C., McGurk, S. R. & Czobor, P. A Meta-Analysis of Cognitive Remediation for Schizophrenia: Methodology and Effect Sizes. Am J Psychiat 168(5), 472–485 (2011).

3. 3.

Katsumi, A., Hoshino, H., Fujimoto, S. & Niwa, S. Efficacy of Cognitive Remediation in Schizophrenia: A Short Review of Its Variable Effects According to Cognitive Domain. Open Journal of Psychiatry 5, 170–176, https://doi.org/10.4236/ojpsych.2015.52019 (2015).

4. 4.

Wexler, B. E. & Bell, M. D. Cognitive Remediation and Vocational Rehabilitation for Schizophrenia. Schizophr Bull. 31(4), 931–41 (2005).

5. 5.

Bell, M. D., Zito, W., Greig, T. & Wexler, B. E. Neurocognitive Enhancement Therapy with Vocational Services in Schizophrenia: Work Outcomes at Two Year Follow-Up. Schizophr Res. 105(1–3), 18–29 (2008).

6. 6.

McGurk, S. R., Mueser, K. T. & Pascaris, A. Cognitive training and supported employment for persons with severe mental illness: one-year results from a randomized controlled trial. Schizophr Bull. 31, 898–909 (2005).

7. 7.

Nuechterlein, K. H. et al. Individual placement and support for individuals with recent-onset schizophrenia: integrating supported education and supported employment. Psych Rehab J 31(4), 340 (2008).

8. 8.

Chan, J. Y., Hirai, H. W. & Tsoi, K. K. Can computer-assisted cognitive remediation improve employment and productivity outcomes of patients with severe mental illness? A meta-analysis of prospective controlled trials. J Psychiatric Res. 68, 293–300 (2015).

9. 9.

Lystad, J. U. et al. Neurocognition and occupational functioning in schizophrenia spectrum disorders: The MATRICS Consensus Cognitive Battery (MCCB) and workplace assessments. Schizophr Res. 31, 143–9 (2016).

10. 10.

Wykes. T., Spaulding, W.D. Thinking about the future cognitive remediation therapy—what works and could we do better? Schizophr Bull. Sep 1(37) (suppl 2):S80–90 (2011).

11. 11.

Shah, T. M., Weinborn, M., Verdile, G., Sohrabi, H. R. & Martins, R. N. Enhancing Cognitive Functioning in Healthly Older Adults: a Systematic Review of the Clinical Significance of Commercially Available Computerized Cognitive Training in Preventing Cognitive Decline. Neuropsychol Rev. 27, 62–80 (2017).

12. 12.

Nouchi, R. et al. Brain Training Game Improves Executive Functions and Processing Speed in the Elderly: A Randomized Controlled Trial. PLoS ONE 7(1), e29676, https://doi.org/10.1371/journal.pone.0029676 (2012).

13. 13.

Nouchi, R. et al. Brain Training Game Boosts Executive Functions, Working Memory and Processing Speed in the Young Adults: A Randomized Controlled Trial. PLoS ONE 8(2), e55518, https://doi.org/10.1371/journal.pone.0055518 (2013).

14. 14.

Bell, M. D., Choi, K. H., Dyer, C. & Wexler, B. E. Benefits of Cognitive Remediation and Supported Employment for Schizophrenia Patients with Poor Community Functioning. Psychiatr Serv 65(4), 469–475 (2014).

15. 15.

First, M. B., Spitzer, R. L., Gibbon, M. & Williams, J. B. Structured Clinical Interview for DSM-IV Axis I Disorders, Patient Edition (SCID-I/P). (New York State Psychiatric Institute. Biometrics Research, New York, 1996).

16. 16.

Kay, S. R., Fiszbein, A. & Opler, L. The Positive and Negative Syndrome Scale (PANSS) for Schizophrenia. Schizophr Bull. 13, 261–276 (1987).

17. 17.

Heinrichs, D., Hanlon, E. & Carpenter, W. The Quality of Life Scale: an instrument for rating the schizophrenic deficit syndrome. Schizophr Bull. 10, 388–398 (1984).

18. 18.

Bell, M. D., Milstein, R. M., Goulet, J. L., Lysaker, P. H. & Cicchetti, D. The positive and negative syndrome scale and the brief psychiatric rating scale: Reliability, comparability and predictive validity. Journal of Nervous and Mental Disorders 180, 723–728 (1992).

19. 19.

Bryson, G., Greig, T., Lysaker, P. & Bell, M. D. Quality of life benefits of paid work activity in schizophrenia: a longitudinal analysis. Schizophr Bull. 28(2), 249–257 (2002).

20. 20.

The Psychological Corporation. Wechsler test of adult reading. (Harcourt Assessment, San Antonio, TX, 2001).

21. 21.

Nuechterlein, K. H. & Green, M. F. MATRICS Consensus Cognitive Battery Manual. (MATRICS Assessment Inc, Los Angeles, CA, 2006).

22. 22.

Bond, G. et al. Implementing supported employment as an evidence-based practice. Psychiatr Serv 52(3), 313–22 (2001).

23. 23.

Petry, N. M., Martin, B., Cooney, J. L. & Kranzler, H. R. Contingency Management to Reduce Substance Use in Individuals Who are Homeless with Co-Occurring Psychiatric Disorders. J Consulting and Clinical Psychol 68(2), 250–257 (2000).

24. 24.

Brunner, E., Domhof, S. & Langer, F. Nonparametric analysis of longitudinal data in factorial experiments. (John Wiley and Sons, New York, NY, 2002).

Download references

Acknowledgements

This study was funded by a MERIT grant from VA Rehabilitation Research and Development Service (#D4752R) “Cognitive Training to Enhance Work Program Outcomes”. Morris Bell, Ph.D. Principal Investigator; and by a VA Rehabilitation Research and Development Senior Research Career Scientist grant to Morris Bell, Ph.D., and VA Rehabilitation Research and Development Career Development Awards to Jason Johannesen, Ph.D (#D7008W). and Holly Laws, Ph.D. (IK1RX002095). This study is registered with ClinicalTrials.gov. NCT00829400. Date of Registration is: 01/24/2009.

Author information

M.B. and J.J. designed the study and carried out procedures. H.L. and B.P. performed analyses and prepared tables and figures. All authors discussed analytic approaches and results and participated in writing the manuscript.

Competing Interests

The authors declare that they have no competing interests.

Correspondence to Morris D. Bell.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.