The collection of electronic patient-reported outcome (ePRO) data at the point of care is feasible and usually provides more reliable information on the patient's experience than that reported by clinicians
ePRO data are of research quality, and are an important element of learning health-care systems and 'big-data' initiatives
To maximize the utility of big data in learning health-care systems, they must include the 'patient's voice' via the incorporation of PROs into routine care
To date, big-data initiatives have not adequately included ePROs; this situation can be addressed through data standardization
The routine collection of ePRO data simultaneously improves the quality of care for patients and facilitates big-data initiatives
Evidence obtained over the past few years indicates that the routine collection and inclusion of ePRO data in patient care improves clinical outcomes
Recording of patient-reported outcomes (PROs) enables direct measurement of the experiences of patients with cancer. In the past decade, the use of PROs has become a prominent topic in health-care innovation; this trend highlights the role of the patient experience as a key measure of health-care quality. Historically, PROs were used solely in the context of research studies, but a growing body of literature supports the feasibility of electronic collection of PROs, yielding reliable data that are sometimes of better quality than clinician-reported data. The incorporation of electronic PRO (ePRO) assessments into standard health-care settings seems to improve the quality of care delivered to patients with cancer. Such efforts, however, have not been widely adopted, owing to the difficulties of integrating PRO-data collection into clinical workflows and electronic medical-record systems. The collection of ePRO data is expected to enhance the quality of care received by patients with cancer; however, for this approach to become routine practice, uniquely trained people, and appropriate policies and analytical solutions need to be implemented. In this Review, we discuss considerations regarding measurements of PROs, implementation challenges, as well as evidence of outcome improvements associated with the use of PROs, focusing on the centrality of PROs as part of 'big-data' initiatives in learning health-care systems.
Subscribe to Journal
Get full journal access for 1 year
only $17.75 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Levit, L. A., Balogh, E. P., Nass, S. J. & Ganz, P. A. (eds) Delivering high-quality cancer care: charting a new course for a system in crisis (National Academies Press, 2013).
US Food and Drug Administration. Guidance for industry — patient-reported outcome measures: use in medical product development to support labeling claims. Health Qual. Life Outcomes 4, 79 (2006).
Howie, L., Hirsch, B., Locklear, T. & Abernethy, A. P. Assessing the value of patient-generated data to comparative effectiveness research. Health Aff. 33, 1220–1228 (2014).
Locklear, T. et al. Reaching consensus on patient-centered definitions: a report from the Patient-Reported Outcomes PCORnet Task Force. NIH Collaboratory https://www.nihcollaboratory.org/Products/Reaching%20Consensus_April_9_2015.pdf (2015).
Shapiro, M., Johnston, D., Wald, J. & Mon, D. Patient-generated health data, white paper. HealthIT.gov https://www.healthit.gov/sites/default/files/rti_pghd_whitepaper_april_2012.pdf (2012).
Fromme, E. K., Eilers, K. M., Mori, M., Hsieh, Y. C. & Beer, T. M. How accurate is clinician reporting of chemotherapy adverse effects? A comparison with patient-reported symptoms from the Quality-of-Life Questionnaire C30. J. Clin. Oncol. 22, 3485–3490 (2004).
Basch, E. et al. Adverse symptom event reporting by patients versus clinicians: relationships with clinical outcomes. J. Natl Cancer Inst. 101, 1624–1632 (2009).
Atkinson, T. M. et al. Reliability of adverse symptom event reporting by clinicians. Qual. Life Res. 21, 1159–1164 (2012).
Mesa, R. A. et al. Effect of ruxolitinib therapy on myelofibrosis-related symptoms and other patient-reported outcomes in COMFORT-I: a randomized, double-blind, placebo-controlled trial. J. Clin. Oncol. 31, 1285–1292 (2013).
Verstovsek, S. et al. A double-blind, placebo-controlled trial of ruxolitinib for myelofibrosis. N. Engl. J. Med. 366, 799–807 (2012).
Geyer, H. L. et al. Distinct clustering of symptomatic burden among myeloproliferative neoplasm patients: retrospective assessment in 1470 patients. Blood 123, 3803–3810 (2014).
Cleeland, C. S. & Williams, L. A. Symptom burden in hematologic malignancies. Blood 123, 3686–3687 (2014).
Rock, E. P. et al. Patient-reported outcomes supporting anticancer product approvals. J. Clin. Oncol. 25, 5094–5099 (2007).
Burris, H. A. et al. Improvements in survival and clinical benefit with gemcitabine as first-line therapy for patients with advanced pancreas cancer: a randomized trial. J. Clin. Oncol. 15, 2403–2413 (1997).
Temel, J. S. et al. Early palliative care for patients with metastatic non-small-cell lung cancer. N. Engl. J. Med. 363, 733–742 (2010).
Bakitas, M. et al. Effects of a palliative care intervention on clinical outcomes in patients with advanced cancer: the Project ENABLE II randomized controlled trial. JAMA 302, 741–749 (2009).
Bakitas, M. A. et al. Early versus delayed initiation of concurrent palliative oncology care: patient outcomes in the ENABLE III randomized controlled trial. J. Clin. Oncol. 33, 1438–1445 (2015).
Zimmermann, C. et al. Early palliative care for patients with advanced cancer: a cluster-randomised controlled trial. Lancet 383, 1721–1730 (2014).
Grudzen, C. R. et al. Emergency department-initiated palliative care in advanced cancer: a randomized clinical trial. JAMA Oncol. 2, 591–598 (2016).
Smith, T. J. et al. American Society of Clinical Oncology provisional clinical opinion: the integration of palliative care into standard oncology care. J. Clin. Oncol. 30, 880–887 (2012).
Dionne-Odom, J. N. et al. Benefits of early versus delayed palliative care to informal family caregivers of patients with advanced cancer: outcomes from the ENABLE III randomized controlled trial. J. Clin. Oncol. 33, 1446–1452 (2015).
Fayers, P. & Bottomley, A. Quality of life research within the EORTC — the EORTC QLQ-C30. Eur. J. Cancer 38 (Suppl. 4), 125–133 (2002).
Basch, E. et al. Development of the National Cancer Institute's patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). J. Natl. Cancer Inst. 106, dju244 (2014).
Dueck, A. C. et al. Validity and reliability of the US National Cancer Institute's Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). JAMA Oncol. 1, 1051–1059 (2015).
Basch, E. Patient-reported outcomes — harnessing patients' voices to improve clinical care. N. Engl. J. Med. 376, 105–108 (2017).
Basch, E. et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA 318, 197–198 (2017).
Sperti, E. & Di Maio, M. Outcomes research: integrating PROs into the clinic — overall survival benefit or not, it's worth the trouble. Nat. Rev. Clin. Oncol. 14, 529–530 (2017).
Basch, E. et al. Patient online self-reporting of toxicity symptoms during chemotherapy. J. Clin. Oncol. 23, 3552–3561 (2005).
Abernethy, A. P. et al. Feasibility and acceptability to patients of a longitudinal system for evaluating cancer-related symptoms and quality of life: pilot study of an e/Tablet data-collection system in academic oncology. J. Pain Symptom Manage. 37, 1027–1038 (2009).
Abernethy, A. P. et al. Validation of the Patient Care Monitor (Version 2.0): a review of system assessment instrument for cancer patients. J. Pain Symptom Manage. 40, 545–558 (2010).
Bennett, A. V., Jensen, R. E. & Basch, E. Electronic patient-reported outcome systems in oncology clinical practice. CA Cancer J. Clin. 62, 337–347 (2012).
Snyder, C. F. et al. PatientViewpoint: a website for patient-reported outcomes assessment. Qual. Life Res. 18, 793–800 (2009).
Judson, T. J. et al. Feasibility of long-term patient self-reporting of toxicities from home via the Internet during routine chemotherapy. J. Clin. Oncol. 31, 2580–2585 (2013).
Wood, W. A. et al. Feasibility of frequent patient-reported outcome surveillance in patients undergoing hematopoietic cell transplantation. Biol. Blood Marrow Transplant. 19, 450–459 (2013).
Cancer Support Community. Cancer Experience Registry. Cancer Support Community http://www.cancersupportcommunity.org/cancer-experience-registry (2017).
Abernethy, A. P. et al. Improving health care efficiency and quality using tablet personal computers to collect research-quality, patient-reported data. Health Serv. Res. 43, 1975–1991 (2008).
Velikova, G. et al. Automated collection of quality-of-life data: a comparison of paper and computer touch-screen questionnaires. J. Clin. Oncol. 17, 998–1007 (1999).
Yarnold, P. R., Stewart, M. J., Stille, F. C. & Martin, G. J. Assessing functional status of elderly adults via microcomputer. Percept. Mot. Skills 82, 689–690 (1996).
Lewis, G., Sharp, D., Bartholomew, J. & Pelosi, A. J. Computerized assessment of common mental disorders in primary care: effect on clinical outcome. Fam. Pract. 13, 120–126 (1996).
Drummond, H. E., Ghosh, S., Ferguson, A., Brackenridge, D. & Tiplady, B. Electronic quality of life questionnaires: a comparison of pen-based electronic questionnaires with conventional paper in a gastrointestinal study. Qual. Life Res. 4, 21–26 (1995).
O'Connor, K. P., Hallam, R. S. & Hinchcliffe, R. Evaluation of a computer interview system for use with neuro-otology patients. Clin. Otolaryngol. Allied Sci. 14, 3–9 (1989).
Fortner, B., Okon, T., Schwartzberg, L., Tauer, K. & Houts, A. C. The Cancer Care Monitor: psychometric content evaluation and pilot testing of a computer administered system for symptom screening and quality of life in adult cancer patients. J. Pain Symptom Manage. 26, 1077–1092 (2003).
Gwaltney, C. J., Shields, A. L. & Shiffman, S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health 11, 322–333 (2008).
Di Maio, M. et al. Symptomatic toxicities experienced during anticancer treatment: agreement between patient and physician reporting in three randomized trials. J. Clin. Oncol. 33, 910–915 (2015).
Muehlhausen, W. et al. Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013. Health Qual. Life Outcomes 13, 167 (2015).
Jensen, R. E. et al. Review of electronic patient-reported outcomes systems used in cancer clinical care. J. Oncol. Pract. 10, e215–e222 (2014).
Cirillo, M. et al. Clinician versus nurse symptom reporting using the National Cancer Institute-Common Terminology Criteria for Adverse Events during chemotherapy: results of a comparison based on patient's self-reported questionnaire. Ann. Oncol. 20, 1929–1935 (2009).
Trotti, A. et al. CTCAE v3.0: development of a comprehensive grading system for the adverse effects of cancer treatment. Semin. Radiat. Oncol. 13, 176–181 (2003).
Atkinson, T. M. et al. The association between clinician-based common terminology criteria for adverse events (CTCAE) and patient-reported outcomes (PRO): a systematic review. Support Care Cancer 24, 3669–3676 (2016).
Basch, E., Rogak, L. J. & Dueck, A. C. Methods for implementing and reporting patient-reported outcome (PRO) measures of symptomatic adverse events in cancer clinical trials. Clin. Ther. 38, 821–830 (2016).
Carlson, L. E. & Bultz, B. D. Cancer distress screening. Needs, models, and methods. J. Psychosom. Res. 55, 403–409 (2003).
Jacobsen, P. B. et al. Screening for psychologic distress in ambulatory cancer patients. Cancer 103, 1494–1502 (2005).
Cleeland, C. S. et al. Assessing symptom distress in cancer patients: the M. D. Anderson Symptom Inventory. Cancer 89, 1634–1646 (2000).
Basch, E. et al. Recommendations for incorporating patient-reported outcomes into clinical comparative effectiveness research in adult oncology. J. Clin. Oncol. 30, 4249–4255 (2012).
Basch, E. et al. Long-term toxicity monitoring via electronic patient-reported outcomes in patients receiving chemotherapy. J. Clin. Oncol. 25, 5374–5380 (2007).
Wysham, N. et al. Integration of electronic patient-reported outcomes into routine cancer care: an analysis of factors affecting data completeness. JCO Clin. Cancer Informat. http://dx.doi.org/10.1200/CCI.16.00043 (2017).
Di Maio, M., Basch, E., Bryce, J. & Perrone, F. Patient-reported outcomes in the evaluation of toxicity of anticancer treatments. Nat. Rev. Clin. Oncol. 13, 319–325 (2016).
Center for Medicare & Medicaid Innovation. Oncology Care Model. CMS.gov https://innovation.cms.gov/initiatives/oncology-care (2017).
US Food and Drug Administration. Patient-focused drug development. US Food and Drug Administration https://www.fda.gov/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/OCE/ucm544143.htm (2017).
Basch, E. The missing voice of patients in drug-safety reporting. N. Engl. J. Med. 362, 865–869 (2010).
LeBlanc, T. W. & Abernethy, A. P. Quality of life in higher resolution: the next generation of comparative effectiveness research in malignant hematology. Haematologica 98, 823–824 (2013).
Pakhomov, S. V., Jacobsen, S. J., Chute, C. G. & Roger, V. L. Agreement between patient-reported symptoms and their documentation in the medical record. Am. J. Manag. Care 14, 530–539 (2008).
Flynn, K. E. et al. Patient experiences with communication about sex during and after treatment for cancer. Psychooncology 21, 594–601 (2012).
Reese, J. B., Shelby, R. A., Keefe, F. J., Porter, L. S. & Abernethy, A. P. Sexual concerns in cancer patients: a comparison of GI and breast cancer patients. Support Care Cancer 18, 1179–1189 (2010).
Brucker, P. S., Yost, K., Cashy, J., Webster, K. & Cella, D. General population and cancer patient norms for the Functional Assessment of Cancer Therapy-General (FACT-G). Eval. Health Prof. 28, 192–211 (2005).
Dudgeon, D. et al. Cancer Care Ontario's experience with implementation of routine physical and psychological symptom distress screening. Psychooncology 21, 357–364 (2012).
Baba, K., Fransson, P. & Lindh, J. Use of a modified ESAS in cancer patients: a pilot study of patient and staff experiences. Int. J. Palliat. Nurs. 13, 610–616 (2007).
Rees, E., Hardy, J., Ling, J., Broadley, K. & A'Hern, R. The use of the Edmonton Symptom Assessment Scale (ESAS) within a palliative care unit in the UK. Palliat. Med. 12, 75–82 (1998).
Basch, E. et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J. Clin. Oncol. 34, 557–565 (2016).
Booth, C. M. & del Paggio, J. C. Approvals in 2016: questioning the clinical benefit of anticancer therapies. Nat. Rev. Clin. Oncol. 14, 135–136 (2017).
Mooney, K. H. et al. Automated home monitoring and management of patient-reported symptoms during chemotherapy: results of the symptom care at home RCT. Cancer Med. 6, 537–546 (2017).
Reese, J. B., Shelby, R. A. & Abernethy, A. P. Sexual concerns in lung cancer patients: an examination of predictors and moderating effects of age and gender. Support Care Cancer 19, 161–165 (2011).
Dupont, A. et al. Use of tablet personal computers for sensitive patient-reported information. J. Support Oncol. 7, 91–97 (2009).
Suh, S. Y., LeBlanc, T. W., Shelby, R. A., Samsa, G. P. & Abernethy, A. P. Longitudinal patient-reported performance status assessment in the cancer clinic is feasible and prognostic. J. Oncol. Pract. 7, 374–381 (2011).
Parker, C. et al. Alpha emitter radium-223 and survival in metastatic prostate cancer. N. Engl. J. Med. 369, 213–223 (2013).
Nilsson, S. et al. A randomized, dose-response, multicenter phase II study of radium-223 chloride for the palliation of painful bone metastases in patients with castration-resistant prostate cancer. Eur. J. Cancer 48, 678–686 (2012).
Quinten, C. et al. Baseline quality of life as a prognostic indicator of survival: a meta-analysis of individual patient data from EORTC clinical trials. Lancet Oncol. 10, 865–871 (2009).
Abernethy, A. P. et al. Rapid-learning system for cancer care. J. Clin. Oncol. 28, 4268–4274 (2010).
Etheredge, L. M. A rapid-learning health system. Health Aff. 26, w107–w118 (2007).
Olsen, L., Aisner, D. & McGinnis, J. M. (eds) The learning healthcare system: workshop summary (National Academies Press, 2007).
Abernethy, A. P. et al. Electronic patient-reported data capture as a foundation of rapid learning cancer care. Med. Care 48, S32–S38 (2010).
Cella, D. F. et al. The functional assessment of cancer therapy scale: development and validation of the general measure. J. Clin. Oncol. 11, 570–579 (1993).
Aaronson, N. K. et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J. Natl Cancer Inst. 85, 365–376 (1993).
Gershon, R. C., Rothrock, N., Hanrahan, R., Bass, M. & Cella, D. The use of PROMIS and assessment center to deliver patient-reported outcome measures in clinical research. J. Appl. Meas. 11, 304–314 (2010).
Pilkonis, P. A. et al. Item banks for measuring emotional distress from the Patient-Reported Outcomes Measurement Information System (PROMIS®): depression, anxiety, and anger. Assess 18, 263–283 (2011).
Kristjanson, L. J. Validity and reliability testing of the FAMCARE Scale: measuring family satisfaction with advanced cancer care. Soc. Sci. Med. 36, 693–701 (1993).
Peterman, A. H., Fitchett, G., Brady, M. J., Hernandez, L. & Cella, D. Measuring spiritual well-being in people with cancer: the functional assessment of chronic illness therapy — Spiritual Well-being Scale (FACIT-Sp). Ann. Behav. Med. 24, 49–58 (2002).
Bruera, E., Kuehn, N., Miller, M. J., Selmser, P. & Macmillan, K. The Edmonton Symptom Assessment System (ESAS): a simple method for the assessment of palliative care patients. J. Palliat. Care 7, 6–9 (1991).
Portenoy, R. K. et al. The Memorial Symptom Assessment Scale: an instrument for the evaluation of symptom prevalence, characteristics and distress. Eur. J. Cancer 30A, 1326–1336 (1994).
Cleeland, C. S. & Ryan, K. M. Pain assessment: global use of the Brief Pain Inventory. Ann. Acad. Med. Singapore 23, 129–138 (1994).
Revicki, D. A. et al. Development and psychometric analysis of the PROMIS pain behavior item bank. Pain 146, 158–169 (2009).
Amtmann, D. et al. Development of a PROMIS item bank to measure pain interference. Pain 150, 173–182 (2010).
Jeffery, D. D. et al. Initial report of the cancer Patient-Reported Outcomes Measurement Information System (PROMIS) sexual function committee: review of sexual function measures and domains used in oncology. Cancer 115, 1142–1153 (2009).
Buysse, D. J. et al. Development and validation of patient-reported outcome measures for sleep disturbance and sleep-related impairments. Sleep 33, 781–792 (2010).
Pereira, J. et al. Population-based standardized symptom screening: Cancer Care Ontario's Edmonton Symptom Assessment System and performance status initiatives. J. Oncol. Pract. 10, 212–214 (2014).
Nekolaichuk, C., Watanabe, S. & Beaumont, C. The Edmonton Symptom Assessment System: a 15-year retrospective review of validation studies (1991—2006). Palliat. Med. 22, 111–122 (2008).
Pereira, J. L. et al. Cancer care professionals' attitudes toward systematic standardized symptom assessment and the Edmonton Symptom Assessment System after large-scale population-based implementation in Ontario, Canada. J. Pain Symptom Manage. 51, 662–672 (2016).
El-Jawahri, A. et al. Effect of inpatient palliative care on quality of life 2 weeks after hematopoietic stem cell transplantation: a randomized clinical trial. JAMA 316, 2094–2103 (2016).
El-Jawahri, A. et al. Effect of inpatient palliative care during hematopoietic stem-cell transplant on psychological distress 6 months after transplant: results of a randomized clinical trial. J. Clin. Oncol. http://dx.doi.org/10.1200/JCO.2017.73.2800 (2017).
Temel, J. S. et al. Effects of early integrated palliative care in patients with lung and gi cancer: a randomized clinical trial. J. Clin. Oncol. 35, 834–841 (2017).
LeBlanc, T. W., Roeland, E. J. & El-Jawahri, A. Early palliative care for patients with hematologic malignancies: is it really so difficult to achieve? Curr. Hematol. Malig. Rep. 12, 300–308 (2017).
The work of T.W.LB. is supported by an American Cancer Society Mentored Research Scholar Grant (MRSG-15-185-01-PCSM) and a Cambia Health Foundation Sojourns Scholars Award.
T.W.LB. previously consulted for Flatiron Health, New York, New York, USA, and has served on an advisory board of the Cancer Support Community, Washington, District of Columbia, USA. A.P.A. is an employee of Flatiron Health.
About this article
Cite this article
LeBlanc, T., Abernethy, A. Patient-reported outcomes in cancer care — hearing the patient voice at greater volume. Nat Rev Clin Oncol 14, 763–772 (2017). https://doi.org/10.1038/nrclinonc.2017.153
COVID-19 Emergency and the Need to Speed Up the Adoption of Electronic Patient-Reported Outcomes in Cancer Clinical Practice
JCO Oncology Practice (2020)
Prospective development of a patient‐reported outcomes instrument for desmoid tumors or aggressive fibromatosis
Differences in Patient-Reported Outcomes That Are Most Frequently Detected in Randomized Controlled Trials in Patients With Solid Tumors: A Pooled Analysis of 229 Trials
Value in Health (2020)
American Journal of Evaluation (2020)
Electronic Patient-Reported Outcome–Based Interventions for Palliative Cancer Care: A Systematic and Mapping Review
JCO Clinical Cancer Informatics (2020)