Patient-reported outcomes (PROs) are used in clinical trials to provide evidence of the benefits and risks of interventions from a patient perspective and to inform regulatory decisions and health policy. The collection of PROs in routine practice can facilitate monitoring of patient symptoms; identification of unmet needs; prioritisation and/or tailoring of treatment to the needs of individual patients and inform value-based healthcare initiatives. However, respondent burden needs to be carefully considered and addressed to avoid high rates of missing data and poor reporting of PRO results, which may lead to poor quality data for regulatory decision making and/or clinical care.
Patient-reported outcomes (PROs) may be defined as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else.”1 PROs can provide valuable evidence of the physical and psychosocial impact of disease and treatment on patients’ health-related quality of life (HRQOL) and symptoms2. PROs are increasingly used in clinical trials to provide evidence of the benefits and risks of treatments from a patient perspective and to inform regulatory decisions and health policy. The routine collection of PROs, using questionnaires known as patient-reported outcome measures (PROMs), can assist clinicians with the monitoring of patient symptoms; the identification of unmet needs and concerns; the prioritisation and/or the tailoring of treatment to the needs of individual patients and in value-based healthcare initiatives3.
Millions of individuals complete PROMs worldwide yearly in a variety of settings. The potential burden of completing PROMs must be considered alongside their benefits. Respondent burden is the degree to which a survey respondent perceives their participation in the project as difficult, time-consuming, or emotionally stressful4. The need to justify the benefits of research against the burden and risk is an important ethical consideration5,6,7.
Concerns have been raised about the burden to respondents in completing such measures in clinical trials1,8,9 and routine clinical management10,11,12. A recent review of randomised control trials (RCTs) of ovarian cancer reported that compliance rates for PROs were poor overall with levels of preventable missing PRO data ranged from 17% to 41% in the included trials13. These poor compliance rates may be due to various reasons including respondent burden. If the issue of respondent burden is not addressed, we risk high rates of missing data and poor reporting of PRO results, meaning poor quality data to inform regulatory decision-making or clinical care14. In addition, missing data may be due to various factors, for instance, completion of PROMs may vary depending on ethic/socioeconomic backgrounds15. There is a risk that such differences may lead to biased interpretations of trial results or treatment effects in clinical practice.
PRO data from clinical research is frequently underreported or not reported at all, which is unethical16,17. A recent study evaluated 160 cancer trials and found that the PRO data of nearly 50,000 participants were never published17. If PROs are selected as primary or key secondary outcomes for clinical trials, the findings should be published in the main trial publication. However, if exploratory outcomes the PRO findings may be published in the secondary publication or as supplementary data for the main publication18. Furthermore, participant burden may vary across populations, with skilled individuals and those with access to digital technologies being more likely to be able to complete PROMs15,19. Failure to address issues of respondent burden whether in relation to electronic or paper PROMs, may further increase health inequalities and risk poorer care in under-served groups15,19. For instance, individuals with low literacy levels or cognitive impairment, may find the completion of PROMs burdensome and withdraw from completing the measures15. As a result, these individuals may be unable to derive the potential benefits of PRO collection and utilisation in clinical research and routine practice.
Here we identify and highlight PRO-specific issues pertaining to a respondent burden for consideration when planning PRO collection in clinical trials and routine clinical practice.
Issues pertaining to respondent burden in PRO collection for clinical trials and routine practice
These include issues relating to the rationale for PRO assessment, measure selection and delivery, patient involvement and engagement (PPIE) in questionnaire development and study design, and the perception and assessment of respondent burden (Fig. 1).
Rationale for PRO assessment
Clearly define research objective(s)
The need for clearly defined research objective(s) for PRO data collection, which should inform the selection of the most appropriate PROMs, was highlighted by Shetty et al.20. It was also recommended that the formulation of these objectives and subsequent selection of measures should consider patients’ physical and mental capacity and potential respondent burden20,21. The collection of PRO data for trials as well as routine care should be evidence-informed to ensure that the data collection justifies the burden and potential risks22,23.
Measuring relevant/useful concepts
The importance of measuring PRO concepts that are relevant and useful to the target patient population was emphasised by Retzer et al.22. Static PROMs may include questions that do not apply to sub-groups of patients within a target population11. For instance, patients with prostate cancer on hormonal therapy often experience hot flashes and/or breast enlargement which do not occur in those managed conservatively24. Vickers et al. reported patients conservatively managed for prostate cancer were asked to complete the Expanded Prostate Cancer Index Composite (EPIC) questionnaire, sometimes left comments such as “I’m not a woman you know” against questions about hot flashes or breast enlargement24. There is a potential risk that asking patients to complete irrelevant questions may lead to disengagement and perception of PRO collection as burdensome25. In addition, as new therapies are introduced, trialists and clinicians may find legacy measures that have been used for several years in drug development are inadequate for capturing all important symptomatic side effects and other relevant patient experiences22. For example, legacy measures do not assess symptomatic side effects from immunotherapy in cancer, which is a limitation for the assessment of the patient experience26. Therefore, PROs selected for use in clinical trials and routine practice should be re-evaluated regularly to ensure they remain relevant and appropriate. Possible solutions for addressing the issue of relevance include combining existing measures with new items, the use of item banks or, potentially, the development of new measures26.
A study that utilised cognitive debriefing to evaluate PROMs in patients with multiple sclerosis found items that required the recall of the frequency of an event or symptom (such as five times per week), which then required categorisation in an ordinal scale, was particularly burdensome for respondents as they required greater cognitive effort27. The importance of choosing appropriate recall periods, in terms of the potential burden it can place on respondents, was discussed by a few studies28,29. Respondents need to be able to accurately recall changes in their health status without undue strain.
Differences in response burden may also depend on the perceived difficulty of questionnaire completion due to the cognitive requirements of the measure and how unwell the patient may feel29. Patients with cognitive impairment are more likely to report a higher response burden when completing PRO measures29. In addition, recall periods that are too short may underestimate symptom burden in conditions where symptoms fluctuate diurnally or on a day-to-day basis while recall intervals that are too long may either over- or underestimate patients’ symptoms28. Norquist et al. recommended that decisions on the recall period should consider respondent burden alongside other key determinants including the purpose and intended use of the measure, disease characteristics, and the treatment being investigated28.
The literacy level required to understand and complete PROMs may also influence respondent burden30. It should not be assumed that because patients are engaged in a clinical situation that medical terms can be used and will be understood by patients. It is generally recommended that items be formulated at the sixth-grade reading level or lower; depending on the intended target population and should be justified30.
Number of measures to administer
Often, a single PROM may not capture all the concepts of interest and so may not provide all the vital PRO data required to address all the research questions in a trial or in routine practice31. Therefore, it may be necessary to administer more than one PROM. However, trialists and clinical teams need to be careful as the utilisation of multiple PROMs will increase the time required to provide PRO data and may lead to an increase in patient burden22,32,33. Therefore, it is important that trialists and clinical teams carefully balance the quantity and quality of data desired, PROM coverage/comprehensiveness and precision against the time requirement and potential respondent burden (including anxiety caused, and fatigue)23,34,35. This is important as any increase in patient burden may lead to low compliance, and issues with the integrity and validity of data22.
Historically, PROs have relied on long questionnaires which may be more precise but more time-consuming for patients, which are potentially burdensome and may affect compliance11,23,36,37. For instance, the original Kidney Disease Quality of Life (KDQOL) questionnaire has 134 items. A shorter 80-item version was developed soon after and a much shorter 36-item version was produced to further reduce potential respondent burden38. However, a review and meta-analysis showed that the length of a given PRO questionnaire may not necessarily be associated with participant response burden29,39. Furthermore, a feasibility study that used the entire 80 items of the Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) Item Library, instead of using a selection of items, reported a high compliance rate40. The study suggested that trialists should consider using the entire library, especially for trials of experimental drugs to ensure they adequately capture potential adverse events40. Bragstad et al. also highlighted that one challenge with using different short versions of a measure may be the difficulty of comparing the scores of the various versions with each other41. Finally, it was noted that while brevity should be a consideration, it should not outweigh the need to assess outcomes that participants consider important23.
Despite evidence that the length of the PRO questionnaire may not lead to a burden or affect compliance rates, some authors suggest keeping PRO data collection as brief as possible, without compromising on reliability and validity, especially when patients are very ill or when their condition causes fatigue, lack of energy or tiredness41,42,43. Patient’s health status and functional disability should be considered during the PROM selection process to ensure that length and content are tailored to limit response burden44,45. Several studies focused on the development of short forms of existing measures and found that this enhanced the feasibility of collecting PROs on a regular basis and potentially could reduce respondent burden23,36,41,46,47,48,49,50,51,52,53,54,55. The use of statistical methods such as factor analysis and Rasch can assist with item reduction during the validation of PROMs.
Mode(s) of delivery
Modes of PRO delivery may include paper, smartphone applications, web-based completion, telephone interviews, interactive voice response or audio-computer-assisted interviews10,22,42,43. The electronic collection of PROMs (ePROMs) was recommended as an acceptable mode of delivery by several authors10,34,42,56,57. A study reported that the use of ePROMs could minimise respondent burden and improve compliance56. The majority of participants preferred physician-provided devices while the paper format was the least preferred mode of delivery56. Additionally, the majority of those participants who opted for paper did not own an electronic device (smartphone or computer). Another study reported that very few older men enroled for a trial chose ePROMs and opted for paper formats instead22. Therefore, irrespective of the mode of delivery consideration should be given to participants’ literacy, ability to utilise technology, cultural and personal needs43. It was suggested that patients/study participants should be given a range of modes to choose from10,57 as preferences may be determined by various factors including age, computer literacy, access to the internet and electronic devices, and language difficulties all of which may contribute to respondent burden10,43. Conversely, the current ISPOR recommendation discourages mixing of paper and electronic PRO collection modes and suggest that mixing of electronic modes may be considered for clinical trials only after equivalence has been established58. This issue will be an important consideration/discussion going forward in an era where decentralised trials and BYOD (bring your own devices) are coming to the fore.
The potential impact of developing and collecting ePROMs rather than relying entirely on the traditional paper and pen mode was explored by Retzer et al22. Recent technological advances have facilitated the development of ePROMs which were reported as easier and faster for patients to complete thus reducing respondent burden and in turn enhancing compliance with PRO collection59.
A study by Dumais et al. suggested various ePROM features which may enhance compliance by encouraging patient engagement with PRO collection. Increasing engagement could potentially minimise the perception of respondent burden and vice versa56. These features include (1) an estimation of ePROM completion time at the outset, (2) the ability to track progress using a progress bar or indicator, (3) graphical presentation of PRO data to personally track their health status, (4) positive messaging to encourage completion when needed, and (5) a thank you message after completion of a daily diary. The study also reported a preference by participants for navigation buttons to be placed at the bottom of the screen and one question presented per screen page as this was considered easier to read56. Conversely, a systematic review suggested that presenting one question per screen page may be burdensome for respondents as this format may require more time to click through all the pages to the end of the questionnaire45. Work with patient partners and other stakeholders can help in choosing the appropriate PROM functionality for the target population.
Computerised adaptive testing (CAT)
CAT, which is based on the principles of item response theory, was proposed by several studies as a potential approach for tailoring ePROMs to the individual respondent thereby addressing respondent burden due to the presence of irrelevant questions within a questionnaire11,37,45,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82. Most of the studies utilised the approach in conjunction with the PROMIS item banks.
The use of CAT could enhance the efficiency of PRO collection and significantly reduce the number of questions patients need to answer thus decreasing the time required for completion, without affecting precision or validity60,63,70,73,76,77,80. Unlike static forms such as the Hip Disability and Osteoarthritis Outcome Score-Physical Function Short form (HOOS-PS) which requires patients to answer a question about running even if a patient indicates issues with getting in/out of the bath55.
Schedule of assessments
The schedule of assessments will vary substantially by study but should consider participant burden, whilst maximising clinically relevant data to address the research question or clinical use83. A qualitative study found no consensus among patients and clinicians on the optimal frequency of administration of PROMs for the routine management of patients with chronic kidney disease10. However, interviewees generally believed it would be burdensome if an ePROM was administered more than once a month.
Some studies suggest that brief PROMs for symptom assessment sometimes administered as daily diaries may be burdensome for patients especially those with chronic conditions that require long-term monitoring32,84. or who may not experience a significant day-to-day variation of symptoms28. Furthermore, the power trialists hope to gain from the additional data obtained from daily administration might be compromised by high levels of missing data potentially due to respondent burden28,32,84. Therefore, fewer PRO assessment time points were recommended to minimise respondent burden and optimize resource usage8,32,45. Trialists should also consider the nature of the condition and pharmacodynamics of drug interventions when deciding the schedule of PRO assessments42. It may be that not all PROMs need to be administered at every time point, depending on the PRO objective. For example, measures of symptom severity could be administered more frequently than measures capturing function or health-related quality of life.
For the routine clinical management of patients, Aiyegbusi et al. suggested that the schedule of assessment would depend on disease trajectory in individual patients, with stable patients requiring less frequent administration10. However, deteriorating patients, who may require closer monitoring, might be unwilling to complete PROMs at the level of frequency required due to the burden of illness10. It was further suggested that patients should have the option of completing ad-hoc ePROMs if they felt unwell rather than wait for the next scheduled assessment10.
Mercieca-Bebber et al recommended that PRO assessments in routine clinical practice should coincide with patients’ clinic visits so that clinicians have the results to review before hospital appointments45. The importance of minimising institution/staff burden was also highlighted as an overly burdensome PRO assessment schedule for staff may lead to a reduction in their engagement with the administration and monitoring of compliance rates, thus potentially resulting in high rates of missing data45.
Support for completion
Aiyegbusi et al suggested that PROMs administration for routine clinical practice was best completed at home prior to clinical appointments away from the pressures of busy clinics and potential interference from medical personnel10. Although remote/home completion of PROs may be more relevant for routine practice and less so for clinical trials, the growing interest in the use of digitally-enabled decentralised clinical trial (DCT) designs means that remote collection of PROs may become more relevant for clinical trials in future85. In clinical research settings, where patients complete paper versions of PROs at home, the provision of postage-paid, self-addressed envelopes may enable the easier return of the questionnaires. However, posting these back to researchers and clinical teams may still be burdensome for some respondents especially those in rural areas45. If assessments were to occur in routine clinic/outpatient settings then support for example, provision of childcare (to facilitate completion of PROMs without distractions) and travel assistance (to ensure patients attend their appointments and arrive on time to so they have enough time to complete their PROMs/ePROMs without being rushed) have been suggested45.
Patients may not be able to complete PROMs, often through cognitive impairment, or ill-health. In these situations, reports from proxies (typically family or other carers) may be used. Proxy reporting is a longstanding consideration in palliative care research86 as well as childhood health research87. However, regulatory agencies such as the FDA1 and EMA9 discourage the use of proxy reports, citing concerns about disparities between proxy and patient reports1,9. Nonetheless, in other situations such as PRO data collection for registries88 in routine clinical paediatric care89, the anticipated need for proxy reporting is recognised. The need for clear guidance on proxy reporting has been noted in palliative care86, and there is some guidance for the use of self- or proxy reporting in children and adolescents90. Additional work is required to optimise the use of proxy-reported data91 as well as adapt existing PROMs or develop new ones for people who have difficulties completing the currently available PROMs (e.g., develop versions with pictures, icons or emoticons).
Patient involvement in questionnaire development and trial design
During questionnaire development, interviews or focus groups with patients who have lived experiences of the condition of interest can help determine the most appropriate recall intervals for that patient population given the intended use of the PRO measure28. In addition, involving patients in cognitive debriefing during the questionnaire, validation could help identify issues with items or formatting which may lead to respondent burden if not addressed45.
A patient interviewee who participated in a qualitative study that explored PRO collection and reporting in oncology trials, believed patients need to be involved and share the responsibility for PRO decisions and made to feel like members of the research team22,92. Patients can provide feedback on the acceptability and relevance of PRO measures, suitability of assessment time points in capturing desired outcomes, respondent burden, strategies to educate and engage participants, and many other important aspects of study design45,52. A study noted that increased consultation with patients at the design phase of clinical trials could lead to the development of a schedule of PRO assessments which may be more relevant for patients and improve completion rates but may not be aligned with clinic visits21. However, as previously mentioned, a qualitative study reported that patients were generally in support of aligning the schedule of assessment to clinic visits10. Engaging patients and clinicians in study design can help identify a clinically relevant assessment schedule that minimises burden.
Perception and assessment of respondent burden
Factors that may influence the perception of burden
Patient comprehension of the purpose of PRO assessment in the context of a trial was mentioned as a potential influence on their perception of burden42. Therefore, trialists need to ensure that patients understand the link between their PRO assessments and the questions the trial is trying to answer42. Patients also need to understand that substantial missing PRO data would lead to difficulties analysing and interpreting the data which may mean that no useful conclusions can be drawn22. These views were corroborated by the qualitative study by Aiyegbusi et al. which reported that patients believed that the provision of explanations of the significance of questions and results would encourage them to complete PROMs on a regular basis for routine clinical care10. Poor compliance rates may also occur due to a lack of feedback and utilisation of PROMs results in routine clinical practice. If patients complete PROMs, but the results are not discussed with them or are not seen to be utilised for their care, patients may lose the motivation to complete PROMs over time10,35,93.
In clinical trials and routine clinical practice, trialists and clinicians often worry that patients are too ill and so perceive the completion of PROMs as an extra burden that patients may struggle with. The view that a high degree of respondent burden exists (in the absence of any evidence, and often without feedback from patients) might be a reason why some trialists and clinicians are reluctant to facilitate PRO assessments in clinical trials or routine practice10. This reluctance could lead to the suboptimal collection of relevant PROs and result in missed opportunities to understand the impact of disease and treatment, which could hamper the design of future trials and routine care. Understanding and implementation of mitigation strategies for respondent burden will ensure that opportunities for collecting relevant and useful data are not missed.
Other factors which may influence the perceived burden of PROs by clinicians include administration requirements, scoring, and how readily the PRO information can be used to inform clinical decision-making as well as the potential impact on clinic workflow is an important consideration, and IT requirements30.
Assessment of respondent burden
A few studies discussed the assessment of respondent burden specific to PROMs29,94,95. The number of items could be an indicator for respondent burden95. Bryan et al.94. mentioned the assessment of respondent burden based on (1) number of items; (2) word count; (3) time for completion (minutes); (4) Flesch–Kincaid grade level (i.e., a measure of readability used to determine how difficult is a text to read).
Atkinson et al. developed the Response Burden Questionnaire29, a 6-item measure that assesses how patients perceive: (a) how well the questions related to their actual concerns, (b) how comfortable they were with answering the questions, (c) how well the interview characterised their health and well-being, d) the length of time required to complete the questionnaires, (e) whether questions seemed unimportant or repetitive, and (f) what additional information should have been gathered29.
Increasingly, research ethics committees and institutional review boards, in their assessment of PRO research, are considering the burden of PRO collection on study participants96. An objective assessment of the potential burden could assist such committees in their decision-making process.
Key considerations to address respondent burden
Considering and adequately addressing respondent burden relating to PRO collection may ensure that long-term follow-up of patients can be conducted for clinical research and still obtain high-quality data. Avoiding unnecessary burdens is important for integrating PROs into clinical decision-making in routine clinical practice. Efforts should be made to ensure that patients are able to complete PROMs, missing PRO data are minimised and appropriate technology is harnessed to ensure that the data is incorporated effectively and efficiently with existing clinical workflows. It is the responsibility of researchers and clinicians to minimise respondent burden that may arise from completing PROMs.
Key considerations include:
Having a clearly defined rationale in terms of research objective(s) for PRO data collection should inform the selection of the most appropriate PROMs20,21.
Capturing outcomes that matter to the patient population and involving patients in the selection of measures may mean respondents are willing to complete lengthier measures10,11,22,42.
When selecting PROMs, the characteristics of candidate measures should be considered as these may contribute to potential respondent burden. Various issues as highlighted in the previous sections need to be carefully considered4,22,23,24,27,28,29,30,32,33,34,37,41,42,43,44,45.
The delivery of PROMs may influence respondent burden. The increasing uptake of technological innovations such as smartphones has facilitated the development and acceptability of ePROMs. A key advantage of utilising ePROMs is that valuable features, impossible with the paper format, can be provided to patients which may help reduce respondent burden and enhance compliance56. The collection of ePROMs could also reduce administrative burden and enhance the integration of PROs with existing clinical workflow59. However, consideration needs to be given to how to effectively integrate ePROMs into existing workflows and patients should still be given the option of other modes including paper format10,22,34,42,43,56,57.
Computer adaptive testing is a potential approach for tailoring ePROMs to the individual respondent97, thereby eliminating the burden due to the presence of irrelevant questions within a questionnaire37,45,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77.
The schedule of assessment needs to be considered. While the general view is to have fewer assessment timepoints and to align assessments with clinic/study visits10,32,45, it is important that patients and clinicians are involved in these decisions. The nature of the condition and effects of drug interventions should also be taken into consideration28,32,42,84.
Early patient involvement and engagement can inform the selection and delivery of PROMs that are acceptable and pose minimal risk of undue burden to trial participants and patients in routine practice10,21,22,28,45,52. There is also a need to provide adequate information to patients about the value of PROs in terms of the aims of a research study or routine clinical practice and the implications of incomplete or missing data10,22,42. Having this information might provide the motivation patients need to complete PROMs on a regular basis for routine clinical care10. Recently, there has been interest in developing ways to measure research participation burden98,99,100. The respondent burden can be assessed regularly and necessary steps are taken to address it29,94,95.
FDA. Guidance for Industry. Patient-reported Outcome Measures: Use in Medicinal Product Development to Support Labeling Claims (US Department of Health and Human Services Food and Drug Administration, Silver Spring, MD, 2009).
Wilson, R. Patient led PROMs must take centre stage in cancer research. Res. Involv. Engagem. 4, 7 (2018).
Stevenson, D. A. & Carey, J. C. Health-related quality of life measures in genetic disorders: an outcome variable for consideration in clinical trials. Am. J. Med. Genet. Part C Semin. Med. Genet. 151c, 255–260 (2009).
Encyclopedia of Survey Research Methods. (2008).
World Medical Association Declaration of Helsinki. Ethical principles for medical research involving human subjects. JAMA 310, 2191–2194 (2013).
NHMRC. National Statement on Ethical Conduct in Human Research (National Health and Medical Research Council, Department of Health, Australia, 2007 (updated 2018)).
NIH. Guiding Principles for Ethical Research (NIH, 2016).
FDA. Core Patient-Reported Outcomes in Cancer Clinical Trials Guidance for Industry (US Department of Health and Human Services Food and Drug Administration, Silver Spring, MD, 2021).
EMA. Appendix 2 to the Guideline on the Evaluation of Anticancer Medicinal Products in Man—the Use of Patient-reported Outcome (PRO) Measures in Oncology Studies (European Medicines Agency, Committee for Medicinal Products for Human Use (CHMP), 2016).
Aiyegbusi, O. L. et al. Patient and clinician perspectives on electronic patient-reported outcome measures in the management of advanced CKD: a qualitative study. Am. J. Kidney Dis. 74, 167–178 (2019).
Snyder, C. F., Jensen, R. E., Geller, G., Carducci, M. A. & Wu, A. W. Relevant content for a patient-reported outcomes questionnaire for use in oncology clinical practice: putting doctors and patients on the same page. Qual. Life Res. 19, 1045–1055 (2010).
Bingham, C. O. III et al. Montreal Accord on Patient-Reported Outcomes (PROs) use series—Paper 4: patient-reported outcomes can inform clinical decision making in chronic care. J. Clin. Epidemiol. 89, 136–141 (2017).
Mercieca-Bebber, R. et al. A systematic evaluation of compliance and reporting of patient-reported outcome endpoints in ovarian cancer randomised controlled trials: implications for generalisability and clinical practice. J. Patient Rep. Outcomes 1, 5 (2017).
Bell, M. L. & Fairclough, D. L. Practical and statistical issues in missing data for longitudinal patient-reported outcomes. Stat. Methods Med. Res. 23, 440–459 (2014).
Calvert, M. J. et al. Patient reported outcome assessment must be inclusive and equitable. Nat. Med. 28, 1120–1124 (2022).
Cruz Rivera, S. et al. The need for ethical guidance for the use of patient-reported outcomes in research and clinical practice. Nat. Med. 27, 572–573 (2021).
Kyte, D. et al. Systematic evaluation of patient-reported outcome protocol content and reporting in cancer trials. J. Natl Cancer Inst. 111, 1170–1178 (2019).
Calvert, M. et al. Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension. JAMA 309, 814–822 (2013).
Sisodia, R. C., Rodriguez, J. A. & Sequist, T. D. Digital disparities: lessons learned from a patient reported outcomes program during the COVID-19 pandemic. J. Am. Med. Inf. Assoc. 28, 2265–2268 (2021).
Shetty, P. N., Hawken, J., Sanghavi, K. K. & Giladi, A. M. Correlation of patient-reported outcomes measurement information system questionnaires with the brief Michigan hand questionnaire in patients with 5 common hand conditions. J. Hand Surg. Am. 46, 709.e701–709.e711 (2021).
King-Kallimanis, B. L. et al. Patient-reported outcomes after treatment discontinuation: commercial clinical trial data from four cancer types. Value Health 24, 1302–1307 (2021).
Retzer, A. et al. International perspectives on suboptimal patient-reported outcome trial design and reporting in cancer clinical trials: a qualitative study. Cancer Med. 10, 5475–5487 (2021).
Ettridge, K. et al. A randomised online experimental study to compare responses to brief and extended surveys of health-related quality of life and psychosocial outcomes among women with breast cancer. Qual. Life Res. 30, 407–423 (2021).
Vickers, A. J. & Chen, L. Y. Manifesto: towards a clinically-oriented psychometrics. Health Qual. Life Outcomes 15, 83 (2017).
Schwartz, C. E. et al. Responsiveness of standard spine outcome tools: do they measure up? J. Neurosurg. Spine 21, 1–8 (2020).
King-Kallimanis, B. L. et al. Patient reported outcomes in anti-PD-1/PD-L1 inhibitor immunotherapy registration trials: FDA analysis of data submitted and future directions. Clin. Trials (Lond., Engl.) 16, 322–326 (2019).
Ploughman, M., Austin, M., Stefanelli, M. & Godwin, M. Applying cognitive debriefing to pre-test patient-reported outcomes in older people with multiple sclerosis. Qual. life Res. 19, 483–487 (2010).
Norquist, J. M., Girman, C., Fehnel, S., DeMuro-Mercon, C. & Santanello, N. Choice of recall period for patient-reported outcome (PRO) measures: criteria for consideration. Qual. life Res. 21, 1013–1020 (2012).
Atkinson, T. M. et al. Perceptions of response burden associated with completion of patient-reported outcome assessments in oncology. Value Health 22, 225–230 (2019).
Francis, D. O., McPheeters, M. L., Noud, M., Penson, D. F. & Feurer, I. D. Checklist to operationalize measurement characteristics of patient-reported outcome measures. Syst. Rev. 5, 129–129 (2016).
Martin, R. & Disantis, A. Editorial Commentary: patient-reported outcome measures with established clinically important outcome values should be used for hip arthroscopy patients: sifting through the white noise. Arthroscopy 37, 3122–3124 (2021).
Xu, J., Evans, T. J., Coon, C., Copley-Merriman, K. & Su, Y. Measuring patient-reported outcomes in advanced gastric cancer. Ecancermedicalscience 7, 351 (2013).
Miki, T., Kondo, Y., Takebayashi, T. & Takasaki, H. Difference between physical therapist estimation and psychological patient-reported outcome measures in patients with low back pain. PLoS ONE 15, e0227999 (2020).
Jhamb, M. et al. Design and rationale of health-related quality of life and patient-reported outcomes assessment in the frequent hemodialysis network trials. Blood Purif. 31, 151–158 (2011).
Girgis, A., Durcinoska, I., Arnold, A. & Delaney, G. P. Interpreting and acting on the PRO scores from the Patient-reported Outcomes for Personalized Treatment and Care (PROMPT-Care) eHealth system. Med. Care 57, S85–S91 (2019).
Kuijlaars, I. A. R. et al. Shortening the Haemophilia Activities List (HAL) from 42 items to 18 items. Haemophilia 27, 1062–1070 (2021).
Squires, M. D. et al. Physical function computer adaptive test outcomes in diabetic lumbar spine surgical patients. Spine J. 19, 1048–1056 (2019).
Peipert, J. D., Nair, D., Klicko, K., Schatell, D. R. & Hays, R. D. Kidney Disease Quality of Life 36-Item Short Form Survey (KDQOL-36) normative values for the United States Dialysis Population and New Single Summary Score. J. Am. Soc. Nephrol. 30, 654 (2019).
Rolstad, S., Adler, J. & Rydén, A. Response burden and questionnaire length: is shorter better? A review and meta-analysis. Value Health 14, 1101–1108 (2011).
Shepshelovich, D. et al. Feasibility assessment of using the complete Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) Item Library. Oncologist 24, e146–e148 (2019).
Bragstad, L. K. et al. Psychometric properties of a short version of Lee Fatigue Scale used as a generic PROM in persons with stroke or osteoarthritis: assessment using a Rasch analysis approach. Health Qual. Life Outcomes 18, 168 (2020).
Turner, R. R., Quittner, A. L., Parasuraman, B. M., Kallich, J. D. & Cleeland, C. S. Patient-reported outcomes: instrument development and selection issues. Value Health 10, S86–S93 (2007).
Trick, W. E., Deamant, C., Smith, J., Garcia, D. & Angulo, F. Implementation of an audio computer-assisted self-interview (ACASI) system in a general medicine clinic: patient response burden. Appl. Clin. Inf. 6, 148–162 (2015).
Jensen, R. E. et al. Validation of the PROMIS physical function measures in a diverse US population-based cohort of cancer patients. Qual. Life Res. 24, 2333–2344 (2015).
Mercieca-Bebber, R. et al. Design, implementation and reporting strategies to reduce the instance and impact of missing patient-reported outcome (PRO) data: a systematic review. BMJ Open 6, e010938 (2016).
Choi, S. W., Reise, S. P., Pilkonis, P. A., Hays, R. D. & Cella, D. Efficiency of static and computer adaptive short forms compared to full-length measures of depressive symptoms. Qual. Life Res. 19, 125–136 (2010).
Clover, K. et al. PROMIS depression measures perform similarly to legacy measures relative to a structured diagnostic interview for depression in cancer patients. Qual. Life Res. 27, 1357–1367 (2018).
Jenkinson, C., Fitzpatrick, R., Crocker, H. & Peters, M. The Stroke Impact Scale: validation in a UK setting and development of a SIS short form and SIS index. Stroke 44, 2532–2535 (2013).
Kumlien, C., Nordanstig, J., Lundström, M. & Pettersson, M. Validity and test retest reliability of the vascular quality of life Questionnaire-6: a short form of a disease-specific health-related quality of life instrument for patients with peripheral arterial disease. Health Qual. Life Outcomes 15, 187 (2017).
Scuderi, G. R. et al. The Knee Society Short Form reduces respondent burden in the assessment of patient-reported outcomes. Clin. Orthop. Relat. Res. 474, 134–142 (2016).
Snell, D. L., Siegert, R. J., Surgenor, L. J., Dunn, J. A. & Hooper, G. J. Evaluating quality of life outcomes following joint replacement: psychometric evaluation of a short form of the WHOQOL-Bref. Qual. Life Res. 25, 51–61 (2016).
Skevington, S. M. & McCrate, F. M. Expecting a good quality of life in health: assessing people with diverse diseases and conditions using the WHOQOL-BREF. Health Expect. 15, 49–62 (2012).
Cook, K. F. et al. Evaluation of the Validity and Response Burden of Patient Self-Report Measures of the Pain Assessment Screening Tool and Outcomes Registry (PASTOR). Mil. Med. 182, e1851–e1861 (2017).
Gandek, B., Roos, E. M., Franklin, P. D. & Ware, J. E. Jr. A 12-item short form of the Knee injury and Osteoarthritis Outcome Score (KOOS-12): tests of reliability, validity and responsiveness. Osteoarthr. Cartil. 27, 762–770 (2019).
Gandek, B., Roos, E. M., Franklin, P. D. & Ware, J. E. Jr. A 12-item short form of the Hip disability and Osteoarthritis Outcome Score (HOOS-12): tests of reliability, validity and responsiveness. Osteoarthr. Cartil. 27, 754–761 (2019).
Dumais, K. M. et al. Preferences for use and design of electronic patient-reported outcomes in patients with chronic obstructive pulmonary disease. Patient 12, 621–629 (2019).
Wall, L. B., Vuillermin, C., Miller, P. E., Bae, D. S. & Goldfarb, C. A. Convergent validity of PODCI and PROMIS domains in congenital upper limb anomalies. J. Hand Surg. Am. 45, 33–40 (2020).
Eremenco, S. et al. PRO data collection in clinical trials using mixed modes: report of the ISPOR PRO mixed modes good research practices task force. Value Health 17, 501–516 (2014).
Aiyegbusi, O. L., Nair, D., Peipert, J. D., Schick-Makaroff, K. & Mucsi, I. A narrative review of current evidence supporting the implementation of electronic patient-reported outcome measures in the management of chronic diseases. Ther. Adv. Chronic Dis. 12, 20406223211015958 (2021).
Ottenhof, M. J. et al. Applying computerized adaptive testing to the FACE-Q skin cancer module: individualizing patient-reported outcome measures in facial surgery. Plast. Reconstr. Surg. 148, 863–869 (2021).
Geerards, D., Pusic, A., Hoogbergen, M., van der Hulst, R. & Sidey-Gibbons, C. Computerized quality of life assessment: a randomized experiment to determine the impact of individualized feedback on assessment experience. J. Med. Internet Res. 21, e12212 (2019).
Smith, A. B., Hanbury, A. & Retzler, J. Item banking and computer-adaptive testing in clinical trials: standing in sight of the PROMISed land. Contemp. Clin. Trials Commun. 13, 005–005 (2018).
Kaat, A. J. et al. Longitudinal validation of the PROMIS physical function item bank in upper extremity trauma. J. Orthop. Trauma 31, e321–e326 (2017).
Morris, S., Bass, M., Lee, M. & Neapolitan, R. E. Advancing the efficiency and efficacy of patient reported outcomes with multivariate computer adaptive testing. J. Am. Med. Inform. Assoc. 24, 897–902 (2017).
Katzan, I. L., Fan, Y., Uchino, K. & Griffith, S. D. The PROMIS physical function scale: a promising scale for use in patients with ischemic stroke. Neurology 86, 1801–1807 (2016).
Harle, C. A. et al. Overcoming barriers to implementing patient-reported outcomes in an electronic health record: a case report. J. Am. Med Inf. Assoc. 23, 74–79 (2016).
Bass, M., Morris, S. & Neapolitan, R. Utilizing multidimensional computer adaptive testing to mitigate burden with patient reported outcomes. AMIA Annu. Symp. Proc. 2015, 320–328 (2015).
Baum, G., Basen-Engquist, K., Swartz, M. C., Parker, P. A. & Carmack, C. L. Comparing PROMIS computer-adaptive tests to the Brief Symptom Inventory in patients with prostate cancer. Qual. life Res. 23, 2031–2035 (2014).
Kane, L. T. et al. Use of computerized adaptive testing to develop more concise patient-reported outcome measures. JB JS Open Access 5, e0052 (2020).
Dvorzhinskiy, A. et al. The performance of PROMIS computer adaptive testing for patient-reported outcomes in hip fracture surgery: a pilot study. Arch. Orthop. Trauma Surg. 142, 417–424 (2021).
Zheng, Y., Chang, C. H. & Chang, H. H. Content-balancing strategy in bifactor computerized adaptive patient-reported outcome measurement. Qual. life Res. 22, 491–499 (2013).
Weiss, D. J., Wang, C., Cheville, A. L., Basford, J. R. & DeWeese, J. Adaptive measurement of change: a novel method to reduce respondent burden and detect significant individual-level change in patient-reported outcome measures. Arch. Phys. Med. Rehabil. 103, S43–S52 (2021).
Kosinski, M., Bjorner, J. B., Ware, J. E. Jr, Sullivan, E. & Straus, W. L. An evaluation of a patient-reported outcomes found computerized adaptive testing was efficient in assessing osteoarthritis impact. J. Clin. Epidemiol. 59, 715–723 (2006).
Devine, J. et al. Evaluation of Computerized Adaptive Tests (CATs) for longitudinal monitoring of depression, anxiety, and stress reactions. J. Affect. Disord. 190, 846–853 (2016).
Rose, M., Bjorner, J. B., Becker, J., Fries, J. F. & Ware, J. E. Evaluation of a preliminary physical function item bank supported the expected advantages of the Patient-Reported Outcomes Measurement Information System (PROMIS). J. Clin. Epidemiol. 61, 17–33 (2008).
Kocalevent, R. D. et al. An evaluation of patient-reported outcomes found computerized adaptive testing was efficient in assessing stress perception. J. Clin. Epidemiol. 62, 278–287.e271-273 (2009).
Plummer, O. R. et al. A concise shoulder outcome measure: application of computerized adaptive testing to the American Shoulder and Elbow Surgeons Shoulder Assessment. J. Shoulder Elb. Surg. 28, 1273–1280 (2019).
Niazi, S. K. et al. Feasibility study of three-phase implementation of International Consortium for health outcomes measurement depression and anxiety standard set in an outpatient consultation-liaison psychiatry practice. Psychosomatics 61, 8–18 (2020).
Deutscher, D. et al. Upper quadrant edema patient-reported outcome measure is reliable, valid, and efficient for patients with lymphatic and venous disorders. Phys. Ther. 101, pzab219 (2021).
Young-Afat, D. A. et al. Introducing BREAST-Q computerized adaptive testing: short and individualized patient-reported outcome assessment following reconstructive breast surgery. Plast. Reconstr. Surg. 143, 679–684 (2019).
Rose, M. et al. Computerized adaptive testing—ready for ambulatory monitoring? Psychosom. Med 74, 338–348 (2012).
Koltsov, J. C. B., Greenfield, S. T., Soukup, D., Do, H. T. & Ellis, S. J. Validation of patient-reported outcomes measurement information system computerized adaptive tests against the foot and ankle outcome score for 6 Common foot and ankle pathologies. Foot Ankle Int. 38, 870–878 (2017).
Biber, J. et al. Patient reported outcomes—experiences with implementation in a University Health Care setting. J. Patient-Rep. Outcomes 2, 34–34 (2018).
Low, C. A. et al. Estimation of symptom severity during chemotherapy from passively sensed data: Exploratory Study. J. Med. Internet Res. 19, e420 (2017).
De Brouwer, W., Patel, C. J., Manrai, A. K., Rodriguez-Chavez, I. R. & Shah, N. R. Empowering clinical research in a decentralized world. npj Digital Med. 4, 102 (2021).
Evans, C. J. et al. The selection and use of outcome measures in palliative and end-of-life care research: the MORECare International Consensus Workshop. J. Pain Symptom Manag. 46, 925–937 (2013).
Varni, J. W., Limbers, C. A. & Burwinkle, T. M. Parent proxy-report of their children’s health-related quality of life: an analysis of 13,878 parents’ reliability and validity across age subgroups using the PedsQL™ 4.0 Generic Core Scales. Health Qual. Life Outcomes 5, 2 (2007).
AHRQ. Registries for Evaluating Patient Outcomes: A User’s Guide 3rd edn (eds Gliklich, R.E., Dreyer, N.A. & Leavy, M.B.) (AHRQ, Rockville, MD, 2014).
Haverman, L. et al. Effectiveness of a web-based application to monitor health-related quality of life. Pediatrics 131, e533–e543 (2013).
Matza, L. S. et al. Pediatric patient-reported outcome instruments for research to support medical product labeling: report of the ISPOR PRO good research practices for the assessment of children and adolescents task force. Value Health 16, 461–479 (2013).
Roydhouse, J. K. et al. The use of proxies and proxy-reported measures: a report of the international society for quality of life research (ISOQOL) proxy task force. Qual. Life Res. 31, 317–327 (2022).
FDA. Guidance for Industry, Food and Drug Administration Staff, and Other Stakeholders: Patient-Focused Drug Development: Collecting Comprehensive and Representative Input (U.S. Department of Health and Human Services Food and Drug Administration, Silver Spring, MD, 2020).
Anderson, N. E. et al. Using patient-reported outcome measures during the management of patients with end-stage kidney disease requiring treatment with haemodialysis (PROM-HD): a qualitative study. BMJ Open 11, e052629 (2021).
Bryan, S. et al. Choosing your partner for the PROM: a review of evidence on patient-reported outcome measures for use in primary and community care. Healthc. Policy 10, 38–51 (2014).
Hajewski, C. J. et al. Performance of the patient-reported outcome measurement information system in patients with patellofemoral instability. Orthop. J. Sports Med. 8, 2325967120915540 (2020).
Cruz Rivera, S. et al. Ethical considerations for the inclusion of patient-reported outcomes in clinical research: the PRO Ethics Guidelines. JAMA 327, 1910–1919 (2022).
Aiyegbusi, O. L., Kyte, D., Cockwell, P., Anderson, N. & Calvert, M. A patient-centred approach to measuring quality in kidney care: patient-reported outcome measures and patient-reported experience measures. Curr. Opin. Nephrol. Hypertens. 26, 442–449 (2017).
Lingler, J. H., Schmidt, K. L., Gentry, A. L., Hu, L. & Terhorst, L. A. A New measure of research participant burden: brief report. J. Empir. Res. Hum. Res. Ethics 9, 46–49 (2014).
Ulrich, C. M. et al. Development and preliminary testing of the perceived benefit and burden scales for Cancer Clinical Trial Participation. J. Empir. Res. Hum. Res. Ethics 13, 230–238 (2018).
Cameron, D. et al. Assessing participation burden in clinical trials: introducing the patient friction coefficient. Clin. Ther. 42, e150–e159 (2020).
This work was funded by Merck Healthcare KGaA, Darmstadt, Germany. O.L.A. receives funding from the NIHR Birmingham Biomedical Research Centre (BRC), NIHR ARC West Midlands, NIHR Birmingham-Oxford Blood and Transplant Research Unit (BTRU) in Precision Transplant and Cellular Therapeutics, The Health Foundation, Innovate UK (part of UK Research and Innovation), Janssen Pharmaceuticals, Gilead Sciences Ltd, GSK, and Sarcoma UK. O.L.A. declares personal fees from Gilead Sciences Ltd, GlaxoSmithKline (GSK) and Merck outside the submitted work. J.R. is supported by a Select Foundation Fellowship. J.R. has received funding from the Food and Drug Administration (FDA). J.R. reports personal fees from Amgen, outside the submitted work and reports consultancy with the University of Birmingham Enterprise. S.C.R. receives funding from UK SPINE and European Regional Development Fund—Demand Hub. M.J.C. receives funding from the NIHR, UK Research and Innovation (UKRI), NIHR BRC, the NIHR Surgical Reconstruction and Microbiology Research Centre, NIHR ARC West Midlands, NIHR Birmingham-Oxford Blood and Transplant Research Unit (BTRU) in Precision Transplant and Cellular Therapeutics, UKSPINE, European Regional Development Fund—Demand Hub and Health Data Research UK at the University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Innovate UK (part of UKRI), Macmillan Cancer Support, UCB Pharma, Janssen, GSK and Gilead. M.C. has received personal fees from Astellas, Aparito Ltd, CIS Oncology, Takeda, Merck, Daiichi Sankyo, Glaukos, GSK and the Patient-Centred Outcomes Research Institute (PCORI) outside the submitted work.
The authors declare no competing interests.
Peer review information
Nature Communications thanks Tanja Stamm and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Aiyegbusi, O.L., Roydhouse, J., Rivera, S.C. et al. Key considerations to reduce or address respondent burden in patient-reported outcome (PRO) data collection. Nat Commun 13, 6026 (2022). https://doi.org/10.1038/s41467-022-33826-4
This article is cited by
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.