Main

Insights from PPIE can ensure that research is more relevant to the needs of patients, caregivers and service users1,2,3,4. The UK’s National Institute for Health and Care Research (NIHR) has been a pioneer in promoting an understanding of PPIE and facilitating its implementation and evaluation in research. The organization has produced several guidance documents including a handbook on PPIE and the UK Standards for Public Involvement in Research, which researchers can use to review their plans for PPIE in research projects5,6. In addition, there are guidelines on diversity and inclusion in public involvement7,8, reimbursement for PPIE9 and co-production of research10. Globally, there is increasing recognition of the importance of PPIE with new initiatives at national and international levels11.

However, despite enthusiastic support from policymakers and researchers, the reporting of PPIE in study reports is often absent or minimal; where present, it is usually focused on the mechanics of how patient and public input to the study was obtained. Robust evidence of the impacts of PPIE in research is limited12,13. It should be acknowledged that evaluating impact can be challenging. Researchers may be less inclined to undertake a formal evaluation of PPIE impact as the pathways to impact are often convoluted, and attribution is not always straightforward14. Furthermore, researchers may be uncertain about which approaches to adopt for the evaluation of PPIE impact and the utilization of available guidelines might be challenging in practice15. These issues create a relative evidence void for PPIE and a barrier to the efficient development and sharing of best practice. For PPIE to be meaningful, we need to understand what works and what does not; just as in any other area of science, we need to evaluate it robustly and report it transparently. As a research community, we recognize that all clinical research provides an opportunity to evaluate the impact of PPIE to support a cycle of innovation and improvement.

The ongoing Therapies for Long COVID in non-hospitalised individuals (TLC) Study was set up to investigate the burden of long COVID following coronavirus disease 2019 (COVID-19) infection on patients16. Patient partners were actively involved in the various stages of the study and provided vital input that led to tangible outcomes and impact. Here, we discuss the lessons learned from conducting PPIE for the TLC study, which are applicable to health research beyond long COVID. We identify areas of good practice and reflect on areas for improvement. Finally, we provide two checklists of considerations for PPIE to guide the planning and implementation of PPIE in future health research. Box 1 provides the definitions of key terms.

PPIE and long COVID: the TLC Study

The World Health Organization (WHO) defines long COVID as a ‘post COVID-19 condition that occurs in individuals with a history of probable or confirmed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection, usually 3 months from the onset of COVID-19 with symptoms that last for at least 2 months and cannot be explained by an alternative diagnosis’17. People with long COVID experience a variety of symptoms that affect their physical and mental health leading to impaired quality of life and difficulties with employment17,18,19,20. While our understanding of the pathogenesis, symptoms and complications of long COVID has improved, the long-term effects are yet to be fully understood. It is therefore essential that individuals with lived experiences of long COVID are actively involved and contribute meaningfully (through high-quality PPIE) to research projects investigating the condition21. Indeed, the term ‘long COVID’ was itself coined by patients, further demonstrating how PPIE can frame discussions in health research22.

In response to the challenge posed by long COVID, the NIHR funded several studies — including the TLC Study23 — to provide a better understanding of the condition and to explore potential interventions. Most of these long COVID studies involved substantial PPIE (summarized in ref. 24). Figure 1 illustrates the aims and the work packages of the TLC Study. Work packages 1 and 3 have been completed25,26,27,28; for work package 2, the literature review and retrospective cohort study are completed18,19, while the Bio-Wear sub-study is currently being set up. The results of the feasibility trial as part of work package 4 are being written up for publication, while the economic evaluation is ongoing. A summary of the PPIE methodology used in the TLC study is provided in Box 2.

Fig. 1: Patient and public involvement in the TLC Study.
figure 1

The aims of the TLC Study were, firstly, to evaluate the symptom burden and underlying pathophysiology of long COVID syndromes in non-hospitalized individuals and the impact on quality of life and work capability and, secondly, to identify potential therapies and co-produce a remotely delivered non-pharmacological support intervention. To achieve these aims, four work packages (WPs) — each with substantial PPIE elements — were designed and initiated.

Full size image

Impacts and outcomes of PPIE in the TLC Study

PPIE, whether at the level of consultation or collaboration, has influenced the design and conduct of every aspect of the TLC study, from the earliest stages of grant proposals right through to the dissemination of study findings.

Development of grant proposal and project setup

The grant application for the TLC study was codeveloped with individuals with long COVID and caregivers, and it had a patient co-applicant. The application was successful and the reviewing panel noted ‘strong patient and public involvement (PPI) and links with other relevant COVID-19 research’.

Initially, the researchers and patient partners did not consider the core PPIE group to be sufficiently representative in terms of ethnicity and gender — characteristics identified in literature as being closely linked to the incidence of long COVID18. Patient partners addressed this by recruiting additional individuals from ethnic minority groups via their personal networks and links to national long COVID support groups, thus improving the diversity of the core PPIE group. Our record of the sociodemographic characteristics of patient partners (kept with their consent) and log of PPIE activities has enabled us to actively monitor diversity/inclusion throughout the study.

Study design

Patient partners highlighted that our initial eligibility criteria did not include patients who visited hospital accident and emergency (A&E) units due to COVID symptoms but were not subsequently admitted as inpatients. In response, we widened the inclusion criteria in the study protocol. Based on their lived experiences, patient partners considered the original 6 months follow-up for the Bio-Wear sub-study (Fig. 1; WP2) too long and felt it risked a high dropout rate; after further discussions, we reduced it to 3 months. At a PPIE meeting, patient partners stressed the importance of non-pharmacological interventions that directly improved physical function and work capability. For them, inability or reduced ability to work due to their symptoms was a major impact of long COVID. This feedback shaped the focus of our systematic review of these interventions27,28.

Undertaking research

Development of the Symptom Burden Questionnaire for long COVID

According to patient partners, existing tools for symptom burden measurement did not fully address the breadth of long COVID symptoms — highlighting an unmet need that led to the development of the Symptom Burden Questionnaire for Long COVID (SBQ-LC). Patient partners’ firsthand reports of long COVID verified the literature review findings and informed the early design of the questionnaire. Their valuable feedback on the long list of symptoms helped refine the wording of individual questionnaire items, which improved clarity. Patient partners identified issues with the functionality of the Atom5 study platform (which enabled video assessments and questionnaires via a smartphone application) during the usability testing of the questionnaire prototype, which were subsequently addressed. They assisted with the recruitment of individuals who field-tested the draft questionnaire through links with long COVID patient support groups — namely Long COVID SOS, Long COVID Scotland and Long COVID Support. As a result, 330 responses were received of which there were 274 complete responses.

Over-the-counter medication survey

We were alerted by a patient partner to the fact that some individuals with long COVID were spending substantial sums of money on over-the-counter medications or alternative therapies to manage symptoms — with little or no evidence of efficacy and potential risks of harm. As a result, we codesigned a survey to assess the situation. Patient partners suggested several treatments being used by the long COVID community that researchers were previously unaware of; this meant that the final survey was more comprehensive and relevant.

Co-production of a non-pharmacological virtual intervention

We held a consensus meeting with six individuals with long COVID, ten healthcare professionals, four researchers and three other experts to discuss the design of a non-pharmacological virtual intervention for people with long COVID25. A co-production team including patient partners was formed to select a virtual intervention and design a feasibility study.

Patient partners trialed heart rate monitoring using a smartwatch but expressed concerns over its potential to cause anxiety and harm among participants by encouraging physical exertion and risking post-exertional malaise. As a result, this was excluded from the feasibility study. Recruitment to this feasibility study was then facilitated by patient partners, to ensure diversity in terms of ethnicity, sex and geographical location. This was achieved through their links with churches, the Caribbean African Health Network, Long COVID Warrington group, Long COVID Manchester group, COVID Aid Support Community and COVID-19 Research Support group.

Dissemination of study findings

Patient partners contributed to the content of study newsletters and press releases, ensuring they were patient friendly. They shared study updates and publications within their support groups (Long COVID SOS and Long COVID Scotland) and broader networks. Several patient partners spoke to the media about study findings, and one spoke about PPIE at an NIHR long COVID event on 01 November 2021. Patient partners suggested and produced patient-focused content for the study webpage, which is publicly available via YouTube. These include a series of videos in which they talked about living with long COVID and shared their tips on how they are adapting to their new life. Patient partners also led a long COVID webinar where they shared their experiences of the condition and described their input to the TLC project. Despite having little or no prior experience of academic writing, as co-authors patient partners provided comments on intellectual content of this and other papers19,24,26 and ensured that the wording was acceptable from a patient perspective.

Successes

We believe patient partners should provide their reflections on what they consider a successful implementation of PPIE in addition to the activity/impact logs kept by researchers. This section incorporates the feedback obtained from our patient partners on the successes and challenges for PPIE in the TLC study.

Patient partners confirmed that their early involvement with the study provided the opportunity for them to identify important issues, including those related to study design, that the researchers had missed. They acknowledged that their feedback led to changes in the study design and the text of manuscripts. Patient partners and researchers stated that one of the strengths of the study’s PPIE approach was the flexibility in terms of giving them the option of contributing as a group and/or on a one-to-one basis. This meant that those who were not able to attend meetings due to health or work demands or personal preference were still able to contribute substantially to the study. The introduction of breaks during PPIE meetings worked well and allowed patient partners to rest as they often became fatigued over time.

The training we provided to patient partners was targeted to anticipate and respond to their needs at each stage of the study. An alternative approach would have been to deliver a preplanned full curriculum of relevant material. However, patient partners indicated that they preferred our more agile, responsive approach. For instance, during the co-production of the feasibility trial, the researchers spent time explaining trial processes and key terms during weekly meetings. The approach was less formal, more bespoke, and perhaps not always identified as ‘training’. We recommend that approaches to training and support for PPIE group members should be carefully considered and tailored to the needs of the individuals and the program of work.

Patient partners in the co-production team reported unforeseen positive impacts; working on the project gave them a sense of purpose while coping with the daily challenges of their condition. They also reported benefiting from the peer support that developed between them as a result of working closely together25.

Challenges

The biggest challenge — especially in the early stages — was balancing public engagement against time and resource constraints. The TLC study was set up against a background of urgent population-level medical need, a demanding project timeline and a need to provide answers that would inform national policy. The study also aimed to support and partner with individuals whose lives had been recently turned upside down by long COVID, and who often experienced symptoms that were a direct barrier to their ability to contribute to research. A key learning from the setup period is the need to prioritize communication not only with patient partners but also with the wider patient communities. This is important even when logistical demands of a study (for example, ethics applications) are extremely pressing, and there are no results to share. A practical response is to recognize and obtain resources that may be required at this key pressure point, to ensure patient and public communities are adequately supported.

The pressured timeline also made it difficult to provide as much time as we would have liked to our patient partners to review study documents. This was a particular concern where individuals were affected by long COVID symptoms such as fatigue. However, by adopting a flexible approach to involvement, patient partners were able to provide feedback verbally or in writing. We continually provided other opportunities to contribute if they were unable to provide input to a particular aspect due to time constraints or their health at the time. This way, we maximized their involvement in the study within the overall constraints of the study timeline.

Some challenges arose directly from new insights that patient partners provided. For instance, they identified the need to conduct research into over-the-counter and alternative therapy medication use, which was not part of the original workplan submitted for funding. While we encourage and support such contributions, it is important to note that these may come with resource implications. Furthermore, resource needs for facilitating and reimbursing PPIE may not be fully known and budgeted for when grant applications are made. This may later determine the extent of PPIE that is undertaken during a project. Including additional budget for patient-led initiatives in future bids could support such activities.

We found that some patient partners felt external pressure to respond to queries about TLC’s research on social media, which ideally should be routed to the researchers. It is therefore important to clarify how social media will be managed as part of the terms of reference. This is particularly important for research with considerable public interest so that patient partners are not burdened. As co-authors on publications, patients are required to provide an email address for the online submission systems. However, some patient partners did not feel comfortable sharing personal accounts and so we offered generic study email addresses as an option. We would recommend this approach for future studies.

We recruited fewer men to the PPIE group despite approaching several men during the study. It is unclear why men have been reluctant to participate, but it is worth mentioning that women are more likely to develop long COVID. Furthermore, maintaining adequate numbers of patient partners was a challenge as patients’ symptoms improved over time and several returned to work either full time or part time. As a result, some were unable to participate further in the study. For those still able and interested in contributing, our flexible approach was found to be helpful.

Key considerations for PPIE

We generated an initial long list of ‘key considerations’ for PPIE based on the group discussions, interviews with patient partners and researchers, and the log of PPIE activities (Box 2). The list was discussed with patient partners and revised based on their suggestions for edits and additional items. This list was used to create a survey (in SmartSurvey), which was sent to patient partners and researchers involved in the TLC Study.

The survey consisted of 32 items, organized under eight sections, including six that coincided with stages of research, namely: (i) development of grant proposal; (ii) project setup; (iii) study design; (iv) undertaking research; (v) dissemination of study findings and engagement; and (vi) evaluation of PPIE. Two other important aspects of PPIE were also included, namely: (i) practical considerations for PPIE; and (ii) membership of PPIE group, as well as a free text box for suggestions and comments.

The survey was completed by 34 individuals: 20 (58.2%) patient partners and 14 (41.2%) researchers. Of the 20 patient partners, 18 (90.0%) were female, there were 9 (45.0%) in both 30–49 and 50–69 (years) age categories, and 13 (65.0%) had 1–3 years of PPIE experience. Of the patient partners, 15 (75.0%) were white, 4 (20.0%) were Black, and 1 (5%) was of Asian descent. Of the 14 researchers, 9 (64.3%) were female and 9 (64.3%) were aged 30–49 years old; 8 (57.1%) were white and 6 (42.9%) were from various ethnic minority groups. Half of the researchers had 5+ years of PPIE experience and 4 (28.6%) had 1–3 years of PPIE experience.

Respondents rated each item in the survey on a scale of importance. Most of the items were considered as ‘very important’ or important’ considerations for PPIE in research. Even items with the lowest rankings still had high endorsement from survey respondents — possibly reflecting the fact that the survey items, codeveloped with patient partners and researchers, adequately captured the values of the respondent group. The item ‘enough time should be given to obtain and collate patient feedback, and modify the application as required’ had the highest overall rating, with 100% of respondents considering it an important/very important consideration. On the other hand, the item ‘patients should be involved as co-applicants on grant applications’ had the lowest overall rating as important or very important (67.7%); the raw data show that it was considered as important/very important by 85.7% of researchers and only 55.0% of patient partners.

A total of 23 items rated as important/very important by 80% or more survey respondents (prespecified threshold), formed our checklist of key considerations for PPIE for all studies (Table 1). We strongly recommend that research teams endeavor to incorporate these in their PPIE work as appropriate. The 9 items rated as important/very important by less than 80% of respondents were grouped together as a checklist of additional, desirable considerations for PPIE in research (Table 2). While these have not made it to our checklist of key considerations, they still had high endorsement from patient partners. We still believe they are important and should be discussed with patient partners when conducting clinical or health research.

Table 1 Key consideration(s) for PPIE for all studies
Full size table
Table 2 Desirable consideration(s) for PPIE in research
Full size table

Discussion

This Review outlines the tangible benefits and impacts of PPIE on the various work packages of the TLC Study. The invaluable contributions of our patient partners to the codesign and dissemination of study findings have enabled us to tailor our research to address patient needs and engage with a wider audience. Unforeseen benefits such as peer support and a sense of purpose that patient partners derived by actively contributing to the study further highlight the wider value of PPIE in research. Funders and researchers generally consider the inclusion of patient partners as co-applicants and co-authors on grant applications and peer-reviewed articles as important. While patients still value these types of involvement, these might not be their top priority. It is therefore necessary to explore with patients and the public how they wish to engage in research.

Some of the impacts we have described here have also been reported by others24,29. For instance, the meta-analysis by Crocker et al. found that the involvement of people with lived experience of a condition under study (in PPI interventions) was significantly associated with improved enrollment (odds ratio 3.14 versus 1.07; P = 0.02)29. For the TLC study, patient partners assisted with recruitment for the field-testing of the SBQ-LC and the feasibility study for the non-pharmacological intervention by promoting these via their networks. Patient partners also played key roles in the dissemination of our study findings, similarly to other long COVID studies described by Routen et al.24.

There is a need for PPIE to be embedded in research at an organizational level and not just conducted ad hoc for each project30. There is growing recognition that PPIE should occur at all levels in the delivery, research and regulation of healthcare interventions. International regulatory agencies including the Medicines and Healthcare products Regulatory Agency, US Food and Drug Administration and European Medicines Agency have produced guidance and strategy documents for PPIE31,32,33. They also acknowledge that there is an opportunity for greater PPIE in drug development and regulation34. Increasingly, funding organizations such as the NIHR expect that patient and public input is obtained and plans for this are detailed in grant applications. Some journals have introduced policies requiring authors to report if, and how, patients and the public were involved in their research and/or the drafting of manuscripts. While these are welcome advances, care should be taken that they do not become tick-box exercises.

Further research on the evaluation of PPIE is required. Reflection and evaluation are key for progress toward developing ever more meaningful, effective PPIE. The potential to learn from advancing experience in PPIE will be lost if it is not reported. The contributions of the TLC study patient partners provide an example of how PPIE may positively impact all stages of health research, even in an urgent public health context. Other researchers may draw on our experience to better plan, implement and evaluate PPIE for other long COVID research as well as research related to other health conditions.