Main

The COVID-19 pandemic has disproportionately affected people living in residential care, who are estimated to account for more than one-third of all COVID-19 deaths1,2,3. This disproportionate impact can be understood, in part, in terms of the vulnerability of people residing in care homes and a lack of early intervention and support4,5. However, the structural and institutional risk factors exacerbating this crisis are not well understood. In many countries, adult social care services are delivered by a combination of for-profit (FP), non-profit (NP) and public providers6. Although there is no universally applicable definition of ‘ownership’ across different country contexts, FP providers are commonly understood as private companies operating on a FP basis, NPs are understood as registered NP or charitable organizations, and ‘public’ providers are understood as those operated by central or local government. It is well documented that the outsourcing of social care has substantially increased the market share of private and, in particular, FP care providers7,8, which has motivated a large body of research investigating the association between care home ownership and quality of care9. For example, several systematic reviews on ownership variation among care homes have found that NP and public providers typically deliver higher-quality services than FP providers10,11, a finding that has since been replicated7,8,12. There is no consensus on what organizational and/or behavioral features drive these differences. However, a sizable literature has found consistent correlations between, for example, FP ownership and inferior staffing qualifications and employment conditions, which are considered important mediating factors9,13,14,15,16.

During the COVID-19 pandemic, some have expressed concerns that FP providers have failed their residents by prioritizing profits over care, prevention and caution17, resulting in reports of higher rates of COVID-19 infections and deaths in FP care homes18,19. Although many of these reports are not peer-reviewed (located, for example, in newspaper publications), there is a growing body of academic research investigating the variation in COVID-19 outcomes, such as outbreaks, infection rates and mortality, across care home ownership types20,21. The disproportionate impact of the pandemic on FP care homes has been hypothesized to be driven by some of the pre-existing ownership differences in, for example, resident vulnerability and staffing, but this evidence has not been systematically reviewed and synthesized.

The COVID-19 pandemic has tested the capability of not only individual care homes but also that of commissioning systems in which they operate. In many countries, the adult social care sector has had to adapt to substantial austerity measures22,23 at the expense of staffing, quality and support, which may have been detrimental to the capacity of care homes to cope with the pandemic24. As such, the growing body of research on ownership variation across COVID-19 outcomes offers an important opportunity to revisit ownership-specific variation across care home providers.

The aim of this living systematic review is to identify, appraise and synthesize the available research on ownership variation in outbreaks and infections across FP, public and NP care homes for older people and to update our findings as new research becomes available. A review protocol was registered prospectively on the Open Science Framework25 and on the International Prospective Register of Systematic Reviews (PROSPERO) (CRD42020218673).

Results

Search results

Our combined searches yielded 6,434 records, of which 5,915 remained after removal of duplicates. In total, 100 studies were independently assessed in full text. Of these, 30 studies were deemed eligible for inclusion. Forward citation searches of included studies from the first search identified an additional two eligible studies. Thirty-two studies were thus included in this review (Fig. 1).

Fig. 1: Preferred reporting items for systematic reviews and meta-analyses (PRISMA) 2020 flow diagram79.
figure 1

A total of 32 studies fulfilled the eligibility criteria.

Full size image

Description of studies

Descriptive characteristics of all included studies are shown in Table 1. Most of the included studies were peer-reviewed publications (27 of 32), with two government reports26,27 and three preprints or working papers. The unit of analysis across all studies was care homes. FP care homes were the largest ownership group in all studies that provided detailed sample information. All but two studies28,29 were published in July 2020 or later. Most studies were conducted in the US (25 of 32), followed by Canada (three of 32), England (two of 32), Scotland (one of 32) and France (one of 32). Most included studies were cross-sectional, and only three studies included more than one time point in their analysis30,31,32.

Table 1 Study, sample and outcome characteristics
Full size table

Ownership was usually analyzed by comparing FP, public and NP care homes (18 of 32), usually with FPs as the reference category. Thirteen studies compared FP and NP care homes, in which the NP category also included public sector care homes, although this was not always explicitly described. Two studies focused on private equity (PE) providers in their operationalization of ownership33,34. Twenty studies adjusted for whether care homes were chain affiliated (CA) to investigate COVID-19-related outcomes.

The most investigated outcome was COVID-19 outbreaks (16 of 32), followed by COVID-19-related mortality (15 of 32) and incidence of COVID-19 infection (14 of 32) among care home residents. Six studies investigated staff access to PPE and/or shortage of PPE31,33,34,35,36. Five studies investigated COVID-19-related outcomes among care home staff34,37,38,39,40. Most studies investigated multiple COVID-19 outcomes. Four studies were published after initially being included as preprints29,39,41, and we used results from published versions in the results presented below.

Data sources, time coverage and overlap

Most of the included research merged multiple data sources on COVID-19 outcomes, information on care homes and area characteristics to construct the dataset. Less than 15% of included studies (four of 32) collected primary data on investigated COVID-19 outcomes. The majority of studies used data routinely reported by care homes to public health departments and other government entities. Almost a third of included studies (12 of 32) used data from the Centers for Medicare & Medicaid Services (CMS), which required American nursing homes to report COVID-19-related data, including confirmed and suspected infections and deaths among residents and staff42 from May 2020. Providers were encouraged, but not required, to retrospectively self-report COVID-19 outcomes before this date. Supplementary Table 2 provides an overview of data sources and the time period of dependent variables across all included studies.

Twenty-one studies analyzed self-reported COVID-19 outcomes (confirmed and suspected cases), whereas ten studies only investigated confirmed COVID-19 outcomes. In one study, it was unclear whether the investigated outcomes were confirmed or self-reported28. All studies that investigated PPE and staffing shortages relied on self-reported outcomes. Most studies investigated COVID-19 outcomes collected during the time span of March 2020 to July 2020, and only two studies analyzed data from later than September 2020 (refs. 43,44). Most studies investigated outcomes covering a 1–2-month period although five studies investigated a period of less than 2 weeks34,35,37,45,46. The findings presented below thus relate to the first wave of the pandemic.

Supplementary Table 3 provides an overview of the temporal overlap of included studies across all outcomes. Although several studies relied on similar data sources, there was relatively little temporal overlap across studies using similar sources. The most obvious case of data overlap was in two studies that assessed similar outcomes using the same sample and time period but, notably, came to different conclusions29,47. This is discussed further below.

Risk-of-bias assessments

Our risk-of-bias (RoB) assessments (based on the conducting systematic reviews and meta-analyses of observational studies of etiology (COSMOS-E) guidance) are detailed in Supplementary Table 4. The main concerns related to systematically missing data and selection bias in the included studies. For example, studies that investigated characteristics of excluded observations from the CMS dataset (due to missing or incomplete data) found that excluded care homes were more likely to be FP and were also associated with many risk factors, such as the ethnicity and socio-economic status of care home residents (discussed in detail below)31,35,36,37. This is a potentially serious limitation of studies using this data source (for the purpose of this review), as it suggests that poorly performing FPs may be systematically under-represented in the sample, which may underestimate the observed effect of FP ownership on COVID-19 outcomes. Because of this limitation, all studies using this dataset were downgraded to (at least) moderate risk. For studies using public and government data, we assumed that the risk of information bias was low, unless given a reason to downgrade.

All assessments of confounding bias performed as part of the RoB were based on consideration of factors known either to exacerbate the effect of COVID-19 (refs. 48,49,50,51) or to affect the performance of care homes for older people7,8. Almost all studies adjusted for care home size (27 of 32), and characteristics of quality and staffing were also commonly included. Only five studies reported their outcome(s) as proportions of the number of beds or residents27,29,33,41,43. Twenty-one studies adjusted for ethnic composition, and 17 studies included information about the socio-economic status of residents. Rurality and/or population density was included in 12 of the studies, and local or community incidence of COVID-19 was controlled for in 19 of the studies. See Supplementary Table 5 for details on the direction of effect and model adjustments in all included studies.

Direction of effects

Most included studies were conducted in the US, Canada or the UK, which all have substantial FP care home provision. Although these countries differ with respect to commissioning systems in place, past research indicates important similarities in the behavior of FP care homes across these countries52. However, to ensure that our results are sensitive to study context, we present all findings according to the country of origin. The harvest plot in Fig. 2 displays the direction of effect for all included studies across different ownership categories and COVID-19 outcomes. Bar height indicates sample size, and color denotes country context. See Extended Data Fig. 1 for a harvest plot of the direction of effect across different data sources and RoB levels and Table 2 for details on our Grading of Recommendations Assessment, Development and Evaluation (GRADE) assessments of the certainty of evidence for each finding.

Fig. 2: Harvest plot on the direction of effect across ownership, sample size (care homes) and study context.
figure 2

Bar height indicates sample size, and color denotes study context. Positive and negative effects are understood as associations in either direction that are statistically significant at the 5% level. Note that positive effects refer to elevated COVID-19 outcome values. ‘Null effect’ is understood as differences that are not statistically significant at the 5% level. Outbreaks usually refer to the presence of any COVID-19 infections among care home residents, except for one study that defined outbreaks as at least two cases26. COVID-19 incidence usually refers to the cumulative number of COVID-19 infections among care home residents but also includes binary outcomes on large outbreaks (for example, refs. 39,54). COVID-19 mortality usually refers to cumulative cases among care home residents, except for one study that used a binary indicator of at least six deaths28 and another that analyzed a dichotomized outcome of any number of deaths versus no deaths32.

Full size image
Table 2 Overview of investigated outcomes and GRADE assessments
Full size table

Resident outcomes

In most studies investigating COVID-19 outbreaks among residents (12 of 15), FP ownership was not found to be a statistically significant risk factor, suggesting that FP care homes were neither more nor less likely to have at least one infected resident. The certainty of evidence for a higher incidence of COVID-19 outbreaks among FP care homes was thus considered low.

FP care homes were found to be associated with a higher number of cumulative COVID-19 infections (nine of 14). This direction of effect was consistent across multiple research contexts, including the US, England and Canada, and the certainty of this evidence was rated as moderate. No studies found FP ownership to be associated with fewer COVID-19 infections.

Seven studies found higher rates of COVID-19-related deaths in FP care homes in Canada (Ontario29), California53 and across aggregated samples of US states28,34,43,44,54, whereas seven studies analyzing data from Canada (Ontario47), England32, Connecticut20,27, New York state55 and two studies using aggregate data from multiple states37,56 did not find a statistically significant association. However, the English study only investigated variation in the probability of having at least one COVID-19-related death and did not analyze variation in the cumulative numbers of deaths32. More importantly, most US studies (five of seven) that analyzed cumulative state data (as opposed to data from single states) reported a statistically significant relationship between FP ownership and mortality28,34,43,54. Moreover, one of two studies using cumulative state data that did not identify statistically significant ownership variation analyzed COVID-19 deaths for only 1 week (25 May 2020–31 May 2020)37. The certainty of evidence showing a positive association between COVID-19 deaths and FP ownership was rated as moderate. No studies found FP ownership to be associated with fewer COVID-19 deaths.

All studies that investigated PPE outcomes found FP ownership to be positively associated with insufficient access to or shortage of PPE31,34,36,46, and the certainty of evidence underlying this association was rated as moderate. FP ownership was not consistently associated with staffing shortages, and the certainty of this evidence was rated as low.

Chain status was generally associated with a higher likelihood of COVID-19 outbreaks (moderate certainty of evidence) but not with a higher incidence of infections (low certainty of evidence). Five studies from Canada, England and the US found CA ownership to be associated with a higher incidence of COVID-19 deaths, whereas eight studies using both single and cumulative US state data did not identify any statistically significant variation, and the certainty of this body of evidence was considered low. The one Canadian study that investigated this ownership category found CA to be associated with more COVID-19 infections and deaths47. The two English studies investigating this group did not identify any variation in COVID-19 outbreaks and incidence across CA ownership39, although one found CA to be associated with higher risk of at least one COVID-19 death among a care home’s residents32.

PE ownership of care homes was not found to be consistently associated with worse outcomes compared to other ownership categories, and one study even found PE providers to be less likely to report PPE shortages and confirmed COVID-19 outbreaks34. The certainty of evidence across all PE outcomes was considered very low.

Staff outcomes

Evidence on the relationship between ownership variation and risk of infection among staff was scarce, with only five studies investigating variation in this population. In England, FP ownership was associated with a higher incidence of infection among staff39, but no association was identified among CA care homes. The three studies conducted in the US did not identify any statistically significant variation related to FP ownership, but one study found CA to be correlated with a higher incidence of infections and deaths among staff34. The certainty of evidence was rated as low or very low across all investigated staffing outcomes.

Indirect ownership effects

The raw unadjusted incidences of deaths and infections were higher in FP care homes in all included studies, but this association was not always statistically significant in adjusted models. This suggests that there are important mediating factors that influence the effect of ownership on COVID-19 outcomes. However, ownership categories were usually treated as covariates for model adjustment, and specific results relating to this variable were not often directly discussed and interpreted by study authors. For studies in which discussion of the effect of ownership on COVID-19 was provided (16 of 32), the below-mentioned mediating pathways were considered most important.

Table 3 provides an overview of key risk factors that were correlated with FP ownership and how adjusting for these factors influenced the observed relationship between FP ownership and COVID-19 outcomes. There was evidence suggesting that ownership was correlated with multiple risk factors. For example, FP care homes were more likely to be larger (in terms of number of clients)20, to serve minorities or clientele of low socio-economic status37 and to be more crowded26,29. Moreover, one Canadian study reported that the effect of FP ownership was mediated by older care home design standards, number of residents, the staff-to-bed ratio and CA47. When including these covariates, the effect of FP ownership on COVID-19 mortality and incidence lost statistical significance. However, it was not clear from the analysis which of these variables accounted for this change. Notably, two studies conducted by the same team analyzing data on the same sample (long-term care homes in Ontario), time period (29 March 2020–20 May 2020) and outcomes (outbreaks, incidence and mortalities) adjusted for different covariates and came to different conclusions29,47. Specifically, Stall et al.47 did not find FP ownership to be a statistically significant predictor of COVID-19 mortality and incidence in their fully adjusted models, whereas Brown et al.29, who adjusted for crowding instead of CA and outdated design standards, did report a statistically significant association between FP ownership and higher rates of infections and deaths.

Table 3 Overview of risk factors associated with both FP ownership and COVID-19 outcomes
Full size table

Although the multicollinearity between ownership and other risk characteristics may account for some of the observed ownership variation, Table 3 and Supplementary Table 5 demonstrate that a substantial body of research found FP care homes to perform worse during the pandemic, even when controlling for these characteristics. The importance of investigating variation across ownership categories, despite their associations with other risk factors, is elaborated in the Discussion.

Discussion

Our synthesis and critical appraisal of 32 studies suggests that FP ownership is not consistently associated with a higher probability of COVID-19 outbreaks. We assessed evidence demonstrating that consequences of outbreaks, in terms of cumulative infections and deaths, were exacerbated by FP ownership. The main mechanism identified in relation to ownership was that FP care homes were more likely to report PPE shortages31,33,36, which is likely to have influenced the ability of these care homes to safeguard residents during the early stages of the pandemic57. Further, FP care home ownership was found to be associated with other risk factors such as crowdedness29, client vulnerability37 and inferior quality ratings26,58.

There was evidence of moderate certainty of a correlation between CA and the risk of a COVID-19 outbreak, but certainty of evidence of elevated numbers of infections and deaths among CA providers was rated as low. PE ownership was not consistently associated with inferior COVID-19 outcomes.

When interpreting the results of this review, it is critical to consider the conceptual and practical meaning of ‘ownership’. Throughout the literature on ownership variation in the care home sector, it is rarely specified whether (1) variation associated with FP ownership should be interpreted as a direct consequence of the unobservable profit motive often assumed to drive FP providers or (2) if ownership differences should rather be understood and analyzed in terms of the specific factors correlated with different ownership categories (for example, quality rating, care home size and vulnerability of residents). Importantly, ‘ownership’ is simply a legal status, which, in isolation, does not provide much information about the operation and characteristics of care homes. It is nonetheless commonly assumed that these organizational categories can be used to predict behavior, for example, by conjecturing that FP care homes will prioritize financial gains over client concerns to the extent that regulation allows15,59.

However, there are many factors that should be considered when evaluating the performance of care homes, as there is plenty of variation among these organizations independent of ownership. For example, both NP and FP care homes are commonly affiliated with multifacility chains and vary widely in size and client groups served. Further, care homes of all ownership types receive poor quality ratings7,8. Thus, the purpose of analyzing variation across ownership is not that it is a guaranteed marker of quality, deficiency or irresponsible management; rather, it constitutes a proxy for certain characteristics and behaviors that may (or may not) be associated with relevant outcomes. For example, FP care homes are commonly associated with inferior staffing conditions58, which previous research has found to be an important driver of lower performance and service quality14,15,16. Moreover, multiple recent studies found that minority and Medicaid residents are housed at higher rates in FP care homes, which may also help to explain poor COVID-19 outcomes60,61,62. In other words, if FP care homes are consistently and systematically correlated, on average, with risk factors, such as outdated design standards, inferior staffing conditions, crowding, client vulnerably, etc., this should be considered part of the (average) underlying effect of this ownership category.

Throughout the synthesis, we incorporated these nuances by critically analyzing our results according to different model specifications and by considering other ownership types, such as multi-CA and PE-owned care homes. Understanding and analyzing systematic variation across ownership groups is of immense policy relevance, given that the majority of care homes in many developed countries are controlled by the FP sector. If social care markets facilitate the entry of providers that, on average, are associated with undesirable or risk-associated characteristics, this can, in the aggregate, have important consequences for the quality of care services for the aging population. Whether or not future regulation should focus on the legal ownership status or individual underlying characteristics remains unclear. Yet, in our synthesis of the cumulative body of evidence, we identified a systematic pattern of exacerbated COVID-19 outcomes among FP care homes, which was only partly explained by an array of possible mediating factors. This suggests that regulations targeting single risk factors, rather than the market structure responsible for the outsourcing and privatization of social care, may not suffice.

The findings of this review highlight the importance of ownership in accounting for poor COVID-19 outcomes across care homes. The adult social care sector found itself exceedingly exposed at the beginning of the pandemic63, in large part due to delayed government support and intervention, but also as a result of many years of political and financial neglect17. With this review, we do not suggest that challenges faced by care homes during the pandemic can (or should) be understood through the lens of ownership alone. It is clear that care homes have faced severe challenges that cannot be reduced to ownership. However, outsourcing to FP providers has become the status quo in many care markets, often based on the rationale that open-market competition will optimize the functioning of care homes. This claim has been subjected to extensive scrutiny and is not well supported by empirical evidence7,8,9,10,64. This review adds to this evidence base by systematically appraising and synthesizing the available research on how consequences of the COVID-19 pandemic in care homes varied by ownership type during the first wave of the pandemic. Although our results represent multiple national settings, most of the included research was conducted in the US, likely owing in part to the public availability of the national CMS dataset. Efforts are currently being made in the UK to create a similar type of systematic, live and linked dataset on care homes65, which is an important endeavor if the consequences of this pandemic are to be understood and addressed going forward.

Our findings should be interpreted in light of some caveats, most of which relate to characteristics of the included studies. First, most studies were conducted in the US and Canada, and the results thus primarily relate to North America. Second, the majority of US studies relied on CMS data, whereas all the Canadian studies were conducted in Ontario using the same sample of long-term care facilities, which means that there is overlap in analyzed data across certain studies. Second, the body of included research was too heterogeneous to be meaningfully meta-analyzed, and this review thus represents a critical appraisal and narrative synthesis conducted in line with synthesis without meta-analysis (SWiM) and COSMOS-E guidance66,67. Third, throughout our RoB assessments, we assumed that the reporting of COVID-19 outcomes was not systematically related to ownership. However, there is some suggestive evidence of a longer turnaround period for resident test results among FP providers68, which, if generally true, may bias the effect of FP ownership toward the null due to under-reporting. Lastly, it is known that COVID-19 research is rapidly published69, which may expose our results to publication bias in favor of articles that have been fast tracked for reporting timely and significant outcomes. However, by not restricting our studies to peer-reviewed research, we were able to also consider evidence presented in preprints and government reports in our synthesis.

This review presents a critical appraisal and synthesis of available evidence on ownership variation across COVID-19 outcomes during the first wave of the pandemic. It reports evidence of moderate certainty that FP ownership was a risk factor for elevated cumulative COVID-19 infections and deaths among care home residents. Ownership and characteristics associated with FP care home providers may thus present key regulatable factors that can be addressed to improve health outcomes in vulnerable populations and reduce health disparities.

Methods

COVID-19-related research is published at a high frequency, and the time between submission and publication is substantially shorter than is typical for research more generally69, which makes it particularly important that this rapidly growing body of evidence is critically appraised and systematically synthesized regularly. To ensure that this review represents a recent and relevant synthesis of the available evidence on the topic, it is being conducted as a living systematic review, ‘a systematic review that is continually updated, incorporating relevant new evidence as it becomes available’ (refs. 70,71). Specifically, we plan to update our results every 6 months for 2 years after initial publication. An update will entail (1) running the full search string for the updated time period and (2) forward and backward citation tracking of all included studies72. If inclusion of new evidence results in important changes to the findings and conclusions, the publication may be updated to reflect these changes. If updated results do not change the main conclusions, we will instead write a short report detailing the number of studies and characteristics of the updated evidence. Updates will be published on the Open Science Framework25. This review is conducted and reported in accordance with PRISMA guidelines73 (see Supplementary Data 1 for a PRISMA checklist).

Search strategy

Before developing our search strategy, we performed scoping searches using Google Scholar and preprint repositories to identify relevant articles and performed citation searches on all identified articles. This preliminary sample of includable studies was used to design our search strategy and to test its specificity. The search strategy was piloted and adjusted until it retrieved all pre-identified studies. The full search strategy can be found in the Supplementary Data 2. Our search string was implemented in the following databases: ABI/INFORM Global, the Coronavirus Research Database, the Criminology Collection International (Criminal Justice Database and NCJRS Abstracts Database), the International Bibliography of the Social Sciences, the Politics Collection (PAIS Index, Policy File Index, Political Science Database and Worldwide Political Science Abstracts), the Social Science Database, the Sociology Collection (Applied Social Science Index and Abstracts, Sociological Abstracts and Sociology Database) via ProQuest. We searched Embase, Global Health, Medline and PsycINFO via Ovid. We also searched Web of Science and CINAHL. We implemented our search strategy on 3 November 2020 and on 6 May 2021 (see Supplementary Data 2 for details). All search results were double screened by A.M.B.-M. and M.D.E. for the first search iteration and A.M.B.-M. and B.V. for the second search iteration.

Inclusion criteria

We assessed study eligibility based on four criteria. First, studies had to investigate variation in COVID-19 outbreaks, infection rates and/or excess or COVID-19-related mortalities among residents or outcomes related to PPE use and availability, staff shortages, preparedness and infection and mortality among staff and visitors. We did not exclude studies based on how COVID-19 outcomes were operationalized (for example, if the infections were confirmed by PCR test or by self-report and whether analyzed outcomes were dichotomized or continuous), although these aspects were considered in our RoB assessments. Second, studies had to investigate variation in any of the above outcomes across ownership categories, which are conventionally operationalized as FP (that is, private care homes run for profit), NP (that is, registered not-FP care homes or charities) and ‘public’ (that is, municipal or local authority care homes). However, ‘ownership’ is not consistently operationalized and defined in the literature, and terms such as FP, NP, ‘private’, CA and ‘public’ are rarely clearly defined. As the objective of this review is to appraise and synthesize research on ownership variation, we considered any definition or classification of ownership. The nuances of and potential for overlap between definitions are outlined in Supplementary Table 1. Third, studies had to employ an observational research design, including, but not limited to, cross-sectional and cohort designs and secondary analyses of registry data. Both published articles and unpublished manuscripts (for example, preprints and reports) were eligible for inclusion. Fourth, studies had to investigate residential care homes for older people, including, but not limited to, long-term care facilities, nursing homes and retirement homes.

Data extraction

Descriptive information on citation details (author, title, journal) and study characteristics (research design, analysis, sample details) were extracted from all included articles. We also extracted detailed descriptive data (for example, country, source and period of data coverage) and outcome and exposure variables (for example, definition, operationalization and cutoffs). All results relating to ownership variation across COVID-19-related outcomes and accompanying interpretations were extracted for all studies. Results were extracted by A.M.B.-M. and independently validated by at least one other reviewer per study.

Risk-of-bias assessment

RoB was assessed using COSMOS-E guidance66. We employed this guidance rather than, for example, ROBINS-I for non-randomized intervention studies, as it is specifically designed for systematically reviewing observational and correlational research. Specifically, we evaluated the following bias domains: confounding, selection bias and information bias. In line with the consistent recommendation to avoid quantitative scoring of risk domains66,74, all RoB assessments were based on the qualitative subjective assessment of the reviewers. RoB assessments were conducted in blind duplicate by two reviewers, and the final rating was decided through discussion and in consensus. All assessments were conducted with the focus of our review (variation in COVID-19 outcomes across ownership) in mind, and assessments may thus not represent the RoB across other investigated associations and outcomes. The overall RoB assessment for each study was based on the lowest assessment in any bias domain. We did not exclude studies based on RoB assessments.

Certainty of evidence

To assess the certainty of the evidence underlying each finding, we employed the GRADE approach, specifically recently published guidance on the use of GRADE when assessing evidence on prognostic factors75,76. For this, a body of observational evidence starts with high certainty, and five domains are used to downgrade certainty: RoB (using COSMOS-E), imprecision, inconsistency, indirectness and publication bias, as well as domains for upgrading evidence. As the included body of evidence could not be meaningfully meta-analyzed (described in detail below), we did not conduct a statistical assessment of publication bias, such as testing asymmetry of funnel plots or the trim-and-fill method, as these tests would then only be conducted on a subset of included studies. To date, there are no guidelines on how to assess publication bias in systematic reviews without meta-analysis. Also, for observational studies, the accuracy of such tests is unclear27. However, as this is the first iteration of a review on a topic with rapidly emerging evidence, it may be prone to ‘lag bias’ (early publication of positive results)29, which will be explored in later updates of this review. In assessing publication bias in this iteration, we evaluated (1) whether there were serious concerns in the reporting of results and (2) whether the direction of results consistently varied across peer-reviewed publications and preprints or government reports across all outcomes.

Data synthesis

Due to a high degree of heterogeneity among included studies in terms of model specifications, operationalization of outcomes, inconsistent ownership categorization of the reference group and overlapping data, we did not perform a statistical meta-analysis of included results. This decision was made with attention to the pitfalls of employing statistical methods and assumptions designed for the analysis of highly homogeneous data or randomized controlled trials of interventions to observational and correlational research77. Our synthesis and reporting of findings was guided by SWiM guidelines67 and can be described as follows.

First, we narratively summarized key characteristics of included studies, such as publication type, sample details, ownership categorization and data sources. Second, we assessed the RoB across all included studies. Third, using GRADE, we assessed the certainty of evidence for each outcome across the full body of contributing studies. Fourth, we constructed harvest plots to graphically illustrate the direction of effects across different outcomes and ownership categories with attention to model specifications, sample size and the RoB of contributing studies78. Harvest plots serve as a way to synthesize and describe heterogeneous bodies of evidence that cannot be meaningfully synthesized meta-analytically. Lastly, we analyzed and examined the role or mediating factors associated with both FP ownership and COVID-19 outcomes79.

Reporting Summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.