Introduction

This paper summarises how commentators and advocates have thought about evidence use in medicine and policy over the previous decades. We draw on existing literature to outline the debates about the main challenges to the evidence-use movements, and the strategies used by advocates to address these challenges. At times, the EBM and EBP movements have developed in parallel. Here, we compare the two movements, and identify lessons for each (and their critics) to learn. Finally, we summarise the main answered, and unanswered questions for researchers and practitioners of evidence-based decision-making.

Advocates of evidence-based policy (EBP) justifiably use evidence-based medicine (EBM) as a template for their activities. Health care is getting more effective, and the proliferation of clinical guidelines to cover more and more of the changing face of clinical practice indicates the success of this institutional approach (Montori, 2008). However, advocates of EBP sometimes use an idealised template of EBM, which indicates that all policy/practice decisions should be predominantly based on research evidence–even that only some forms of research are ‘valid’ enough to support decision-making–the randomised controlled trial (RCT) and systematic review. It is immediately obvious that this strict version simply wouldn’t work for EBP, as it dismisses too much useful information, and does not take account of people’s experiences and values. Critical commentators of EBP observe that ‘evidence-use’ is hard to diagnose and/or categorise (Parkhurst, 2017b; Smith 2013a), that focusing on evidence-use risks ignoring most of the machinery of decision-making (Oliver et al., 2014), and that what needs addressing by researchers and practitioners of evidence-based decision-making, is the ‘human factor’; power, actors, and context (Freudenberg and Tsui, 2013; Oliver et al., 2012; Wesselink et al., 2014).

This strict version is, however, a caricature. These days, most sensible people agree that there are good reasons for research evidence not overruling clinical experience by default: for example, the incompleteness of the evidence base or that clinical and lay experience should be recognised as vital components in the decision-making process, alongside a range of research types (Head, 2010). Instead, EBM is often presented as a practice where clinical decision-making is seen as a site of negotiation between different stakeholders, which include the (researcher-derived) evidence base, the clinician, the patient, and providers of care and services generally. Drawing on this perspective, we present three key lessons from these debates for practitioners of EBM and EBP.

First, that although the tools and practices developed by EBM (privileging RCTs and systematic reviews over practitioner discretion) and EBP (led by demands for evidence from political decision-makers) are different, there are opportunities for greater transparency about decision-making processes in policy. Second, we argue that EBM has developed ways to try and balance researcher, practitioner and public knowledge, without compromising methodological rigour or privileging some perspectives over others. Learning to assemble the evidence jigsaw is a complex process which goes beyond methodological pluralism, but requires research evidence to be applied in varying settings and balanced with other interests, potentially extending to the use of deliberative processes to develop consensus between different stakeholder values and priorities. Third, we argue that, whatever the epistemological stance of the practitioner, it is vital to understand the role of power in evidence-informed decision-making; that examining who exerts power and how, is fundamental to understanding how the practice of evidence-use occurs in both medicine and policy. Simply stating that power needs to be shared, or that deliberative processes should be used to reach consensus, does not helps us to describe, understand, or address underlying power imbalances in knowledge production, governance, or use.

The evolution of EBM: a roadmap for EBP?

EBM has evolved significantly since its inception, with Cochrane’s answer to the injunctive of ‘do no harm’ (Cochrane, 1972). The opening salvo in the EBM movement was a call to use high-quality research evidence to select only effective interventions, ideally the RCT, which can provide evidence on effectiveness (whether an intervention has the desired outcome) and efficiency (whether this effect translates outside the research setting) of an intervention (Sibbald and Roland, 1998; Victora et al., 2004). RCTs reduce the role of chance in determining the outcome of experiments, and thus allow causal statements about the utility of interventions to be made with relative confidence. The 1990s saw the enshrinement of the RCT as the ‘gold standard’ form of evidence for EBM, often being portrayed as sitting atop the ‘hierarchy of evidence’ (Petticrew and Roberts, 2003) an heuristic to determine the credibility of research evidence in developing clinical guidelines and recommendations (see, e.g., GRADE (Ansari et al., 2009) and National institute for Health and Clinical Excellence (NICE) processes (NICE, 2004). Since these early days, limitations to the use of the RCT have been increasingly highlighted (Cartwright, 2007; Oliver et al., 2010; Victora et al., 2004), and in the last decade, the cultural hegemony of the RCT has come under attack as its ethical, pragmatic, political, and methodological limitations ––have become more apparent (Pearce et al., 2015).

To many advocates of EBP, including some politicians, it seemed self-evident that policy should be based on the best available evidence. Coming to prominence in the UK under the Blair Government, EBP assumed that with enough ‘sound evidence’ it was possible to move beyond the vagaries of party politics to reach “a firm high ground in the policy swamp” (Parsons, 2002, p 45). Drawing on the EBM template, the RCT was promoted as an optimal way to test and evaluate policy, with the UK civil service becoming increasingly supportive (Ettelt et al., 2015; Haynes et al., 2012; Oakley, 2000; Pearce and Raman, 2014; Pearce et al., 2015). In the US particularly, there is both a culture of policy innovation and experimentation, and, through the existence of the federal states, the means to conduct large-scale social experiments (Baron, 2009; Kalil, 2014).

However, it is now widely accepted in EBM that a range of evidence types should be used by clinicians and patients when making decisions, and that decision-making processes should be transparent and inclusive. This started in response to a need to cover more of the realities of patient experience and clinical practice (Greenhalgh and Hurwitz, 1999; Hammersley, 2007). We also now recognise the role of values and interpretation in understanding evidence, such that one piece of evidence may have multiple meanings (Pearce and Raman, 2014; Smith, 2014).

While these moves were significant in medicine, methodological pluralism has always been more acceptable in policy (Ingold and Monaghan, n.d.; Murphy and Fafard, 2012). Policy meanings affect the presentation, acceptance and framing of evidence and are vital to identifying levers for action (Grundmann and Stehr, 2012; Pearce et al., 2017).

Policymakers have always had to consider multiple framings of policy problems (Gold, 2009; Russell et al., 2008). In medicine, problem identification and diagnosis is relatively uncontested, whereas getting agreement on the nature of policy problems–let alone causes or solutions - is challenging (Cairney, 2012; Liverani et al., 2013; Weiss, 1979, 1991). The ‘hierarchy of evidence’ has less currency in an environment where policymakers must balance economic, financial, ideological and other perspectives on single issues (Lin, 2003). This is both a strength, in that policy has access to a diverse jigsaw of evidence, and a weakness in that the quality and reliability of such evidence is contested. It is not always clear what evidence is used, how it is identified, or who gets a voice in its production and interpretation (Dobrow et al., 2004; Oliver and de Vocht, 2017; Oliver and Lorenc, 2014).

Indeed, the narrow technocratic version of EBP which subscribes to the primacy of the RCT is now mainly used as a straw man (evidenced in, e.g., Freiberg and Carson, 2010; Marmot, 2004; Pawson, 2002). A significant amount of research and commentary has been conducted on multiple fronts; to guide policymakers to use research evidence more effectively in the UK (Cabinet Office, 1999; Campbell et al., 2007), Australia (Banks, 2009) and the USA (Laura and John Arnold Foundation, 2016); to investigate factors hindering evidence use (Innvaer et al., 2002; Oliver et al., 2013; Orton et al., 2011) and to develope interventions aiming to ‘upskill’ policymakers or help them to access the ‘best’ available evidence more efficiently (see, e.g., (Armstrong et al., 2013; Dobbins et al., 2009; Ward, 2017), or for a recent synthesis (Boaz et al., 2011)). Yet, there are still useful lessons that EBP could learn from EBM, and vice versa. Accepting that both EBM and EBP are more nuanced in practice, than in caricature, what lessons can they learn from one another?

Lesson one: be more transparent

EBP advocates could learn from EBP about how to develop clear and transparent processes and structures to inform decision-making. Institutions such as NICE provide epistemic governance and accountability regarding the creation and utilisation of knowledge–through, for example the clinical guidelines process, wherein groups of clinical and lay individuals assess systematic reviews of the literature and, together, produce sets of recommendations for clinical practice. Elsewhere, health research funders have supported collaborations between clinical and academic staff (such as the CLAHRCs, in the UK, and ZonMy in the Netherlands), in which research topics are co-developed, and/or findings implemented and evaluated.Footnote 1 Focusing more on research, the James Lind AllianceFootnote 2 is an example of the development and application of an explicit process to co-produce research agendas on health topics, drawing on the input of patients, carers, and practitioners to rank proposed research questions on a particular topic (Beresford, 2000; Jonathan Boote et al., 2010; Boote et al., 2002; Oliver, 2006; Stewart et al., 2011). However, there are few evaluations of whether these structures change either research or population-level outcomes.

Within policymaking, there has been similar investment in organisations institutionalise or formalise the process of identifying and codifying of evidence, and the translation of such evidence into policy recommendations. Again in the UK, the Blair/Brown Governments endorsed the “what works” principle, and funded What Works Centres, NICE and the Education Endowment Fund (Parsons, 2001). More recently, a variety of non-governmental bodies have entered the evidence ‘mix’, such as the Behavioural Insights Team, NESTA and IPPR in the UK, and Laura and John Arnold Foundation and Brookings in the US. While these are often well-resourced and sometimes influential, little is yet known about how, and how well, the knowledge produced by these new initiatives is taken up, or whether it leads to changes in population or patient outcomes (Head, 2017).

While many have recognised that the policy process is nuanced and negotiated rather than linear (Ettelt et al., 2015; Pearce et al., 2015; Sarewitz, 2009; Sarewitz et al., 2004), there remains strong support for the view that research evidence ought to inform more of policymaking (see, e.g., Armstrong et al., 2013; Dobbins et al., 2009; Ward, 2017). Yet, nothing in EBP  compares to the transparency with which EBM finds and incorporates evidence into its decision-making processes.

Lesson two: balance inputs to assemble the evidence jigsaw

Next, it is important for decision-makers to think critically about what evidence is needed in the context of a particular decision; what research evidence, whose voices, which interests and values need to be heard. EBP advocates could usefully learn from EBM about the importance of context and individual variation in situations, the expertise of decision-makers, and technical advice on how to interpret research evidence for specific situations which may not be identical to research settings. As we have shown, EBP can learn from EBM in recognising that a ‘horses for courses’ sensibility can help match knowledge requirements with appropriate methodological approaches (Parkhurst and Abeysinghe, 2016; Petticrew and Roberts, 2003). This may should, in theory, also include public interests occasions, or the deployment of other methods on other occasions. EBM has come to mean the integration of individual clinical expertise with external clinical evidence (Sackett et al., 1996), with patient experience and perspectives (Stefanie Ettelt and Mays, 2011; Greenhalgh and Hurwitz, 1999; Trisha Greenhalgh, 2016; Green and Thorogood, 2009; Pope and Mays, 1993), and the standardisation of the profession through formal structures (Greenhalgh et al., 2014). One formative EBM text frames the inclusive nature of modern EBM as a way to ‘help’ the experienced medic, rather than overrule them with science (Sackett et al., 1996).

In addition, EBM has exerted considerable effects to include a greater range of stakeholders in both the production of new evidence (i.e., the research process), and the use of this evidence. Lay members are part of groups which develop clinical practice recommendations after deliberating on the evidence, and Patient and Public Involvement (PPI) boards in health and social care aim to move beyond research participation to actively involve patients and public as advisers in the research process, or possibly co-researchers (National Institute of Health Research, 2014). In short, PPI shifts research from something done ‘to’, ‘for’ or ‘about’ the public to an activity done ‘with’ or ‘by’ the public (Boote et al., 2010)

EBP can also be seen as a site of negotiation, between the historic challenges of balancing multiple perspectives, and more recent efforts to increase public participation and introduce greater rigour into evidence production. While policymaking is sometimes framed as an elite activity (Haas, 1992; Rhodes and Marsh, 1992; Scott, 2001), democratic processes should, in theory, also include public interests, either through representation by elected politicians or by more direct participation. Advocates of greater civil inclusion in policy research and processes argue that there is an instrumental and moral value to including lay people (Epstein et al., 2014; Jasanoff, 2003; Wynne, 1992). Others interpret this as a repudiation of expertise (Kuntz, 2012, 2016), even leading one political scientist to call for democracy to be replaced by epistocracy, ‘rule by the knowledgeable’(Brennan, 2016). In general, however, greater public participation and ‘open policymaking’ is becoming the norm (Burall et al., 2013; Burgess, 2014; Pallett, 2015). In practice, this trend faces the same problem as EBM: how to properly consider ‘lay’ perspectives that may differ radically from evidence provided by well-established sources. Finding ways to include civil society and other interests in democratic processes is challenging, time consuming, and arguably at odds with the smooth running of government. Similarly, in an era of ‘big politics’ many members of the public are feeling increasingly disempowered and therefore disinclined to participate in official inputs to policy (Bang, 2009; Flinders and Dommett, 2013; McCluskey et al., 2004). Despite the growth in knowledge production institutions like the What Works Centres, the structures set up to support EBP, and in particular the strategies employed by these structures, are much less explicitly delineated than the cognate structures for EBM

Lesson three: understand how power shapes knowledge production and use

Some have begun to use policy studies to explore the importance and dynamics of power, inclusivity and advocacy (e.g., Cairney, 2017; Liverani et al., 2013; Pearce, 2014; Wesselink et al., 2014; Head, 2008). Acknowledging the importance of power, however, has not yet been taken on board by practitioners when developing strategies to support evidence informed decision-making (Fafard, 2015; Head, 2010; Murphy and Fafard, 2012; Parkhurst, 2017b; Smith, 2013a).

As indicated above, practitioners of EBM recognise that there are power imbalances between researchers and the public; and between practitioners and patients. However, merely recognising such power differentials does not tell us how to negotiate them appropriately, particularly in terms of how to assign legitimacy to the evidence provided by each stakeholder (researcher; professional; patient), or about the circumstances under which it is appropriate to rely on the experience of one stakeholder over another. While EBM provides a forum for shared discussion and sense-making, the navigation of social hierarchies is uncharted territory.

The response to this dilemma has been to produce new and inclusive structures for the production of knowledge (through co-productive approaches) and for its implementation and use (through inclusion of stakeholders in guideline development groups). It is often assumed, that “appropriate” evidence, filtered and nuanced in the right way by the right actors (often termed “deliberative approaches” or “public deliberation” by scholars in the field), consensus is possible (for example, see Conklin et al., 2008; Duke, 2016; Smith, 2013b).

Yet, although new structures can formalise processes of decision-making, they do not necessarily lead to inclusion of a diverse (or appropriate) set of expert voices. Recruitment to stakeholder/advisory groups or decision-making committees is rarely a transparent process, and often by invitation. Moreover, treating participants in a deliberative process as ‘representatives’ of an (unclear) population is unfair, yet recognising their individual histories and agendas is almost impossible to square with the ideal of value-free, methodologically rigorous decision-making. Finally, consensus is seen as a desirable goal, which indicates that power has been balanced and evidence appropriately considered and used (Horst and Irwin, 2010). We dispute all these assumptions, and suggest that consensus may not always be possible (Hendriks, 2009) and that the effects of consensus-seeking can vary depending on political systems and cultures (Dryzek and Tucker, 2008). Indeed, terms such as ‘evidence uptake’ or ‘improvement of evidence use’ mask the complexities and politics of knowledge transfer (Parkhurst, 2017a)Footnote 3

Increased sensitivity to competing values and policy frames in society can help reconnect policymaking to its inescapably political context, an approach that stands in opposition to the technocratic orientation of EBP (Hawkins and Parkhurst, 2015; Parsons, 2002). As well as producing and utilising appropriate knowledge, maximising the use of evidence ultimately boils down to the importance of human judgement. Just as medical professionals have to judge the applicability of the available evidence to a patient, so policy experts must act as mediators between the knowledge base and policy problem (Grundmann, 2017; Ingold and Varone, 2012; Turnhout et al., 2013). EBM can therefore offer an active research agenda, but provides few lessons to EBP on reconciling different perspectives.

How can we better understand how, and why power matters?

We suggest that means looking at the entire system of ‘epistemic governance’ to better understand how knowledge is produced and used by decision-makers (Pearce and Raman, 2014). For instance, how issues are framed, and how politicised they become, affects what knowledge will be considered legitimate and credible, which research is commissioned and used, and who becomes authoritative and powerful (Hartley et al., 2017, Brown 2015). Both EBM and EBP impose political agendas by impelling practitioners to make choices based on the available research. For example, health policy research focuses on reducing morbidity and mortality, rather than, for instance, problematizing and understanding the roles of social categories (Liverani et al., 2013; Parkhurst and Abeysinghe, 2016). If we see evidence production as driven by ease rather than public need, this raises important question about how research agendas are set; about how the knowledge production system operates; and of whether researchers should be open about their political values or the envisaged political endgame of their research (Pielke Jr., 2007).

This implies that future approaches to evidence use must go beyond calls for more structures or greater methodological pluralism. Instead we need better heuristics to determine what kinds of evidence are needed and for what purposes, considering the kinds of question being asked, the actions which may be taken as a result, and the politicisation of the problem. One potential way to navigate these complex issues is to consider some types of question being asked, and how this might affect the type of evidence being used. Examples may include:

  1. 1.

    What is the nature of this problem?

  2. 2.

    What have others done?

  3. 3.

    How do I win this argument?

  4. 4.

    This is new, what do I do?

The substantive topic in question will affect which types of evidence are required to answer, a dynamic unrecognised within many hierarchies of evidence and evidence appraisal systems (such as GRADE). For example, a doctor may observe a new response to a drug, or a new combination of symptoms (Q type 3). This may be highly political (e.g., huge increase in presentation of gender dysphoria in a small community) or uncontroversial (such as a new side effect of a drug). The question “what do I do?” will need very different consideration for both these situations. Similarly, it is possible to imagine situations in both medicine and policy where there is a desire to ‘win an argument’, but marshalling the appropriate evidence will depend on the actors involved, the politicisation of the topic in question, and so forth.

Politicisation of the topic in question is an under-explored facet of the ‘evidence-based’ paradigm, and one which gets to the heart of the technocratic/democratic tensions in the ideology (van Eeten, 1999). This is at least partly by design. Presenting the EB process as value-free, methodologically driven, and uncontroversial allows decision-makers to de-politicise potentially risky debates, and disarms potential opponents by making it harder to challenge (Boswell, 2017). One reason for EBP coming to such prominence in the New Labour years may be that Blair was more fearful of a predominantly right-wing press than previous governments, helping to drive the depoliticisation of knowledge production and utilisation (Parsons, 2002). While EBM’s new structures focused on co-producing evidence through new methods, EBP’s new structures looked to tame existing methodological pluralism as a means of legitimising decision-making, through producing and legitimising evidence in a way which reduced the scope for political attack and was amenable to the trend for ‘joined-up government’ (Davies, 2009).

Conclusions

We have summarised major trends in the literature on EBP and EBM and identified three key lessons for practitioners and commentators in both. Yet, implicit in the research and solutions (brokers, collaborative structures, shared power and decision-making) of both EBP and EBM is that a mix of appropriate evidence, experts, processes and structures will lead to improved decision-making. In fact, we do not know whether collaborative or co-productive research structures lead to more effective decision-making, or even better research. EBP and EBM are clearly different in terms of the machinery of decision-making, the types and number of actors involved, the structural constraints, the range of decisions to be taken and the diversity of evidence requiring consideration. Yet, constructing EBP as similar to EBM is understandable; it is a cipher connoting a recognisable process, and comes with a ready-made set of polemical devices and methodological tools. While both EBM and EBP share a desire to use evidence more effectively in decision-making, unpacking the history, decision-making processes and responses to some of the methodological and political challenges encountered by each, we have demonstrated that there are also fundamental differences. Using a typology of questions as a lens to consider which kinds of evidence may be useful, shows us that politicisation of a policy issue, alongside methodological concerns, influences what is considered credible and useful evidence. Advocates concerned with increasing evidence uptake, in either field, should draw on ideas about politicisation and legitimacy when devising strategies to increase research use.

As our discussion on politicisation shows, whenever a decision is taken, a problem must be framed in order to bound any discussion. This process of framing is an expression of power, and informs the selection of evidence types which will be used in any discussion (Boswell, 2014; van Hulst and Yanow, 2014). Yet, rare indeed is the expression of power which does not lead to the prioritisation of some interests over others. Thus, we contend that conflict is an inevitable part of both policy and clinical practice, due to the values and perceptions of the actors involved in each. Collaborative decision-sharing structures are, while a nice idea, still subject to the same group dynamics, personal biases and relationships as any other human interaction (Hendriks, 2009). Neither have we solved the problem of how far researchers should go to influence debates about policy and practice, without compromising their neutrality and commitment to strict empiricism (Cairney and Oliver, 2017; Smith and Stewart, 2017).

Thus, while much has been learned over the previous decades, we still do not understand well how the power dynamics of knowledge production inform evidence use, which forms of interaction between evidence users and producers are most productive, or the impact of evidence use on policy or patient outcomes. We argue that key dimensions for future enquiry include: understanding what evidence is and is not produced, who produces evidence, how evidence is framed, how evidence is incorporated into decision-making processes, and how all of these dimensions may be influenced by the types of problems policymakers and clinicians seek to solve. These are the outstanding questions for the field.

Data availability

Data sharing is not applicable to this paper as no datasets were generated or analysed.