In 1971, (Lord) Victor Rothschild, reported to Government on “The Organisation and Management of Government R&D” and how government could become (in his words) a customer for research contracted from the Research Councils and other sources. Rothschild’s thinking implied that management of R&D by “customer” Departments would bring an understanding of research outputs. He proposed the transfer of applied science funds from Research Councils to Government Departments, providing each with a Chief Scientist as proxy customer for research to be commissioned on a “customer/contractor” basis. The Government largely adopted his proposals in 1972 and implemented them in 1974. The Rothschild reforms and the upheavals they brought were controversial at the time, though now in some instances, reversed and otherwise either forgotten or buried in unconscious assumptions. However, the Rothschild framework still underpins important assumptions about Departmental relationships with the science community, which in my view adversely affected the access of Government to expert advice. From the viewpoint of a participant in Government R&D management through this period, I explore the immediate response and the post-1980 history of the Rothschild reforms, discuss the way in which research commissioning became such a heavy task as to impede analysis and advice delivery, and consider alternative approaches, such as the “science broker” model. This article is published as part of a collection on scientific advice to governments.
Lord Rothschild’s report to Government1 on “The Organisation and Management of Government R&D” came out in 1971, a year before I graduated. Its recommendations were largely accepted in 1972 and brought into force in 1974; I became a scientific Civil Servant the following year. In one way and another, my career was dominated by implementation of Rothschild’s ideas though, by the time I retired from the civil service at the end of 2012 (as Deputy CSA at the Department for Environment, Food and Rural Affairs, hereafter Defra), the report itself was, at least in Government, largely forgotten. I reflect here on my practitioner’s experience of implementing Rothschild’s principles and their implications, with particular respect to their effect on “evidence based” policy-making.
Though it caused an enormous furore at the time of its publication (which I outline below), Rothschild’s report is remembered, if at all, for three main reasons:
as the origin of the application of the customer/contractor principle to government-funded research
as an attack on the unity of research and the autonomy of researchers and the Haldane Principle, and
for its recommendation (accepted by Government) to transfer funding for applied research from three of the Research Councils to the control of the corresponding Government Departments.
However, Rothschild’s recommendation of the customer:contractor approach had a more important and lasting adverse effect on the ways in which Government interacted with the science community and consequently on the ability of government to access expert advice for policy-making; I will contend that, in making this a commercial transaction, government scientists took on a lot of contract management work at the expense of analysis and advice, while the research community became more focused on contractual delivery than dialogue with users.
I focus on the operation of Rothschild in two departments, the Ministry of Agriculture Fisheries and Food (MAFF) and to a lesser extent the Department of the Environment (DoE),2 the two departments which merged as Defra in 2001. This is both because that reflects my direct experience (I worked in MAFF and Defra for much of my career after 1983, and worked closely with colleagues in DoE) and because, in practice, these Departments were the most directly affected by Rothschild in the longer term.3 It is also the case that it was MAFF’s concerns in the 1960s about the unresponsiveness of the Agriculture Research Council (ARC) to Ministry and farming industry needs that provided the spark that led to the Rothschild fire.
Background—MAFF and the ARC before Rothschild
The history of relationships between MAFF and ARC concerning agricultural R&D is one of mutual suspicion (Williams, 1973; Thirtle et al., 1991; DeJager, 1993). ARC was determined to ensure that scientists had control over the direction of their own research (see for example, Henderson, 1981); MAFF was determined to ensure that what it saw as national needs should lead research funding (Winnifrith, 1962). The position by the 1960s was that control of R&D had passed from MAFF to ARC under the Agriculture Research Act of 1956.
In the run up to that decision, MAFF and its Minister had fought back at the efforts of the research council to take over:
It is notable that the Minister’s argument as reported is in essence the same as that which leads Rothschild to his famous recommendation that “applied R&D must have a customer” (HMSO, 1971; Rothschild report, Paragraph 8).
Intriguingly, though ARC’s principles for the development of agriculture research (Henderson, 1981: 91) were entirely focused on productivity and output, these concerns about the lack of responsiveness of ARC persisted (Ruttan, 1982). Arnon (1989) notes that MAFF tried to overcome this by parallel funding for applied research at the laboratories and experimental farms of its National Agricultural Advisory Services (NAAS)5 but that there was still poor articulation between “basic” research and applied and developmental research.
In due course, this matter was discussed at high level in Government and a senior official committee (the Osmond Committee, led by the Civil Service Department) was established in 1970 to assess approaches to improving the situation.6 Their report proposed to transfer ARC to MAFF; the journalist Tom Wilkie, in his critical account of “British Science and Politics since 1945” (Wilkie, 1991: 78) describes this as “an attempt by [MAFF] to hijack the [ARC]”, an overstatement, though it is clear that MAFF made a convincing case about its frustrations. The proposal was the “immediate cause” of Dainton and the Council for Scientific Policy (CSP) starting work (HMSO, 1971; CSP Report, Paragraph 2); being aware of the likely objections from that source, the Government also asked Lord Rothschild to consider the issue.
At this point, it is important to note that the whole of the discussion about the direction and management of agricultural R&D from the 1930s to the 1960s was focused on what we would now think of as research for improving agricultural production (Henderson, 1981; Ruttan, 1982; Arnon, 1989; DeJager, 1993). In that respect, there was no difference in long-term goals on the part of ARC or MAFF, only, and significantly, a different understanding of how this goal was best to be delivered. This point becomes important (1) because it points to an assumption underlying the Rothschild analysis about the purpose of agricultural research, and (2) in the context of subsequent changes to MAFF’s goals; in the later 70s and 80s, it became clear that agricultural overproduction was becoming a problem and new goals emerged to do with environmental protection, conservation and land management in particular.
Rothschild’s conclusions and recommendations
The Osmond Committee discussions can be seen to have led directly to Lord Rothschild, newly appointed head of Edward Heath’s Central Policy Review Staff (the “Think tank”; Blackstone and Plowden, 1988), being asked to undertake an enquiry into Government R&D in 1970.7 The report, when it came, was, in the words of a Nature (Anon, 1971a) editorial, “Rothschild’s Cat for All Pigeons” and provoked an expected uproar. The New Scientist (Anon, 1971b) called it “The Rothschild Bombshell”.
Rothschild’s two most important conclusions, typically expressed, were:8
On the strength of these conclusions, Rothschild recommended the establishment of a customer function for R&D in Departments to be supported by a Chief Scientist (II.10) to help customers (policymakers) decide on R&D programmes and their funding, and approve capital expenditure related to research (II.9).9 Crucially, he preferred the title of Chief Scientist “to that of Chief Scientific Adviser because he must be closely associated with the decisions in paragraph …. [II]9 … ”, that is, the Chief Scientist’s role was to be primarily that of an adviser on research commissioning.
This is the nearest Rothschild comes to referring to the policy advisory roles of scientists in Government. Indeed, he expressly dismisses it later (III.30) when, referring to a view of the CSP that executive [Government] Departments needed scientific support, he states flatly that they “do not need scientific support but applied R&D, to achieve specific, pre-determined objectives”.
The most controversial new departure was Rothschild’s examination (III.31 and table 4) of the current programmes of the Research Councils and conclusion that NERC, MRC and ARC10 should be subject to a 50%, 25% and 77% reduction, respectively in their DES vote, with corresponding increases in the votes of relevant departments (DoE, Health and MAFF11) to enable them to commission applied research. The customer:contractor principle “would have been uncontroversial had Rothschild not sought to apply it to the research councils” (Gummett, 1980: 52); after all, the Ministries of Defence and of Technology were used to this approach in their dealings with industrial research contractors. However, a “military-industrial-scientific complex” of the sort described by Edgerton (2006) was, if one substitutes “government” for “military”, largely absent in the agriculture and health sectors, and not a major component of environmental research; while significant industrial research occurred in, for example, agri-chemicals and pharmaceuticals, little or none of this was funded by the relevant ministries and departments, who focused their funding largely on either their own laboratories or Research Council institutes (Henderson, 1981).
That the move to the customer:contractor principle was made conditional on the establishment by departments of a proper Chief Scientist and customer function, and that a 4-year transition phase should apply during which Departments should commission research from the Research Councils from which the funds were taken, did not, as may be imagined, lessen the outrage caused by this proposal, to which I turn below. In Chapter V, Rothschild dismisses the relevance of the revered Haldane principle, on the basis of a careful reading of what Haldane actually said (HMSO, 1918), especially with respect to “research for General Government use”, as having “little or no bearing on the conduct and management of Government R&D in the 70s”.12
Rothschild’s other conclusions and recommendations included some brief comments on the coordinating role of the Government’s Chief Scientific Adviser (VI); note that this refers to research coordination, not to coordination of policy advice.
The reaction to Rothschild
Rothschild could be, and was, criticized on many grounds (Ashby et al., 1972; Gummett, 1980: 198ff, 1991) some valid, others less so. I briefly illustrate here some of the main lines of criticism in order to unpack some of the thinking behind Rothschild’s ideas. I then go on to identify what I think is a more important long-term issue that was not covered at the time, concerning the impact of Rothschild’s reforms on the ability of Government departments to access expertise.
The first and most enjoyable set of critiques emerges from the scandal caused by Rothschild’s language seen as, for example, “… terse … to the point of rudeness” (Winner, 1971). The starkness of Rothschild’s language was seen as unseemly13 in a Whitehall context though he also complained of being “accused of writing in unintelligible Whitehall jargon” (Rothschild, 1977: 84). Typically, he felt that had he written at more expansive length, “I suspect people would have read it even less carefully” (Rothschild, 1977: 88). He felt he was writing for busy people and much of what he had to say was “commonsense” or “self-evident”, an expression of which he was fond (Rothschild, 1972a). Rothschild himself later regretted his terseness and his choice of words; inter alia, he recognized the “emotive effect” of the choice of “customer” and “contractor”, telling the Select Committee that customer could be replaced by “user” and contract by “commission” (Anon, 1972a). At the time though, no one appears to have commented on how this could put researchers at arms length from their customers and contribute to a defensive “us and them” attitude on the part of researchers.
Another line of criticism is quasi-Burkean; Rothschild was “at odds with tradition” (Winner, 197114). and flouted the Haldane Principle (for example, Himsworth, 1971). At the time, even Nature dismissed this argument on the grounds that “the Haldane principle has become an archaic survivor from the past” (Anon, 1971d), an interesting opinion in view of the recent resurgence, some 40 years later, in reverence for this archaic Principle (Willetts, 2010).
Many (for example, Sutherland, 1971) challenged the reasoning, or rather absence of reasoning, behind the choice of the appropriate amounts of budget to transfer from the Research Councils to Government departments. This criticism hits home; Rothschild gives no explanation for the figures in his Table 4; it must have been based on some estimate of the proportion of the RC expenditure that was going to applied R&D but, for example, the Zuckerman Committee (HMSO, 1961), only a decade previously, had identified that around 55% of Research Council spending went to applied research, while Rothschild’s figures average out to around 25%. Subsequently, Rothschild (1977: 68–69) fairly off handedly blamed a slip between his draft and the published version of his report for “the important omission” of a tolerance of +/−15% around the 25% figure, without in any way explaining the reasoning behind this figure either. It is noteworthy that even by the time of the Government’s implementing White Paper (HMSO, 1972, Paragraph 50), the figures for the transfers had been reduced by (another unexplained) factor of around 30%. In reality, subsequent statistical summaries15 of expenditure on R&D show a general if uneven decline in MAFF and DoE, and later Defra spending, especially after 2000, matching a long term if erratic rise in AFRC/BBSRC and NERC expenditure; the Rothschild transfers have long ago been reversed.16
Outrage at the cuts proposed also led many commentators (a trenchant example is Dixon and Sherwood, 1971) to attack Rothschild on the grounds that, either, he implied that the Research Councils had in some way failed, or that, since they had not failed, there was not a problem that needed fixing. Rothschild (1977: 87) was at pains later to say that he did not think the Research Councils had failed; after all he had been Chairman of one, the ARC between 1948 and 1958, and had drawn attention even then to the lack of connection to farmers of the Agricultural Research Services (Henderson, 1981: 65). Rather, this was an issue about the purpose17 of and accountability for the expenditure of large amounts of public money and only covered applied research, that is, research undertaken for a specific social/industrial purpose (Calver and Parker, 2015: 7). In any case, according to the editor of Nature (Anon, 1972b) “to chose this point in political history for a plea for stronger control by customers of how public money is spent is likely to warm the cockles of … hearts in the … Government, which means that Lord Rothschild is in a strong tactical position”. And, one has to add, his proposals found political favour and were largely adopted (see below).
More subtly,Calver (2014: Chapter 4) argues that Rothschild’s thinking betrays an old fashioned inductivist understanding of science and contrasts this with the Popperian views emerging in the thinking of the Royal Society at the time, particularly under the influence of Peter Medawar. Medawar, on this view, saw Rothschild as attacking scientific autonomy, which he saw as essential if to the exercise of scientific creativity. Indeed, Rothschild’s very mechanistic account of the customer/contractor principle lends credence to such an interpretation. In practice, Lord Rothschild’s writings provide no indication of his views, if any, on the philosophy of science;18 his repeated references to “common sense” suggest he might well have made fairly traditional assumptions about science as a purveyor of factual knowledge rather than the more subtle Popperian view of science as tested but provisional understanding. This becomes important when the role of the scientist in policy formulation is considered below. But here it leads to another issue on which Rothschild’s account can be critiqued, also related to the basic/applied distinction.
This is not the place to go into the voluminous literature about the taxonomy of research per se. 19 Suffice to say that Rothschild’s Report appears to be based on a crude, linear model of how science works, which his distinction between “basic” and “applied” science betrays; Rothschild (1977: 85) says clearly that he expects application to be based on the insights of basic research; however, he does go on to point to the ability of applied research to generate issues for basic research and indeed, in an earlier lecture (Rothschild, 1977: 41ff) had expressly referred to the lack of value in making such a distinction, so it is worth examining the origins of his ideas on the management and purpose of research.
Rothschild’s model was very clearly based on his experience as Research Coordinator at Royal Dutch Shell; the address referred to above (Rothschild, 1977: 41ff), which he gave to the Imperial Defence College in 1968 on “Science in Industry”, prefigures many of the features of the Report.20 Shell’s research commissioning was on a customer/contractor basis (even with its own laboratories), and as Research Coordinator, Rothschild identified the “customer” with the heads of the industrial sections at Shell, not senior scientists, whose role was rather to help and advise on how to address the customer needs. More importantly, here, and in the Report, Rothschild implicitly allows for a dialogue between customer and contractor; the phrase “the contractor does it (if he can)” suggests that there may be some discussion about whether the problem set is soluble by science,21 which becomes an important element of the role of Chief Scientists. However, the point I wish to bring out here is that (whether or not he can be accused of linear thinking), Rothschild’s ideas were based on an industry22 model of the value and purpose of applied science, seen as a source of novel products or processes.
Which leads me, finally, to a different critique, not included among those addressed at the time or since, that Rothschild’s understanding of the needs of Government was limited and faulty. Rothschild’s familiarity with the world of business and its research needs, and the nature of his Research Council experience (given the strong post-war focus in the ARC on production; Henderson, 1981), would have led to a thorough understanding of one facet of government research, the promotion of industrial innovation. However, it would not have given him an opportunity to consider the needs of other aspects of policy-making that are less about the future development of novel products and processes, but more focused on analysis of current knowledge to address political problems or on horizon scanning of future social, economic, environmental and technical developments, which might have political significance. In the same address referred to above, Rothschild refers to his having had experience of the Government (Rothschild, 1977: 45) but this turns out to mean his experience in the chair of ARC, not directly with a Government department; his later experience of Government policy-making in the Think Tank was barely 6-months old at the time of the publication of the Report, and as indicated above, the Report was clearly based on prior ideas derived from his time at Shell. Indeed, though he commented on the scientific weakness of departments to the Select Committee (Anon, 1972a), this turns out only to refer to his conception of a Chief Scientist function with respect to R&D commissioning.
Most importantly, Rothschild goes so far as to dismiss the need for proper scientific support within Ministries on typically trenchant terms to the effect that departments do not need strengthening, they need good applied R&D applied to well-specified objectives (Report III:9). In this respect, Rothschild stands charged with grossly underestimating the need for scientific (and other expert) intelligence within policy-making. This is a surprising omission, given that such concerns had been a matter of recent public debate, particularly since the publication of the Fulton Report (HMSO, 1968).
The White Paper of 1972
Green Papers are consultation documents; The Lord Privy Seal’s introduction to A Framework for Government Research and Development (HMSO, 1971) contained what might be thought of as a spoiler; two paragraphs (4 and 5), out of only seven, let the Government’s views on the customer/contractor principle be known; though Paragraph 6 talked about the need for wide debate before conclusion on it, the likely outcome was well-prefigured by the Government’s welcome for the recommendation. And so it came to pass.
The White Paper, with almost the same title, presented to Parliament in July 1972 (HMSO, 1972), only 8 months later, accepted all the key Rothschild innovations, with mostly only minor amendments.23 By 1974, the transfer of funds had happened; by 1980 much of the hullabaloo had died down; the arguments against could be described as feeble and ill-coordinated (see Gummett, 1980: 229ff). Rothschild’s attacking style, getting in his counter-attack first, had left his opponents wrong footed; even the Editorials in the science journals (for example. Anon, 1972b) had found much of Rothschild’s case, especially for accountability, ungainsayable.
One important difference between the original report and the White Paper lay in how the role of the Government’s CSA and the departmental Chief Scientists is described. The GCSA “advises Ministers on the scientific and technological aspects of Government policies …. He will also be responsible … for advising on the way in which the new arrangements … [for R&D] … are working” (WP Paragraph 10). Correspondingly (WP Paragraph 24), MAFF’s Chief Scientist will “contribute scientific advice to the Ministry on the broad range of policy matters with which it deals. Second, he will have a central role in framing and reviewing the Ministry’s [R&D] programmes …”. Thus the advisory role of Chief Scientists was recognized here, even if not by Rothschild (though the equivalent paragraph on DoE—Paragraph 18—only refers to the customer function). In practice, the exigencies of managing the enlarged budgets were to reduce the time available for other activities.
Rothschild and MAFF and DoE—1974 to 84
MAFF24 and DoE25 rapidly appointed Chief Scientists of stature. Charles Pereira was appointed at MAFF (McCulloch, 2008); at DoE, Joe Lyons had been DG Research since its foundation and, in 1976, was followed by Martin Holdgate (Holdgate, 2003). Both were appointed at a very senior level, as Deputy Secretaries, that is, one rung down from the Permanent Secretary and with a board-level voice in departmental affairs. Both settled down to develop relations with their respective research councils (ARC26 and NERC, respectively). In this, they were supported by newly appointed teams to help the commissioning process; in MAFF, the Chief Scientist’s Group amounted to six very senior scientists (at then SPSO grade) as “Science Liaison Officers” and a complete administrative division to handle the procurement and commissioning process (Henderson, 1981). Similar arrangements applied to DoE’s CSG, though there was more reliance on scientists embedded within policy teams. This constituted a significant increase in scientific capability at head quarters, but it was more or less entirely focused on research commissioning.
MAFF’s relationship with ARC was at that stage relatively cordial (Henderson op cit.). Pereira (McCulloch, op cit.) was a distinguished scientist (FRS) who had been Director of ARC’s East Malling Research Station; his deputy on the agriculture side of his brief was also an ex ARC man, Frank Raymond, formerly of the Grassland Research Institute, and Pereira’s successor in 1977 was Bernard Weitz of the National institute for Research in Dairying at Reading University. Under Pereira’s wing, the relationship developed in a gentlemanly way through discussions with Institute Directors and the formation of “cognate programmes” of basic and applied research, which were still in play by the time the author joined the MAFF Agricultural Chief Scientist’s Group in 1988 under Pereira's later successor, Dr David Shannon, himself another ex ARC Institute Director (Roslin).
MAFF R&D budgets flourished through much of this period, with significant real terms increases, which were shared roughly proportionally between extra and intramural spend. However, the Chief Scientists did not gain control during this phase of the Ministry’s internal R&D spend, the relatively enormous sums being spent through the NAAS (which had its own significant research and field station capability), the Veterinary Service, the Directorate of Fisheries Research or the Food Science Laboratory, let alone the still larger sums spent on non-R&D S&T (monitoring and disease surveillance, inspection, chemical and biological analysis and so on).
DoE’s relations with NERC were initially difficult though settled later (Gummett, 1980: 204), and Holdgate (2003: 284) himself records very bumpy relations with NERC and records the experience of being Chief Scientist as “one of the least rewarding … in my professional life”. DoE’s budgets, though larger than in 1974, showed real terms fluctuations through this period, which cannot have helped. In addition, from even before the Rothschild Report, DoE had spread its contractor base much more widely than MAFF, beyond the Research Council and intramural spend (Everest, 1991).
By the early 80s MAFF was spending over £300m in today’s prices on research and DoE about half that. MAFF and DoE developed quite different models for managing their extramural research budgets; in the case of DoE, the Chief Scientist drew up proposals annually (rather as Rothschild had done at Shell, though this was probably not the explicit model) in consultation with policy colleagues who held the budgets, and had them approved by Ministers. In MAFF, by contrast, the CS, advised by a Joint Consultative Organisation (see Henderson, 1981; this later became the Priorities Board for Agricultural Research), which included industry and Research Council members, decided the budgets, which he held, and their allocations, subject to oversight by a Board R&D sub-committee. On the other hand, from the beginning of the 80s, Holdgate (2003: 225) held not only CS responsibilities, but was also in control of key policy functions (he was in charge of environmental protection policy) and of the strategic Central Planning Unit (not that he saw this as other than a burden); Pereira never had his hands directly on policy levers.
It is notable that throughout this period, and later, the Chief Scientists were organizationally separated from Departmental economists who, in both departments, were embedded in one of the policy DG commands. This separation from the social scientists attracted adverse comment (for example, Spedding, 1984), not least because the purpose of the R&D investment was still seen in essentially economic terms. Arguably, it vitiated the presentation of coherent and comprehensive evidence and advice to policymakers; I often had the impression that policymakers could, on the contrary, play different “experts” off each other.
The 80s through the 90s
By the late 1980s, the Chief Scientist role in both departments was downgraded. In DoE, Holdgate’s successor from 1988, David Fisk continued at Under Secretary level to hold a central policy role as well as the CS function, while in MAFF the DG-level post was split between separate, but lower graded Chief Scientists for Agriculture and for Food and Fisheries when Weitz retired in 1980 (Herman, 1981). Frank Raymond (Anon, 2012) and later Geoffrey Burgess (ex Agriculture Development and Advisory Service) and then David Shannon (from 1986) took over as Agriculture Chief Scientists; a Chief Scientist Food and Fisheries at the same grade matched them, occupied in sequence by George Elton, Tom Crossett, Mike Knowles and Howard Denner; the Fisheries function was moved to the agriculture CS in the 1990s. It was not until MAFF and DoE were merged to form Defra in 2001, that a board level CSA post was re-established.27
The advent of Mrs. Thatcher’s government brought further significant changes, which led in effect to a deepening and further embedding of Rothschild principles. Agar (2011) interestingly identifies Mrs. Thatcher’s engagement with Rothschild in her role as Secretary of State for Education in Heath’s Government, at a key meeting in 1971, as a critical point in her development as a Thatcherite; Rothschild, supported by Heath, making the case to her for a market mechanism for research, may have changed her view, on science funding for sure but possibly also on wider issues, towards market solutions. From having supported a “Haldane” line presented by her officials, she moved to support Rothschild. “Years later”, she wrote in her autobiography (Thatcher, 1995: 174–175), “I was able to formulate my own answer to the problem, which is that Government should concentrate on funding basic science and leave its application and development to the private sector”; the “answer to the problem”, when it came, became known as the Near Market Cuts.
The “Barnes Cuts”, as the near market cuts applied to MAFF R&D budgets 28 were known, after the civil servant who led the process, removed some 30% of funding by Government Departments over a short period in the late 1980s to early “90s (Hansard, 1988; House of Lords, 1989; Read, 1989); while the policy was based on the ‘near market’ thinking, a clear money saving target was implicit in Barnes’ terms of reference from the start (C.J. Barnes pers. comm.). My personal observation is that much of the cutting proved to have applied to projects not only of merely mundane scientific interest but also of little interest to industry;29 only a limited amount of this axed research was in practice taken up by the private sector when Government stopped funding, despite the efforts of, for example, the agricultural Levy Boards (Read, 1989). This implies that, pace Rothschild’s principle that Departments should be in a better position than Research Councils to decide on the applied research priorities of the agriculture industry, they had not been very effective at it, and had not noticed the failures of uptake. Much of the cutting fell on in-house research projects but a significant proportion came from Research Council Institute activity, often lineally descended from research being undertaken before the Rothschild reforms, which does not say much in turn for the Research Councils either, as choosers of applied research objectives, vindicating the views of Nature (Anon, 1981)
The Near Market Cuts themselves were linked to another initiative designed to put much Government policy implementation action on a contractor basis, the Next Steps Initiative (Efficiency Unit, 1988). In line with its proposals, “arms length government” became the norm, under the principle that Government could hold contractors to account for agreed programmes of work. Privatizing public sector activities where possible, and where not, turning Government functions over to Next Steps Agencies, was the tool to enable this contractorisation, even if the agencies, being fully owned by Government, could only operate in a quasi-contractual way, with the main sanctions being exercised though the now more tenuous positions of their Chief Executives.
In the case of Government Laboratories, some, such as DoE’s Buildings Research Establishment and MAFF’s Plant Breeding Institute and the Agricultural Development and Advisory Service, could be fairly easily fully privatized but quite a number of labs remained in Government hands because, in practice, they had an important role in providing specialist scientific services (rather than research) in support of policy-making and implementation. For example, MAFF’s Directorate of Fisheries Research was “agencified” as The Centre for Environment, Fisheries and Aquaculture Science (CEFAS); while it was, and remains, the only significant English centre of fisheries research, and an important contributor of marine environmental research, its main functions had to do with the scientific advice that went into fisheries and marine pollution management and the licensing and inspection of, for example, fish farms.
One consequence of this was that the Chief Scientists in MAFF finally came to engage with the whole of the R&D budget, not only extramural but also with the agencies (though not, at this stage, the budget for other non-R&D scientific services provided by the agencies, which accounted for more than half of their budgets and remained under the direct control of policy customers). However, at the same time, the Rothschild principles were applied more tightly and the responsibility for the R&D budgets in MAFF, as had been the case from the beginning in DoE, was given to policy Under-Secretaries, albeit with a double-lock from the Chief Scientists, whose function became more that of advising on research plans and especially on advising on the overall research strategy of the Department. In the early 1980s, Nature (Anon, 1981) could wonder whether MAFF was “Beating a Retreat from Rothschild”, noting that there had been little change in the structure of AFRC programmes since 1972, but by the end of the decade, Shannon could be cited by Arnon (1989) as indicating that customers were more satisfied with the value of AFRC’s work for them, and that MAFF had had a significant influence on how AFRC had applied the Cuts.
Alongside the Near Market Cuts and Next Steps moves, the Cabinet Office and the Treasury, with the help of Department of Trade and Industry (DTI), was sharpening up its guidance on the management of contractors (Cabinet Office, 1989a). The primary impact was to break up the older-style, relatively large and strategic programme Commissions, mainly focused towards individual Institutes, and recast them as much smaller and more narrowly defined objective-driven projects, which might link PIs in several institutes. This combination of changes realised some of the fears of critics of Rothschild from the beginning, that Government departments would not manage research strategically. Certainly, they caused significant difficulties for research institutes when coherent programme units were broken up, and individual PIs found themselves loosing elements of work which made their posts less viable (Read, 1989); Institute Directors lost a significant degree of control as a result.
From personal observation, I would say that a corresponding problem for the commissioning departments was that the move from macro to micro-management substantially increased the cost and effort involved; it brought many more Ministry scientists into the management of commissioning (science staff numbers in MAFF roughly tripled in this period and the administrative resources had to be increased as well). Budgets were high at the time, even after the 30% cuts, and that meant that more effort had to be put into commissioning larger numbers of smaller projects to ensure the money was spent.30 Less effort proportionally was put into monitoring the use of the funds and even less into analysing the policy relevance of the output and evaluation. In other words, the scientists became commissioning machines and tended to lose their advisory function for lack of available time. Further, because the CSG model kept scientists in a central team, they remained at a distance from the policy-making Divisions, at least in MAFF;31 once again, DoE’s model, which had most scientists bedded out in policy units had the advantage of bringing them closer to policy, even if much of their effort was still focused on commissioning.
The Government re-affirmed its support for the Rothschild principles in 1987 in its response to a House of Lords Select Committee on Science and Technology Report (HMSO, 1987), and then again in the 1993 White Paper, Realising our Potential (HMSO, 1993). This latter was overseen by William Waldegrave as the Minister then responsible for science; not surprisingly, given Waldegrave’s personal closeness to Rothschild and memories of the emergence of the Report while he was in the CPRS, as well as the general tenor of Government policy at the time, the White Paper specifically re-endorsed Rothschild’s customer: contractor principle, on the basis of advice from the Prime Minister’s Adviser on Efficiency and Effectiveness, Sir Peter Levene and the then Chief Scientific Advisor to Government, Professor, later Sir William, Stewart (Levene and Stewart, 1993).32 Chapter 5 of the White Paper addresses Civil Department science; this was still very research focused (barely touching on the non-R&D scientific and technical activities of departments) and, like the rest of the White Paper, was largely concerned with wealth creation. It did however, while emphasizing the policy customers’ role in commissioning research, draw attention to the strategic role of the chief scientists and the importance of their networks, and explicitly repeated the 1972 White Paper’s reference to the advisory role of science in policy-making.
Rothschild forgotten?—the late 90s and the new millennium
If the effect of the Rothschild reforms had in practice been to focus Chief Scientists and their staff on research commissioning at the expense of giving scientific advice to policymakers, this all began to change in the wake of the great BSE crisis and the Phillips Inquiry (BSE Inquiry, 2000). Lord Phillips, among other things, gave a substantial amount of advice on how scientists should advise policymakers and, likewise, how policymakers should engage with scientific advice. Some of this was codified at an early stage by the then Chief Scientific Adviser to Government, Sir Robert May, in what became known as the May Guidelines, which have been successively updated by Sir David King (in the wake of Phillips) and Sir John Beddington more recently (the latest version is at Government Office for Science, 2012). The full detail does not need to engage us; the interesting issue from the point of view of this article is that the role of scientist in government becomes refocused towards giving policy advice and away from the commissioning of research, which becomes a secondary and supporting activity.33 Chief Scientific Advisers once more replace Chief Scientists as the senior scientists in Departments.
With the abolition of MAFF following not only the BSE crisis but also the debacle of the Foot and Mouth outbreak of 2000, and its backing into DoE to become the Department for Environment, Food and Rural Affairs, a single Chief Scientific Adviser, Professor (later Sir) Howard Dalton FRS (Murrell, 2008), was appointed, once more at Board level as a Director General; I was his deputy as Director for Science. Over the subsequent 10 years, first with Howard and then with his successor Professor (later Sir) Bob Watson CMG FRS, we restructured the combined MAFF and DoE science efforts, taking advantage not only of the new regime engendered by the Guidelines but also of the Labour Administration’s support for “Evidence-Based Policy Making” (Smith, 1996; HMSO, 2001). In summary, over the next 10 years, we:
Put responsibility for science investments and for scientists with policy-making teams, to ensure that policy-making was fully evidence-based
Coordinated specialist advice whether scientific, social scientific, statistical or economic, within policy teams under common leadership
Developed a system for coordinating Defra’s total investments in evidence gathering (from advisory committee work through analysis, research and data gathering) through strategic investment planning (the Defra Evidence Investment Strategies, for example, Defra, 201134), linked to Government Spending Cycles)
Developed Head of Profession arrangements to ensure specialists retained and developed their particular professional skills
Promoted an outward looking approach to networking internal with external expertise in the Universities and institutes
Putting it another way, we reversed Rothschild’s view that all that departments needed was a good well-specified applied research and reasserted the primacy of analysis and advice. In this respect, the dark clouds of departmental R&D budget cuts, especially since 2001, can be thought as bringing with them a slightly golden lining for departmental science staff of greater engagement with the policy process, more time for analysis and advice, and more priority-driven, less expenditure-driven research commissioning.
Analysis and conclusions
To conclude, Lord Rothschild’s intervention had the very positive effect of bringing accountability into the governance of publicly funded research, an issue with which we are still struggling with today. On the other hand, his ideas, and possibly even more, his challenging way of presenting them, had the unfortunate effect of encouraging academia into a defensive mode, protecting the purity of research and the values of academic freedom, without getting government into a place where it could use the massive strength of public funding to engage expertise on important immediate and long-term national needs. When researchers and commissioners alike are focused on the contract, open, iterative dialogue suffers.
Rothschild never really addressed the question of the function of scientists in government policy-making departments; his mind was wholly focused on research, not only because of the question put to him and the CPRS, but also because of his previous Shell experience. Industry is actually very forward-looking and willing to wait for research results to flow in but policymakers in general cannot afford to wait for the slow products of research to emerge; “A week is a long time in politics”, as Harold Wilson may have said, and politicians often need the best available advice now to take action on the policy problems of today. Research remains essential—there will always be gaps in knowledge to be filled—but the real value of research to Government lies less in its outputs than in the availability of researchers, in other words, those with the expertise and understanding of an issue gained from undertaking research, who are therefore at the leading edge and have the most up to date current, though inevitably incomplete, knowledge. Rothschild’s emphasis on research commissioning focused too much effort on the wrong issue; the re-assertion of the analytical and advisory role of specialists actually addresses issues raised as long ago as the Fulton report (HMSO, 1968), with its call for multidisciplinary policy teams to address the multifaceted problems of today.
Fulton, and several of those who submitted memoranda to his enquiry, also argued for what we now call “Open Government”, and in particular for in-house specialists to be able to engage with specialists in the universities and institutes. Notably, the evidence of W.S. Ryrie (HMSO, 1968 op. cit.: Vol 5(2) Memorandum 144), a Treasury Assistant Secretary at the time, identified the importance of “the way in which issues increasingly converge and interlock”—what we would now call “wicked problems”—and argued for a more collective approach to decision-making and for the use of “outside experts … ad hoc as much as possible”. This was and is seen as an antidote to group think and inward looking analysis, and in particular to the category of understanding that Rumsfeld (2002)35 did not refer to, the Unknown Knowns—in other words the things we do not know about but someone else does; a particularly egregious trap for politicians. In modern terms, we are talking of a brokering role (Pielke, 2007) for departmental scientists, though unlike Pielke, I see this as a role for insiders facing out, listening and guiding, rather than outsiders facing in and trying to sell a particular scientific line (Parker, 2013).
The role requires a significant degree of both scientific and policy understanding, with a particular focus on horizon scanning and risk assessment. It also requires an unusual set of personality characteristics and aptitudes, including an ability to listen and to communicate, translating between disciplines. In turn, delivering people who can fulfil this sort of role calls for a new approach to the education of scientists, to expose them to policy issues and enable development of these skills (Parker, 2015).
Data sharing not applicable to this article as no data sets were analysed or generated during the current study.
How to cite this article: Parker M (2016) The Rothschild report (1971) and the purpose of government-funded R&D—a personal account. Palgrave Communications. 2:16053 doi: 10.1057/palcomms.2016.53.
Victor Rothschild’s report was published with another on The Future of the Research Council System from the Council for Scientific Policy (CSP), under the chairmanship of (then) Sir Frederick Dainton, as a Government Green Paper, with a preamble from the Lord Privy Seal (then the Minister responsible for science), under the general title of A Framework for Government Research and Development (HMSO, 1971).
The Department of the Environment, founded in 1970, initially had responsibilities for local government, planning and transport as well as environment protection. In 1997, it was renamed the Department of the Environment, Transport and the Regions. In 2001, DoE’s environment portfolio was joined together with the agriculture and fisheries portfolios of MAFF to form Defra. Before Defra, I refer simply to DoE.
The rather different history of Rothschild and the Health Department and MRC is told by Duffy (1968), Whitehead (1978) and Kogan and Henkel (1983); in brief, the new arrangements rapidly collapsed and the funds were ultimately returned to MRC, to be managed through a new Concordat with the Health department.
Sic. I discuss later the strong emphasis on agricultural improvement (productivity improvement) and its implications, as well as alternative policy goals.
From 1971, NAAS was rebranded as the Agriculture Development and Advisory Service (ADAS).
The Committee’s report was not published but its papers can be found at the National Archives BA17/293 (CSD papers) and MAF 117/540 (MAFF papers).
Rothschild records that this was among the subjects addressed to the Think Tank (Rothschild, 1977: 115); in actual practice, he took on this task alone and the CPRS was not formally involved (Williams, 1973; Lord Waldegrave Pers Comm). Rothschild told the House of Commons S&T Select Committee later that he was assisted by one member of CPRS staff and Rose (2003: 182) states that the report was “toned down by the fluent Whitehall hand of John Mayne” (an MoD official seconded to the Tank), though there is little sign of any such down-toning (see also Blackstone and Plowden, 1988: 142). Hennessey (1989: 233) goes so far as to say that Trend would not allow the report to be published under the CPRS banner.
Numbers refer to Chapters and paragraphs of Rothschild’s report in HMSO, 1971, thus II.6 refers to chapter II, Paragraph 6.
He also recommends another post of Controller R&D to provide management of the supply side of research (II.12). The two are to have no line relationship (II.11), implicitly to protect independence.
The Natural Environment, Medical and Agricultural Research Councils, respectively. The Science Research Council (SRC) was exempted, on the grounds that it is “largely concerned with pure, and to a lesser extent applied science, which is not synonymous with research”, likewise the Social Science Research Council (SSRC), on the grounds (III.25) that “it is in its infancy”.
These were not one-for-one transfers, some funds also went to the DTI and other departments.
Rothschild must have been aware of the relatively recent elevation of Haldane’s views to a principle by Lord Hailsham (see Edgerton, 2009), though he never mentions this.
The contrast with the episcopal style of Dainton and the CSP, who open with a long quotation, of resoundingly false humility, from the 1662 Preface of the Book of Common Prayer (HMSO, 1971; see, for example, piii of the Dainton report), no less, is striking (not to say unintentionally hilarious); though many commentators agree that Dainton’s report was disappointing, said little that was new, and said it blandly, it was nonetheless more soothing to the research community (See, for example, Sutherland, 1971; Anon, 1971b, c).
Dame Albertine was Britain’s first female DCMO, a post in which she served for 20 years, http://munksroll.rcplondon.ac.uk/Biography/Details/4846
I have used the cash and real terms figures in the Annual Reviews of Government funded R&D for 1983/4 and 1988 (Cabinet Office, 1984,1989a, b), the Forward Look: Government funded Science, Engineering and Technology for 1995 (Office for Science and Technology, 1995) and the Statistical bulletin: UK Government Expenditure on Science, Engineering and Technology for 2013 (Office for National Statistics, 2015), each of which contain past out-turn and planned forward spends, to examine the trends. Changes in the machinery of government on the one hand, and of the remits of the research councils on the other, make any numerical calculation very difficult, but the overall pattern is clear.
Not that the scars have entirely healed; in my personal knowledge, within even the last decade, one Research Council Chief Executive wrote to the then Chief Scientific Adviser to Government to seek return of the funds that had been transferred to a government department almost 40 years earlier following Rothschild’s recommendation.
Rothschild’s emphasis on managing expenditure in relation to the purpose of the investment found official form in the “Primary Purpose” statistics incorporated from 1985 in all Government R&D expenditure statistics (the definitive version of the Primary Purposes is in Cabinet Office, 1991); note that these include (ppB and C) research for government policy and service delivery as well as for technology development (ppD).
Lord Waldegrave (Pers. Comm.) who was close to Rothschild both personally and as a member of CPRS (and was a later successful minister for science) is quite certain that Rothschild had no philosophical ideas about science.
Much ink is spilled by both Rothschild and Dainton (and others) on the taxonomics of research; see in particular, Rothschild’s magisterial and typically rambunctious dismissal of much of it in Forty-five Varieties of Research (and Development) (Rothschild, 1972); out of the 45, he allows only two—basic and applied research. Note that he uses the term “pure” research in place of “basic” in the Report. In reality, the two sides are talking past each other; Rothschild admits the interactions of all types of research (that is, the essential unity of science) but is referring to the purpose of funding particular projects while the Research Councils are concerned with control over scientific agendas.
Including his allowance of a funding margin (of 11% at Shell; 10% in the Report) for what he calls General Research –“fundamental research in an area relevant to the business but in which there are gaps which have not been filled at Universities or elsewhere”.
Rothschild initially spoiled his case here by insisting that research councils should not turn down requests from departments (Report III.30), one of the rare elements of the report on which he later recanted.
Applied research commissioning was not in reality new to government, notably in the defence sector (see Edgerton, 2006), though much of this related to contracts to industry, which was used to this model, rather than to academia and the Research Councils, which were not.
One of the more important being to allow the Research Councils to turn down proposals from Government departments (HMSO, 1972: 53)
I do not deal here with the rather different way in which agricultural R&D in particular was handled in Scotland.
Rothschild transfers also went to MRC (see Footnote 3) and DTI (where they were focused on technology development) and to a much smaller degree to some other Government departments.
Later to become the Agriculture and Food Research Council (AFRC) in 1983, then the Biotechnology and Biological Science Research Council (BBSRC) in 1994.
The Chief Scientist (Food) went to the newly created Food Standards Agency.
The Barnes cuts also applied to Scottish agriculture research spending.
It is important here to distinguish research funding cuts from the privatization of whole or parts of research institutions like the Plant Breeding Institute or the Soil Survey of England and Wales, which continued to operate, albeit in a new context and management. I touch on the privatization issues below.
Perversely, subsequent cuts in budget over the next decades required maintaining a significant amount of effort themselves; it is harder to cut than to increase activity, at least if long-term strategic damage is not to be done to the supplier base.
An effort (which I led) was made in MAFF in 1989 to set up the recently emerged Under Secretary command on agricultural environment policy with its own scientific staff; this succeeded to the degree that a single scientist was appointed to the policy group, and later gained a small staff, but the budget continued to be managed by the CSG, albeit on behalf of the policy customer.
In any case, Lord Waldegrave (Pers.Comm.) considers that by this time, Rothschild’s ideas had become part of the aquis.
However, as late as 2012, the House of Lords Select Committee on Science and Technology (House of Lords, 2012) expressed their view of the role of the Chief Scientists (Paragraph 122) in terms as much of research commissioning as of evidence gathering, though they did stress the importance of the Chief Scientists having an understanding of the policy environment (Paragraph 124).
The Evidence Investment Strategies expand on the bullets in this list, explaining the philosophy and actions more fully.
Donald Rumsfeld, US Secretary of Defence during the Iraq war in 2002, while seeking to explain the background to the problems with WMD, said “As we know, there are known knowns; there are things we know we know. We also know there are known unknowns, that is to say, we know there are some things we do not know. But there are also unknown unknowns, the ones we don’t know we don’t know” (My emphases).