Article | Open | Published:

Measuring impact in the humanities: Learning from accountability and economics in a contemporary history of cultural value

Palgrave Communicationsvolume 3, Article number: 7 (2017) | Download Citation

  • A Correction to this article was published on 27 February 2018

Abstract

This article addresses the future of research assessment within higher education in the UK from a humanities perspective. Recent changes to policy (such as The Browne Report 2010 and the 2014 REF) indicates that humanities research is increasingly required to provide quantifiable or commercial results in order to attain value. Although research assessment exercises have been a formal part of UK higher education since the first Research Assessment Exercise (RAE) in 1986, the last 6 years have seen a significant change in how research is valued within the academy. Specifically, this paper responds to the increasing prioritisation of 'impact' measurement in research assessment criteria. The article situates recent changes in higher education within a historical context of cultural policymaking in the UK from the 1980s to the present day. Such an undertaking highlights the specific challenges and nuances within the shift towards 'impact'. Firstly, this paper details how public cultural institutions (such as museums and art galleries) became subject to practises of economisation and social accountability as a result of 1980s cultural policy. A rich field of literature from museology and arts management provides valuable sources and testimonies that should be considered in the future of the academic humanities. Secondly, this paper considers the implications of the creative industries upon the perception of knowledge production since the 1990s. Following this specific history of cultural assessment mechanisms in the UK, this article concludes by demonstrating that neither the adoption of a purely economic approach nor a refusal of accountability will serve the humanities. Whilst there is a wealth of social science research that explores valuation methods and assessment culture there is a lack of humanities research within this vital debate. This article presents a response from a humanities perspective. As a result, this contribution raises awareness of the urgent need for humanities scholars to engage in these emerging and significant debates concerning the future of research assessment in the UK.

Introduction

This paper addresses the rise of 'impact' within research assessment criteria in the United Kingdom through an exploration of the historical precedence of the mechanisms by which value is measured. The need for such an enquiry has never been greater, given the escalation of accountability metrics within higher education and the consequent demand for scholars to perform within such frameworks. This paper develops a narrative which challenges the unquestioned shift towards 'impact' in the sense it is currently understood in higher education policy. At present, there is a notable absence of analysis that draws on historical perspectives. Humanities-based inquiry into the “impact of impact” has largely been informally presented: in comment articles, in discussions at symposia, and at conference plenaries.Footnote 1 To date, a handful of academic journals have dedicated special issuesFootnote 2to address the rise of impact agendas within the humanities. This paper calls for a sense of ownership in this critical debate and encourages future evidence-based discourse within humanities disciplines. I offer an articulation of the implications caused by the changing field of research assessment criteria in the United Kingdom that affects the humanities, from a specifically humanities perspective.

This article establishes an analytical account of the measurement of the public value of impact within cultural institutions in Britain. I argue that the decisions being made in higher education policy have close ties to cultural policymaking approaches in contemporary history. Therefore, close rhetorical analysis of debates in arts management and museology provides a relevant framework with which to view the emergent research assessment models (Benneworth, 2015) concerning 'impact' within higher education. Such deliberate comparison between the methods and mechanisms of valuing culture in the United Kingdom enables a means to articulate effectively the future of research assessment from a humanities perspective.

To date, analysis of research assessment mechanisms and its their implications for the humanities has been most extensively researched using social sciences methodologies.Footnote 3 Ochsner, Hug and Daniel detail how SSH researchers have used interviews, bibliometric and scientometric literature to address the challenge of capturing the value of the humanities within assessment frameworks. Their paper 'Indicators for Research Quality for Evaluation of Humanities Research: Opportunities and Limitations' highlights a 'missing link between indicators and humanities scholars’ notions of quality' (2012, p 1–4) and identifies that 'bottom-up processes (p 4–5) in which scholars have a stake in the definitions of quality are required in order to implement meaningful change. They suggest that an engagement with specific disciplinary needs might be a means by which this dissonance, between assessment and the assessed, might be better addressed. Similarly, Ochsner, Hug and Galleron developed these findings to highlight European initiatives that 'assess SSH research with its own approaches instead of applying and adjusting the methods developed for and in the natural and life sciences' (2017, p 9). Such research points to the importance of scholars attending to research assessment processes that respond to and reflect their own position.

In response to this research, this article offers a humanistic contribution to the results that SSH research has identified. It seeks to enact what Ochsner, Hug and Daniel describe as a 'bottom up' (2015, p 4–5) approach to the future of research assessment in the humanities, starting with the basic tenets of humanities research. In doing so, this article follows Helen Small, Professor of English Literature at the University of Oxford, who describes:

In the main the humanities value qualitative above quantitative reasoning; they place greater faith in interpretative than in positivistic thinking; unlike the sciences and the scientific wing of the social sciences they do not have a dominant methodology, and many of their truth claims are not verifiable as those of the natural sciences are verifiable; they tend, accordingly, to distrust proceduralism and to value independence of thought. […] They attend to the role of the perceiver in ascertaining even the most philosophically secure of knowledge claims; and they have an interest, often they also take pleasure, in the specificity of the object of study and the specificity of the individual response (its content and its style) over and above the generalized or collective response. (2013, p 57)

As Small outlines, this article will utilise such humanities approaches of 'qualitative' reasoning and 'specificity of the individual response' (2013, p 57) in order to demonstrate how impact might be understood within the contemporary moment within the United Kingdom.

Enacting Small’s methodological focus, this paper details research assessment criteria in the United Kingdom in order to speak with precision about specific policy decisions regarding 'impact' since 2010. Impact, and more broadly social accountability of humanities research, is set to maintain a central position within future research assessment criteria in the United Kingdom. As a result, such a discussion of impact within academia is of global significance, particularly within Europe and Australia.Footnote 4 This article offers one qualitatively-driven contribution to a debate that reaches beyond any singular research framework, methodology, or indeed, nation. The specific context which this paper addresses is the marketisation of undergraduate tuition under the Browne Report (2010) and the first cycle of the Research Excellence Framework (2014 REF henceforth). The following section outlines the particular relevance of this context to the broader discourse of 'impact' within research assessment culture worldwide.

Context of 'impact': The marketization of higher education in the United Kingdom

In the UK 2010 to 2014 should be understood as a significant watershed in terms of the assessment of the value of higher education. This brief section outlines two significant policy changes within higher education in the United Kigdom: the publication of The Browne Report (2010) and the introduction of the 'impact' component of the REF in the 2014 cycle. Regenia Gagnier observes that at this time, 'traditional markers of academic distinction [were] overtaken by internally established criteria of worth [in] compliance or alignment with the University’s competitive drive in a global Higher Education market’ (2013, p 11). The white paper commonly known as The Browne ReportFootnote 5 promoted significant changes to the funding of higher education, including the privatisation of university tuition.Footnote 6 Thus undergraduate tuition was transformed into a commodity in a worldwide marketplace, with HEIs become the providers of varying standards of education at varying prices (Sandel, 2013). Prioritising the financial value of education was a key motivation behind The Browne Report. From 2010, there is an observable increase in the number of league tables and statistics, which attempted to categorise, evaluate, and substantiate the value of specific HEIs for the student–consumer (Molesworth et al., 2010; Brown and Carasso, 2013; McGettigan, 2013). The expansion of such calculable values within research assessment expanded within this competitive intra-institutional context.

The 2014 REF exemplifies the prioritisation of research that offers a similar form of measurable public accountability. Within UK higher education, the formalization of 'impact' as a category of research assessment dates back to 2010 with the introduction of the new Research Excellence Framework (REF henceforth) criteria.Footnote 7 Its predecessor, the Research Assessment Exercise (RAE), which ran approximately every 6 to 7 years from 1986 to 2008, did not feature a specific measurement of impact.Footnote 8 The 2014 REF submission guidelines define impact within academic work as being “of direct relevance to the needs of commerce, industry, and to the public and voluntary sectors; scholarship; the invention and generation of ideas, images, performances, artefacts including design”. Within this context, impact is defined as knowledge that is immediately useful to society. Note that the 2014 REF was not the first time that 'impact' was seen as a valuable component of research assessment criteria. The New Production of Knowledge (Gibbons et al., 1994) outlines similar changes to the valuation of higher education practices; they describe a model, Mode 2, which was an emerging form of knowledge production within higher education institutions (HEIs henceforth). Mode 2 includes criteria to assess 'efficiency or usefulness which are defined in terms of the contributions the work has made to the overall solution of transdisciplinary problems' (1994, p 18). Although their text is largely focused upon scientific research, it does make 'The Case for the Humanities'. Further application of Mode 2 will be discussed in relation to the creative industries below (2016, p 17). The legacy of Mode 2 in which 'knowledge is intended to be useful to someone whether in industry or government, or society more generally and this imperative is present from the beginning' (1994, p 4) is evident in the language of the 2014 REF.

A focus on such public accountability is further underlined by those areas expressly excluded from the REF’s definition of impact: 'the advancement of academic knowledge within the higher education sector' and '[i]mpacts on students, teaching or other activities within the submitting HEI' (2011, p 48). This article responds to this specific definition and exclusion of 'impact' and the subsequent implications upon the academic humanities in the UK. The activities that occur within HEIs are not seen as an impactful result. The REF criteria for impact values the form of the output, as opposed to the academic quality of the research. Stefan Collini, Professor of Intellectual History and English Literature at the University of Cambridge, has been a vocal critic of the changes to valuation in these terms. He argues that

[i]nstead of proposing that ‘impact’ of this kind is a desirable social good over and above the quality of the research, the exercise makes the extent of such impact part of the measurement of the quality of the research. In terms of this exercise, research plus marketing is not just better than research without marketing: it is better research (2009, p 19).

Collini reasons that assessors see impact as an indicator of value in and of itself. The focus on publically accessible outputs means that a variety of humanities research is negatively affected. Collini states how increasingly, it 'is assumed that the only way to justify what goes on “inside” universities is by demonstrating impact that happens “outside”  (2009, p 19). Further evidence of this logic of accountability through public interaction is identifiable in the current mission statement of Research Council UK: 'Ensuring Excellence with Impact'. Such a statement reinforces the central role of impact in the evaluation of research excellence in the UK. The rhetoric of 'economistic officialise' (2009, p 19) that Collini critiques points to an outcome-led valuation of the humanities. To this end, he enquires

[p]erhaps our ears no longer hear what a fatuous, weaselly phrase ‘Research Excellence Framework’ actually is, or how ludicrous it is to propose that the quality of scholarship can be partly judged in terms of the number of ‘external research users’ or the range of impact indicators (2009, p 19).

The specific language used to describe the impact of the humanities demonstrates an integration of business and management models. The accusation stands that the mechanisms of research evaluation are insufficient for measuring the values of cultural, creative or humanistic work.

This accusation, specific to the academic humanities, will now be contextualised within its historical context of cultural policymaking in the UK. Drawing a parallel between debates in accountability within art galleries and museums, I map emerging critiques from the humanities onto a broader history of cultural policy. In this way, the intersections between 'impact' and 'accountability' will be made explicit and put to work. The following two sections highlight key moments in the development of cultural valuation in the UK: firstly, the economisation of the value of the arts under Margaret Thatcher; secondly, the emergence of the creative industries under New Labour. Analysis of these contexts of valuation enables us to better interrogate the current context of measuring the impact of the arts and humanities within HEIs. Building on the evidence that this cultural history provides, this article concludes that assessment criteria is significantly limited and there is a need for bottom-up (Ochsner, Hug and Galleron, 2016) studies concerning measurement of the impact in the humanities.

Arts and the economy: The historical foundations of an output-led assessment of cultural value

A dominant narrative concerning the measurement of cultural value finds its roots in the policymaking approaches of Thatcher’s government (1979–1990). The means of evaluating cultural organisations were drastically altered (Halsey, 1997; Belfiore and Bennett, 2008). Thatcher’s administration was responsible for substantial cuts to the arts sectorFootnote 9, however, what has arguably caused more long-lasting effects was the economisation of the value of the cultural institutions. The 1980s saw the rhetoric of economic justification becoming a formal requirement for government subsidy for creative and cultural ventures. This section documents how policymaking decisions made under Thatcher’s leadership established economic value as the dominant way to account for the impact of the arts and how this persists in the present cultural sector.

The publication of The Economic Importance of the Arts in Britain by the Policy Studies Institute evidences the shift towards economic valuation of arts and culture industries in the 1980s. John Myerscough’s study (1988) argues that the value of the arts is best articulated in economic terms. Myerscough examines how the arts should best be valued 'as a form of productive activity, in terms of levels of employment, income generation and patterns of economic organisation' (1988, p 3). Cultural critic, Eleonora Belfiore clarifies that although Myerscough’s study was heavily criticized by many working in the cultural sector at the time of publication, it 'ultimately glorified' an economic method of justification that still dominates today. Belfiore explains how Myerscough 'opened the way to an increasing number of similar studies claiming to be able to prove and measure the importance of the arts sector to the local and national economy' (2003). Myerscough provides the following formula for calculating the value of culture in society:

The value of the incremental multiplier is represented by

Direct + Indirect + Induced Direct

(1988, p 97)

In précis, the formula expresses that if a museum is making more money than the employees are being paid, and the exhibit itself costs to maintain, then it is valuable. Alternative values that a cultural institution might offer are not accounted for. This logic is seen to have long-term implications on the cultural sector. Theatre critic Michael Billington’s obituary for Thatcher captures the endurance of the economic approach. Writing in The Guardian, Billington argued the Prime Minister was responsible for 'the growth of a siege mentality in arts organisations' and that her legacy is identifiable 'in the frightening fact that we are still having to argue that subsidy of the arts is a fruitful investment rather than a frivolous expenditure' (2013). Billington highlights how Thatcher’s administration was key in the establishment of an infrastructure for accountability and economisation of the cultural sector in the UK.

John Holden maintains that such 'concentration on instrumental “impacts and outcomes” has produced organisational and systemic distortions' (2004, p 19) within the cultural sector today. The distortions are created in response to the “impact” assessment criteria for funding and the positive bias towards cultural proposals that demonstrate wider socio-economic benefits, i.e., impacts. This in itself might not be surprising, as Bakhshi, Freeman, and Hitchen pragmatically remind us: '[w]hether we like it or not, governments choose between alternative expenditures. They cannot spend the same pound twice on a hospital and an art gallery' (2009, p 17). However, as many critics have observed, the desire for an accountable, socially and economically valuable organisation does not often result in the actualisation (Selwood, 2002; Holden, 2004; O’Brien, 2010). Instead, often a focus on outputs undermines the inherent value of the creative work and leads to a poorer quality result. Cultural institutions dedicate a large amount of time to justify their impact and value rather than concentrating on creating work that might produce it. The necessity to generate such data is an indication of the deep-rooted effects of economic accountability upon the sector. Holden describes the effect of this audit culture within the museum sector, envisioning how

all around the country, cultural organizations—museums, theatres, arts centres and the rest—are holding away days to update their business plans. Library managers are drawing up budgets for their local authority bosses, and voluntary groups are filling in forms, seeking resources to restore historic buildings. They all need money, and they are competing for the attention of those who take decisions within that amorphous beast, the ‘funding system’ (2004, p 13).

Holden argues that a significant portion of institutional attention is diverted towards developing business plans and writing funding proposals as opposed to cultural work. The financial imperative is key to understand how institutions’ 'energies have been directed into chasing funding and collecting evidence rather than achieving cultural purposes. In the search for outcomes and ancillary benefits, the essence of culture has been lost' (Holden, 2004, p 20). The dependence on public funding has led to cultural organisations conforming to the requirements of funding criteria. Focusing on outcome-driven valuation means that 'instead of talking about what they do, they demonstrate how they have contributed to wider policy agendas such as social inclusion, crime prevention and learning' (Holden, 2004, p 13). Quantified metrics to stand in for qualitative benefits in a culture in which where something must be reported at any cost.

The predominance of economic evaluation from a contemporary policymaking perspective is clear in Maria Miller’s (then Secretary for the Department for Media, Sport, and Culture) speech at the British Museum, on 24 April 2013. Her speech focused on deficits and austerity and called for the arts to 'reframe the argument [and] to hammer home the value of culture to our economy' (Miller, 2013). Since the 1980s, value in the cultural sector has repeatedly been imbued with an economic imperative (Garnham, 2005). Writing in Arts and Humanities in Higher Education (February 2015) Belfiore draws attention to

the belief that there exist fundamental economic tests or yardsticks according to which policy decisions can and should be made, and that cost-benefit analysis and cognate econometric methods are the best form of such tests (2015, p 98).

Belfiore draws links between value within the public arts sector and the present changes within higher education. Adopting such a relationship, we might consider, arts administrator John Tusa’s complaint that 'we have lost a vocabulary and an area of permitted public discourse where values are valued rather than costed' (2000, p 29) as a clear precursor to Collini’s lament, above, that 'economistic officialise' (2000, p 19) has overrun the mechanisms of valuation within the humanities department.

The narrative of accountability and mechanisms of justification within the cultural sector bears strong resemblance to the changes within contemporary HEIs in the UK. In focusing on the rhetoric, Robert Hewison protests that 'whoever wrote the documents for the REF, does not appear to have been trained in the humanities' (2011). Speaking at a conference at the University of Sheffield on 5 May 2011, Hewison identifies how the language of impact is at odds with a notion of scholarship that is perceived to be considered and existentially open-ended. He describes how the REF documents are

written in the unlovely technocratic language of generic templates, impact sub-profiles, and submitted units, which turn out to be people like me, who put 30 years [of] work into a single book (2011).

Hewison argues that the impact criterion of the REF marks a shift further along the lines of instrumentalisation of humanities research. He describes how research assessment exercises have 'undervalued long-term effort in favour of short-term gains' (2011). As Small identifies in her definition of humanities research above (see p 3) 'the individual response (its content and its style)' (2013, p 57) is a common methodological approach. Such individualized critiques that contest the hegemonic agendas of impact are not compatible with leading economic instincts and therefore can all too easily be dismissed by policy-makers and university management. Slaughter and Rhoades' concept of 'academic capitalism' (1997) is a useful theoretical tool to understand this difficulty; it allows us to conceptualise the ways in which HEIs are 'shifting from a public good knowledge/learning regime to an academic capitalist knowledge/learning regime' (2004, p 28). Academic capitalism sees the 'institution as marketer' (2004, p 1) and 'knowledge [as] a critical raw material to be mined and extracted […] then sold in the marketplace for profit' (2004, p 4). Within this context, knowledge is made useful in its transfer into the marketplace and immeasurable values are disregarded if they cannot be reified. This reinforces Collini’s assertion that impact is the 'research plus marketing' (2009, p 19). The dichotomy of inside-value and outside-value of research is made explicit in the REF’s impact criterion.

Values that are difficult to calculate in statistical terms are subject to detrimental implications within an economically-minded model of valuation. Somewhat ironically, the result of these implications has, to date, been largely conveyed in financial terms. For instance, significant sums of time and money are required in order to generate accounts of value that the REF requires (Martin, 2011; Hazelkorn, 2013). The 2014 REF proved an expensive and time-consuming process for both HEIs and the governing body HEFCE. Ben R. Martin, Professor of Science, Technology and Policy Studies at the University of Sussex, describes how this problematic narrative of assessment is emerging within the context of contemporary higher education. Martin claims that

as mechanisms fail to capture certain aspects of impact, so additions will be made to the assessment machinery, adding to the costs and the compliance burden, encouraging more ‘game-playing’, introducing more perverse incentives and generating more unintended consequence (2011, p 251).

The RAE is reported to have cost up to £100 million over each cycleFootnote 10. Martin argues that given the increased complexity of the REF 'the costs (both direct and indirect) are likely to be greater now the impact assessment has been added to it' (2011, p 251). It is a reasonable assumption to conclude that the REF costs more than the RAE given that HEFCE is running a peer-review process alongside the additional calculation of impact. The REF press office reported that the 2014 cycle accounted for '52,061 academic staff, 191,150 research outputs [and] 6,975 impact case studies' (HEFCE, 2014). This is an administratively large undertaking for each department submitting research outputs and impact case studies. The Stern Review (“Building on Success and Learning from Experience An Independent Review of the Research Excellence Framework”) confirms Martin’s suspicion that the costs in undertaking the REF, both for universities and for the HEFCE administration are “estimated at £246m for UK HE sector, considerably more than estimates for the 2008 framework which cost around £66 million” (Stern 2016, p 45). Here, the government itself acknowledges that the exercise was “not entirely successful” (p 45). However, this recognition of insufficiency is only surrounding the cost of the exercise and does not acknowledge the concerns of “impact” upon the academy. There is no desire to step back from metric-based evaluation, only to improve the financial management. Martin argues that following the first round of the REF, the revised mechanism for measuring impact will likely become so bureaucratic that it potentially could cost more to run the exercise than the total economic value of the funding it seeks to assess. In World-Literature in the Context of Combined and Uneven Development Deckard et al. describe the REF as being a

top-down, state-imposed scheme, centralised and massively bureaucratic […] seen to have had deleterious effects on the scope, ambition, originality and independence of humanities scholarship, especially among younger scholars (2015, p 2).

The current objection to the integration of business and management into the academic humanities bears many similarities to the context of the integration of economic valuation of the public cultural sector. Philip Schlesinger describes how British academics’ working lives 'have been shaped by the audit culture'. In a closing address of the CREATe All Hands Conference, 15–16 September 2014, he observed that 'our research now has to meet impact criteria that were invented for accountability rather than public intellectuality' (2014). Schlesinger argues that the assessment criteria value increasing bureaucratic accountability over meaningful public engagement. In doing so, it can be seen that from a policymaking perspective, the system rewards that which can be counted, and accounted for.

It is essential to outline the ways in which the mechanisms and measurement of impact have an ideological and political position and how this effects the intrinsic value of humanities research. Arts and Humanities in Higher Education’s special issue 'Forum on the Public Value of Arts and Humanities Research' (2015) explores the interrelation between the humanities research and the rise of impact as an assessment criterion. The issue identifies how the mechanisms that funding bodies have implemented should be seen to be occupying an ideological position. Benneworth writes

in acquiescing to demands from policymakers under pressure for clarity and simplicity, a sense of nuance, ambivalence, and tensions has been lost from these public debates around the public value of arts and humanities research (2015, p 5).

I now develop this concern with further reference to the difficulty in speaking against the value of instrumentality. The example of the creative industries offers further historical specificity for understanding the impact agenda within UK policymaking. In this way 'a sense of nuance' (2015, p 5) might be returned to a valuation of humanities research.

The difficulty in making an intrinsic argument in an age of creative industry

The creative industries first rose to prominence in British policy under New Labour in the late 1990s. The DCMS was established under the leadership of Tony Blair in 1997 and the 'creative industries' were successively encouraged under Gordon Brown, through the late 2000s. The term 'creative industries' derives from policy documents that were instituted in 'new policies for industries associated with the arts, media, design and digital content' (Flew, 2012, p 3). Therefore, from its inception, the term 'creative industries' was an invention of policymakers, and was designed to benefit governance. Although the zenith of the ‘creative industries’ may have waned somewhat since the global financial crisis of 2008, however, the concept remains popular among British policymakers (O’Connor, 2010; Flew, 2012). A 2016 press release from the Department for Culture, Media and Sport (DCMS henceforth) announced that 'the UK’s Creative Industries now contribute a staggering £84 billion a year—almost £10 m an hour—to our economy'. Politicians frequently cite the creative industries as our greatest national export. For example, Ed Vaizey (then acting Minister for Culture, Communications and Creative Industries) described the creative industries as 'British magic dust' which 'gives our country a unique edge' (2011).Footnote 11 Clearly, the creative industries are ideologically, as well as financially, valuable to policymakers.

The 'creative industries' in the UK were first introduced in a series of 'Creative Industries Mapping Documents' published by the DCMS in, 1998. These documents catalogued sectors of creative and cultural production that were of perceived benefit to the British economy. The creative industries were defined as industries that 'have their origin in individual creativity, skill and talent and which have a potential for wealth and job creation through the generation and exploitation of intellectual property' (DCMS, 1998). The creative industries have, therefore, always described culture in terms of its economic potential.Footnote 12 The annual Creative Industries: Focus On report for 2016 details how 'exports of services from the Creative Industries accounted for 9.0% of total exports of services from the UK in 2014' (DCMS, 2016). Throughout the late 1990s and early 2000s, the creative industries were used by the DCMS as a symbol of success for post-industrial Britain.

Anne Boddington, Jos Boys and Catherine Speight observe how this New Labour philosophy 'conceived knowledge as a form of currency that could be shared, distributed and acquired' (2013, p 6). Such framing devalues intrinsic qualities of creative knowledge. In addition, the value of currency makes cultural knowledge dependent on external benefactors who exchange the work for something, rather than permitting academic experts to be to arbiter of what work has value. The value of cultural activity in the eyes of the creative industries is measured by the economic benefits it might produce and as a currency or commodity for exchange. This aligns with the definitions of the aforementioned model of Mode 2 within higher education wherein 'social accountability permeates the whole knowledge production process' in which 'sensitivity to the impact of the research is built in from the start' (Gibbons et al., 1994, p 7). In the creative industries the value of cultural knowledge is enmeshed within an interest in commercial values. John Hartley describes the creative industries as a term that 'combines—but then radically transforms—two older terms: the creative arts and the cultural industries' (2005, p 6, emphasis in original). Hartley’s definition demonstrates the ways in which the creative industries bring 'the arts (i.e., culture) into direct contact with large-scale industries such as media entertainment (i.e., the market)' (2005, p 6). In the era of creative industries, arts and humanities practices can be seen to be 'embroiled in markets in a more diffuse and plural sense, because their intellectual values are inevitably shaped by their social context and application' (Gibbons et al., 1994, p 99). I assert that the developing relationship between industry and cultural knowledge production is a nuanced phenomenon and cannot be understood as an entirely positive nor negative result for cultural value. However, given the dominance of economically-minded valuation mechanisms, producers of creative knowledge must be aware of the pressures that such terms as the “creative industries” imply.

There is a long history of critiquing the commodification of culture in media studies, cultural studies, and the humanities.Footnote 13 Terry Flew argues that the creative industries are 'a kind of “Trojan Horse” through which to smuggle neoliberal discourses into the field' which 'subvert the critical mission of cultural studies and related fields of humanities scholarship' (2012, p 6). Such suspicions are not often acted upon within the academy. A study conducted for the global policy think tank RAND Corporation details: '[a]lthough many advocates of the arts believe intrinsic benefits are of primary importance, they are reluctant to introduce them into the policy discussion because they do not believe such ideas will resonate with most legislators and policymakers' (McCarthy et al., 2004). This reluctance of arts and humanities scholars and practitioners to directly engage in debates concerning the benefits and challenges of the creative industries is well documented. Currently, as opposed to a discussion about alternative values, the arts and culture sector largely conform to funding requirements in offering justifications in the form of economic value (Bérubé, 2002; Parker, 2008; Belfiore, 2015). As with the example of the economisation of arts funding criteria in the 1980s (p 8–15), institutions are reluctant to reject or criticise arguments that promote their societal value, albeit in solely economic terms. Belfiore argues that the language of the creative industries has been widely adopted since

it appears to offer a rhetorically powerful articulation of value and an attendant rationale for funding, critically able to win the approval of a Treasury department set on cutting public expenditure (2015, p 101).

This conformity is problematic and ultimately leads to 'the collapse of value into impact of the sort that lends itself to be expressed in monetary terms' (Belfiore, p 105). The rhetoric of the cultural sector reveals that the popularity of adopting the criteria of impact is 'rooted in the anxiety of justification' (p 105) that has emerged under the present conditions of assessment and valuation within the cultural sector. This is significant to consider in the context of higher education: it demonstrates how public accountability and the economisation of cultural value work in tandem.

Describing cultural work as part of the creative industries, and in turn valuing it economically, can be seen to directly influence the contemporary valuation of the humanities within higher education. Sir Steve Smith (then President of Universities UK) addresses the economic preference of policymakers in an afterword to John Holmwood’s A Manifesto for the Public University. Here Smith explains how 'we [UUK] felt the language of economics was the only language that would secure the prosperity of our universities and higher education institutions' (2011, p 129). That the language of economics is believed to be the most persuasive tool to speak to government is unsurprising given the context outlined above. Smith admits that 'we tailored a narrative that did not start with the universities and what might be good for them, but with the economy, and specifically with the best strategy to ensure future economic growth' (2011, p 131). Further examples of institutional conformity to national economic narratives are widespread within HEIs and other educational bodies across the UK. For instance, the Arts and Humanities Research Council (AHRC) published an economic calculation of the value of their research funding. This was achieved by hiring a multinational professional services company, PricewaterhouseCoopers, who calculated that “for every £1 spent on research by the AHRC, the nation may derive as much as £10 of immediate benefit and another £15–20 of long-term benefit” (AHRC 2009, p 3). Such comments are evidence of the perceived need for the academic humanities to conform to economic models of valuation. Ellen Hazelkorn describes this process as marking a 'shift from valuing intellectual pursuits-for-their-own-sake to measuring research outcomes, impact and relevance' (2015, p 27). The intrinsic value of the humanities is disregarded in favour of a perspective that sees all degrees as means to jobs and all certification as a subset of a national labour market demand.

It is not necessarily damaging that significant number of arts and humanities graduates find employment in the creative industries in the UK. In Creative Industries: Focus On Employment (2015) the DCMS reported that 'one in every six jobs in the UK held by graduates in 2014 was in the Creative Economy' (2015, p 7). However, it is troubling that within public discourse there are very few instances when government officials have given any credibility to the idea of the intrinsic value of the arts and humanities research. Tessa Jowell (then Secretary of State for Culture, Media and Sports) provides the closest iteration, conceding that

too often politicians have been forced to debate culture in terms only of its instrumental benefits to other agendas […] in this country we have avoided the more difficult approach of investigating, questioning and celebrating what culture actually does in and of itself (2004, p 8).

Often such arguments against the utility of research are derided as elitist. There is a danger of conflating a defence of 'the best that is known and thought in the world' (Arnold, 1869, p viii) with what Michael Bérubé caricatures as 'an old-school, brandy-sniffing, meerschaum-chomping elitist snob' (2002, p 8). Holden testifies how in taking a hard-line in intrinsic defences of the value of the culture, critics appear 'to assert the value of their own judgement above that of others' (2004, p 24). Such accusations of snobbery inherent in the defence of the innate value of the arts and humanities should not be idly dismissed. One must not forget that to understand and enjoy museum objects, art and other cultural objects, a person has likely been afforded some degree of privilege (Bourdieu et al., 1991).

With these considerations in mind, the concluding section of this article will seek to move beyond such stale divisions caricatured as the 'floppy bow ties vs. hard-headed realists' (O'Brien, p 25). Whilst the division between intrinsic and instrumental value continues to form deep intellectual trenches between policymakers and practitioners, today there is less of a separation than might be observed from this binaristic perspective. A report commissioned by the National Museum Director’s Council frames the limitations of thinking 'intrinsic vs. instrumental' most explicitly: 'it is no good trying to relate all the value of arts and culture to monetary valuations' and yet it is 'equally unhelpful to try to justify the arts as some kind of special case, different from all other spending priorities and subject to unique criteria' (Selwood, 2010, p 5). Such observations capture the complex negotiation of cultural value in contemporary public sphere. Cultural institutions are caught in a difficult position, wherein they are forced to participate in the economic game playing inherent in the funding models and policy demands, even if they understand these metrics to misrepresent their work. This in part is due to a difficulty that Bakhshi, Freeman and Hitchen highlight: 'there is a contradiction between the plea that the intrinsic value of art should be accounted for, and the idea that it is beyond accounting' (2009, p 15). The increase in accountability metrics created an increased demand upon public cultural organizations to provide economic evidence in order to receive further funding. However, to uphold that there is value in the arts and humanities beyond economic valuation need not be an inherently elitist statement. Work within SSH evidences that humanities scholars themselves recognise alternative values, beyond the economic and commerical, in the work that they doFootnote 14. For example 'being a courageous risk-taker with authentic intellecutal interests' (Guetzkow et al., 2004, p 206) is highlighted in a study of originality within the humanities and social sciences. Sven Hug, Michael Ochsner and Hans-Dieter Daniel conducted the first empirical study for assessing research quality in the humanities entirely from within the humanities in 2013. This article calls for humanities scholars to build upon such evidence, in providing an alternative approach that engages with policymaking as opposed to avoiding it.

Frederic Jameson described the current landscape of higher education as the 'subsumption of whole fields and disciplines under the patronage of private business and, as it were, the assimilation to wage labour of the standard nonacademic type of researchers whose work is subsidized by monopolies who set the agenda and are likely to profit from the results' (2008, p 571). Within this, humanities scholars find themselves in a feedback loop in which their lack of articulation of value paints them as ineffective or elitist. Public debate is saturated with instrumental arguments. Bérubé contends that there is 'something awry with the idea that such activities [arts and humanities research] need to be justified with regard to their social or economic benefits to colleges or communities' (2002, p 34–35). Here, Bérubé summarizes the changing nature of valuation within the academy in the US, but that context increasingly applies to a contemporary British academia. What has thus far been left unexplored is the potential of rejecting the valuation of cultural work in solely instrumental terms. It is essential for humanities scholars to recognise the to

steer a course between the Scylla of mechanized procedures that depend on proxy indictors (e.g., citations or the amount of grant money previously awarded) and the Charybdis of appeals to ineffable connoisseurship (e.g., I-know-it-when-I-see-it pronouncements) (VolkswagenStiftung, 2014, p 1).

This dilemma highlights the need for further considerations of the intellectual qualities of the humanities from within the discipline. Existing literature from SSH has predominantly drawn on social science methodologies (Hemlin (1993); VolkswagenStiftung, 2014; Hug et al., 2013). This article concludes by proposing that as the value of the humanities comes under increasing public scrutiny, scholars will be required to use their disciplinary tools to best demonstrate the value of the work that they do. Or else, as the above narrative of the public cultural sector forewarns us, these values may cease to be understood in public discourse.

Alternative assessment criteria in valuations of the humanities

This article has demonstrated the difficulty of presenting unquantifiable cultural values in a context dominated by economic imperatives. This final section proposes a potential route for future research in this area. Within higher education, artists and humanists 'tend to regard self-justification as a dubious enterprise best left to the writers of admissions brochures and back-patting liberal-arts mission statements' (Bérubé, 2002, p 25). Bérubé likens the writers of brochures (who doubtless are themselves, in fact, humanities graduates employed within the creative industries) to sheep.Footnote 15 He imitates: 'the Humanities teach us what it is to be human, the Arts deepen our spirit, the Humanities preserve our common cultural heritage, bleat, bleat, bleat' (2002, p 25). The impersonation exposes the difficulty of avoiding clichés when publically communicating the value of humanities scholarship. The individual testimonies of Hewison and Schlesinger above (see p 12, 15, respectively) demonstrate how humanities scholars are resistant to engage in processes of marketing, to speak 'economistic officialise' (Collini, 2009, p 19), and this is a role quickly and happily assumed by those trained in business. Humanities scholars must ask how might they retain ownership of this discourse since these articulations of value have long-term implications for their disciplines.

A strong critical starting point in an application of non-instrumental values to political debate is found in Oliver Bennett’s 'Beyond Machinery: The Cultural Policies of Arnold, 1869' (2005). Bennett argues that, despite being alive over a century ago, Matthew Arnold can be of use to contemporary debates concerning cultural policymaking.Footnote 16 Arnold unashamedly sought to understand the spirit of humanity as opposed to “an outward set of circumstances” (1869, p 48). In direct application to contemporary politics, Bennett admits that much of Arnoldian cultural theory is flawed. Specifically, there is a problematic deference to great men of culture and a suspicion of freedom of thought for anyone else. As above, it is important that the arts and humanities remain aware of the accusations of elitism in an intrinsic defence. However, Bennett puts forward the idea that 'it is Arnold’s critical disposition, rather than any specific judgements that he makes, which is important' (2005, 479). Bennett’s work reminds us, in re-reading Arnold, that the methods by which the arts and humanities engage in debates concerning their value matter. Therefore, Bennett encourages an Arnoldian disposition of a 'constant and public interrogation of what is actually constituted as the best' (2005, p 479). This might be a useful methodological tool through which a socially and historically informed assessment of mechanisms for attributing value might be further developed. In our present moment, it is worth considering how 'the search for cultural value would become itself the driving force of cultural policy' (2005, p 480) as Arnold also argued in Culture and Anarchy. The broader history of impact within the cultural sector that I have outlined above has demonstrated the expansive expertise and enduring potential of critical conversations. I propose this knowledge might be better applied and put to use in the present 'impact' agenda facing the academic humanities.

The responsibility for the resolution of this dilemma appears to be two-fold. True, ultimately policymakers are required to pay attention to alternative values, but firstly, it requires humanities scholars to address these debates with their own analytical discourse. Arnold rejected a system of valuation, which permitted “every opinion, no matter how eccentric or ill-grounded, to pass itself off as the equal of any other” (Collini, 1994 p 59). Collini’s approach to understanding our present moment through a reconnection and re-reading of the past offers an effective means with which to engage with the future valuation of the humanities in public discourse. Writing in the introduction to Culture and Anarchy published in 1932, J. D. Wilson observes 'of all the nineteenth-century prophets who pronounced the condition of England, Matthew Arnold knew his England best' (1932, P xiv–xv). Similarly, in our contemporary context working within HEIs, there is no one who can better articulate the value of the humanities than those working within it. If what matters is not measured, and therefore not being counted in the current metrics of value, it might cease to exist at all in public discourse. Dave O’Brien summarizes that although 'measurement is important to policymaking' (2015, p 91) it is equally significant to observe that measurement itself should be 'examined by critically engaging with the historical trends and managerial ideologies that have brought these methods to prominence' (2015, p 91). Such examination is the work for the humanities, to historicize, to read cultural policy, to interrogate it. This article is an example of such an approach, that is urgently required in order to better understand and articulate the creation of cultural value and the mechanisms that drive its assessment.

The UK is currently experiencing a challenge to the ideals of higher education and how it should be evidenced. The processes of marketisation and economic determination are seen to dominate the governance of higher education. However, as Schlesinger argues 'we should take our distance from the underlying conditions that shape our research agendas' (2014). It is important to work towards a positive valuation of the humanities within this context as opposed to a reactionary one. It is the responsibility of humanities scholars to ensure that alternative values and accounts are pursued. Martha Nussbaum reminds us

a catalogue of facts, without the ability to assess them, or to understand how a narrative is assembled from evidence, is almost as bad as ignorance, since the pupil will not be able to distinguish ignorant stereotypes purveyed by politicians and cultural leaders from the truth, or bogus claims from valid ones. World history and economic understanding, then, must be humanistic and critical if they are to be at all useful in forming intelligent global citizens (2010, p 94).

It matters that we respond to the demand for scholarly impact by supplying valuable research. Those working in the humanities should be accountable. Accountable to ourselves, to society and to the parts of society that are not able to speak up for themselves. With this in mind, future directions of the humanities research assessment must act in reframing Thatcher’s indicative 'there is no alternative' (to the forces of economic-thinking) into the interrogative 'is there no alternative?' Therefore, humanities scholars can create pathways to impact that do not lie solely in the submission of units to the REF, but in a far wider reconsideration of value of in an era of marketisation.

Data availability

Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.

Additional information

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Change history: A correction to this article has been published and is linked from the HTML version of this article.

Change history

  • Correction 27 February 2018

    On page 7 of the PDF under the subheading “Alternative assessment criteria in valuations of the humanities” in the second paragraph there was a quotation incorrectly attributed to Matthew Arnold (1869) which has now been corrected to attribute Oliver Bennett (2005).

Notes

  1. 1.

    For commentary see Lears (2015); Anderson (2010). For conferences, see Hewison (2011); Schlesinger (2014)

  2. 2.

    See Arts and Humanities in Higher Education, 2015, Vol.14 (1) for special issue on impact.

  3. 3.

    See Oancea and Furlong (2007); Bakioglu and Kurnaz (2009)

  4. 4.

    For Europe see Spaapen et al. (2007); Hazelkorn (2013); Looseley (2011). For Australia see Donovan (2008); Gascoigne and Metcalfe (2005); Sinnerbrink (2013).

  5. 5.

    Full title: Securing a Sustainable Future for Higher Education: an Independent Review of Higher Education Funding & Student Finance

  6. 6.

    The nature of the initial request of Lord Mandelson, to examine the contributors to the costs of education pre-empted the heavily economic focus of this report.

  7. 7.

    Which replaces the Research Assessment Exercise (RAE) first introduced under Margaret Thatcher’s Conservative administration in 1986. Impact is present in higher education rhetoric for several years preceding this formal categorisation in the UK framework. See Donovan (2007); Spaapen et al. (2007).

  8. 8.

    As some readers will recall the focus of the RAE assessment was focused solely on the academic excellence of the research that institutions had produced.

  9. 9.

    The UK is currently experiencing cuts to public funding in the arts which are worse than under Thatcher. See Cartwright (2011).

  10. 10.

    See Sastry and Bahram (2006)

  11. 11.

    YouTube interview as part of a week-long digital event to raise awareness for the launch of the CREATE UK strategy. See http://www.thecreativeindustries.co.uk/resources/strategy

  12. 12.

    Each of the 13 sectors are described through the following lenses of analysis: Industry Revenues, UK Market Size, Balance of Trade, Employment, Industry Structure, International Critical Acclaim, Secondary Economic Impact, Potential for Growth and Growing the Sector—Issues for Consideration. The list demonstrates the economic categorisation that shapes the definition of the 'creative industries'.

  13. 13.

    For cultural studies see McRobbie (1996); Harney (2010). For critical humanities response see Brown (2003); Bourdieu (1998).

  14. 14.

    See Hemlin (1993); Guetzkow et al. (2004); Ochsner et al. (2017).

  15. 15.

    Incidentally, this ovine metaphor is also utilised in William Deresiewicz’s Excellent Sheep: The Miseducation of the American Elite (2014).

  16. 16.

    Which follows assertions made by Walters, T. in 'The Question of Culture (and Anarchy)' (1997) MLN 112 pp. 349-65

References

  1. AHRC. (2009) Leading the World: the Economic Impact of U.K. Arts and Humanities Research. Arts and Humanities Research Council. http://www.ahrc.ac.uk/documents/publications/leading-the-world/. Accessed 12 Nov 2016

  2. Anderson R (2010) The ‘Idea of a University’ Today History & Policy. http://www.historyandpolicy.org/policy-papers/papers/the-idea-of-a-university-today. Accessed 20 Jun 2017

  3. Arnold M (1869) Culture and Anarchy. Smith, Elder & Co, London

  4. Bakhshi H, Freeman A, Hitchen G (2009) Measuring Intrinsic Value How to stop worrying and love economics. http://www.missionmodelsmoney.org.uk/sites/default/files/23974660-Measuring-Intrinsic-Value-Hasan-Bakhshi-Alan-Freeman-Graham-Hitchen-2009_0_0.pdf. Accessed 30 Aug 2016

  5. Bakioglu A (2009) Quality criteria of research perceived by academics in social sciences at higher education. Online Submiss 6(3):1–13

  6. Belfiore E (2015) ‘Impact’, ‘value’ and ‘bad economics’: Making sense of the problem of value in the arts and humanities. Art Humanit High Educ 14(1):95–110

  7. Belfiore E (2003) ‘Economic Impact—Inconclusive evidence’. Arts Professional. http://www.artsprofessional.co.uk/magazine/article/economic-impact-inconclusive-evidence. Accessed 12 Apr 2017

  8. Belfiore E, Bennett O (2008) The Social Impact of the Arts: An Intellectual History. Palgrave Macmillan, London

  9. Bennett O (2005) Beyond machinery: the cultural policies of matthew arnold. Hist Polit Econ 37(3):455–482

  10. Benneworth P (2015) Putting impact into context: the Janus face of the public value of arts and humanities research. Art Humanit High Educ 14(1):3–8

  11. Bérubé M (2002) The utility of the arts and humanities. Art Humanit High Educ 2(1):23–40

  12. Billington M (2013) Margaret Thatcher casts a long shadow over theatre and the arts. https://www.theguardian.com/stage/2013/apr/08/margaret-thatcher-long-shadow-theatre. Accessed 10 Jun 2016

  13. Boddington A, Boys J, Speight C (2013) Museums and Higher Education Working Together: Challenges and Opportunities. Ashgate, Farnham

  14. Bourdieu P (1998) A reasoned utopia and economic fatalism. New Left Rev 227:125–30. January—February

  15. Bourdieu P, Darbel A, Schnapper D (1991) The Love of Art: European Art Museums and Their Public. (Beattie C, Merriman N, Trans.). Stanford University Press, Palo Alto

  16. Brown R, Carasso H (2013) Everything for Sale? The Marketisation of UK Higher Education. Routledge/Society for Research into Higher Education, London

  17. Brown W (2003) Neo-liberalism and the end of liberal democracy. Theory and Event 7 (1). Web.

  18. Browne J (2012) An Independent Review of Higher Education Funding and Student Finance in England. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/31999/10-1208-securing-sustainable-higher-education-browne-report.pdf. Accessed 28 Dec 2011

  19. Cartwright A (2011) 'Arts and the Cuts Part I: Arts on the frontline' Counterfire. http://www.counterfire.org/arts-and-the-cuts/10138-arts-and-the-cuts-part-i-arts-on-the-frontline. Accessed 3 Jun 2017

  20. Collini S (1994) Matthew Arnold: A Critical Portrait. Clarendon Press, Oxford

  21. Collini S (2009) Impact on humanities: researchers must take a stand now or be judged and rewarded as salesmen. Times Literary Supplement, p 18–19

  22. Creative Industries Council. (2016) Creative jobs and exports outpace rest of UK economy. http://www.thecreativeindustries.co.uk/uk-creative-overview/news-and-views/news-creative-jobs-and-exports-outgrow-rest-of-economy#. Accessed 20 Jul 2016

  23. DCMS. (1998) Creative Industries Mapping Documents 1998. https://www.gov.uk/government/publications/creative-industries-mapping-documents-1998. Accessed 30 Jul 2016

  24. DCMS. (2016) Creative Industries: 2016 Focus on. https://www.gov.uk/government/statistics/creative-industries-2016-focus-on. Accessed 3 Aug 2016

  25. Deckard S (2015) World-Literature in the Context of Combined and Uneven Development. Liverpool University Press, Liverpool

  26. Deresiewicz W (2014) Excellent Sheep: The Miseducation of the American Elite. Free Press, New York, NY

  27. Donovan C (2007) Introduction: future pathways for science policy and research assessment: metrics vs. peer review, quality vs. impact. Sci Public Policy 34(8):538–542

  28. Donovan C (2008) The Australian research quality framework: a live experiment in capturing the social, economic, environmental and cultural returns of publicly funded research. New Directions for Evaluation, p 47–60

  29. Flew T (2012) The creative industries: culture and policy. Sage, London

  30. Gagnier R (2013) Operationalizing hope: the neoliberalization of British universities in historico-philosophical perspective. Occassion: Interdisciplinary Studies in the Humanities, p 1–16

  31. Garnham N (2005) From cultural to creative industries. Int J Cult Policy 11(1):15–29

  32. Gascoigne T, Metcalfe J (2005) Commercialisation of Research Activities in the Humanities, Arts and Social Sciences in Australia. CHASS Occasional Papers N 1. Council for Humanities, Arts and Social Sciences, Australia

  33. Gibbons M, Limoges C, Nowotny H, Schwartzman S, Scott P, Trow M (1994) The new production of knowledge: The dynamics of science and research in contemporary societies. Sage, Thousand Oaks

  34. Guetzkow J, Lamont M, Mallard G (2004) What is originality in the humanities and the social sciences? Am Sociol Rev 69(2):190–212

  35. Halsey AH (1997) Trends in access and equity in higher education: Britain in international perspective. In: Halsey AH, Lauder H, Brown P, Stuart Wells A (eds) Education, Culture, Economy and Society. Oxford University Press, Oxford, p 638–645

  36. Harney S (2010) Creative industries debate – unfinished business: labour, management, and the creative industries. Cult Stud 24(3):431–444

  37. Hartley J (2005) Creative Industries. Blackwell, Oxford

  38. Hazelkorn, E (2013) Europe Looks for Better Ways to Measure the Value of the Arts and Humanities The Chronicle of Higher Education, 10 January 2013. http://www.chronicle.com/blogs/worldwise/europe-looks-for-better-ways-to-measure-the-value-of-the-arts-and-humanities/31413. Accessed 3 Aug 2016

  39. Hazelkorn E (2015) Making an impact: new directions for arts and humanities. Art Humanit High Educ 14(1):25–44

  40. HEFCE. (2011) Assessment framework and guidance on submissions. http://www.ref.ac.uk/media/ref/content/pub/assessmentframeworkandguidanceonsubmissions/GOS%20including%20addendum.pdf. Accessed 12 Nov 2016

  41. HEFCE (2014) REF: Key Facts. http://www.ref.ac.uk/media/ref/content/pub/REF%20Brief%20Guide%202014.pdf. Accessed 28 May 2016

  42. Hemlin S (1993) Scientific quality in the eyes of the scientist. A questionnaire study. Scientometrics 27(1):3–18

  43. Hewison R (2011) Opening Address [Audio podcast]. The Value of the Humanities in the 21 st Century, University of Sheffield. https://www.sheffield.ac.uk/faculty/arts-and-humanities/value-of-arts. Accessed 28 Jun 2016

  44. Holden J (2004) Capturing Cultural Value. DEMOS, London

  45. Hug SE, Ochsner M, Daniel H-D (2013) Criteria for assessing research quality in the humanities: a Delphi study among scholars of English literature, German literature and art history. Res Eval 22(5):369–383

  46. Jameson F (2008) How not to historicize theory. Crit Inq 34(3):563–582

  47. Jowell T (2004) Government and the Value of Culture. Department for Culture, Media and Sport: http://www.culture.gov.uk/images/publications/valueofculture.pdf

  48. Lears J (2015) Liberal Arts vs. Neoliberalism. Commonweal: https://www.commonwealmagazine.org/liberal-arts-vs.-neoliberalism. Accessed 12 Jun 2016

  49. Looseley D (2011) Making an ‘impact’: some personal reflections on the Humanities in the UK. Art Humanit High Educ 10(1):9–18

  50. Martin BR (2011) The research excellence framework and the ‘impact agenda’: are we creating a Frankenstein monster? Res Eval 20(3):247–254

  51. McCarthy KF, Ondaatje EH, Zakaras L, Brooks A (2004) Gifts of the Muse: Reframing the Debate about the Benefits of the Arts. http://www.rand.org/pubs/monographs/2005/RAND_MG218.pdf. Accessed 10 Jan 2016

  52. McGettigan A (2013) The Great University Gamble. Pluto Press, London

  53. McRobbie A (1996) All the world’s a stage, screen or magazine: when culture is the logic of late capitalism. Media Cult Soc 18(2):335–342

  54. Miller M (2013) Testing times: Fighting culture’s corner in an age of austerity. https://www.gov.uk/government/speeches/testing-times-fightingcultures-corner-in-an-age-of-austerity. Accessed 13 Oct 2016

  55. Molesworth M, Scullion R, Nixon E (2010) The marketisation of higher education and student as consumer. Routledge, London

  56. Myerscough J (1988) The Economic Importance of the Arts in Britain. Policy Studies Institute, London

  57. Nussbaum M (2010) Not for Profit: Why Democracy Needs the Humanities. Princeton University Press, London

  58. Oancea A, Furlong J (2007) Expressions of excellence and the assessment of applied and practice-based research. Res Pap Educ 22(2):119–137

  59. O’Brien D (2015) Cultural value, measurement and policy making. Art Humanit High Educ 14(1):79–94

  60. O’Brien D (2010) Measuring the Value of Culture. Department for Culture, Media and Sport, London, AHRC/ESRC

  61. O’Connor J (2010) The cultural and creative industries: a literature review. Arts Council England: Creativity, Culture and Education (CCE) Literature Reviews

  62. Ochsner M, Hug SE, Daniel H-D (2012) Indicators for research quality in the humanities: opportunities and limitations. Bibliometrie—Praxis Und Forschung 1(0):4-1–4-17

  63. Ochsner M, Hug SE, Galleron I (2017) The future of research assessment in the humanities: bottom-up assessment procedures. Palgrave Commun 3:17020. Accessed at http://doi.org/10.1057/palcomms.2017.20

  64. Parker J (2008) ‘What have the humanities to offer 21st-century Europe?’ Reflections of a note taker. Art Humanit High Educ 7(1):83–96

  65. Sandel M (2013) What money can’t buy: the moral limits of markets. Penguin, London

  66. Sastry T, Bahram B (2006) Using Metrics to Allocate Research Funds. HEPI, Oxford

  67. Selwood S (2002) The politics of data collection. Cult Trend 12(47):13–87. 1

  68. Selwood S (2010) Making a difference: The cultural impact of museums London. NMDC. http://www.nationalmuseums.org.uk/media/documents/publications/cultural_impact_final.pdf

  69. Schlesinger P (2014) CREATe Plenary “The Public Intellectual” All Hands Conference, Glasgow. Retrieved January 2017: http://www.create.ac.uk/blog/2014/07/02/create-all-hands-conference-glasgow-september-15-16th-2014/. Accessed Jan 2017

  70. Sinnerbrink R (2013) Philosophy’s contribution to society. The Australian, 7 September. http://www.theaustralian.com.au/higher-education/opinion/philosophyscontribution-to-society/story-e6frgcko-1226713506686#sthash.P5ygBf78.dpuf. Accessed 3 Nov 2016

  71. Slaughter S, Rhoades G (2004) Academic Capitalism and the New Economy: Markets, State and Higher Education. John Hopkins University Press, Baltimore

  72. Small H (2013) The Value of the Humanities. Oxford University Press, Oxford

  73. Smith S (2011) Afterword. A positive future for higher education in England. In: Holmwood, J A Manifesto for the Public University. Bloomsbury Academic, London, p 127–142

  74. Stern N (2016) Building on success and learning from experience: an independent review of the Research Excellence Framework. Department for Business, Energy and Industrial Strategy. HMSO, London

  75. Spaapen J, Dijstelbloem H, Warmelink F (2007) Evaluating research in context: A method for comprehensive assessment, 2nd edition. Consultative Committee of Sector Councils for Research and Development, The Hague

  76. Tusa J (2000) Art Matters: Reflecting on Culture. Methuen, London

  77. Vaizey E (2011) Interview on the Creative Industries [YouTube Video] Department for Culture, Media and Sport. https://youtu.be/-znvEy4Ts1c. Accessed 10 Jun 2017

  78. VolkswagenStiftung. (2014) What is intellectual quality in the humanities? volkswagenstiftung.de. Hannover

  79. Walters T (1997) The Question of Culture (and Anarchy) MLN 112 p 349–365

  80. Wilson JD (1932) 1981 Introduction to Arnold Culture and Anarchy [1869] 1981. Cambridge University Press, Cambridge

Download references

Author information

Affiliations

  1. School of English, University of Exeter, Exeter, UK

    • Zoe Bulaitis

Authors

  1. Search for Zoe Bulaitis in:

Competing interests

The author declares that there are no competing financial interests.

Corresponding author

Correspondence to Zoe Bulaitis.

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1057/s41599-017-0002-7