Introduction

The introduction of ‘research impact’ as an element of evaluation constitutes a major change in the construction of research evaluation systems. ‘Impact’, understood broadly as the influence of academic research beyond the academic sphere, including areas such as business, education, public health, policy, public debate, culture etc., has been progressively implemented in various systems of science evaluation—a trend observable worldwide (Donovan, 2011; Grant et al., 2009; European Science Foundation, 2012). Salient examples of attempts to systematically evaluate research impact include the Australian Research Quality Framework–RQF (Donovan, 2008) and the Dutch Standard Evaluation Protocol (VSNU–Association of Universities in the Netherlands, 2016, see ‘societal relevance’).

The most articulated system of impact evaluation to date was implemented in the British cyclical ex post assessment of academic units, Research Excellence Framework (REF), as part of a broader governmental policy—the Impact Agenda. REF is the most-studied and probably the most influential impact evaluation system to date. It has been used as a model for analogous evaluations in other countries. These include the Norwegian Humeval exercise for the humanities (Research Council of Norway, 2017, pp. 36–37, Wróblewska, 2019) and ensuing evaluations of other fields (Research Council of Norway, 2018, pp. 32–34; Wróblewska, 2019, pp. 12–16). REF has also directly inspired impact evaluation protocols in Hong-Kong (Hong Kong University Grants Committee, 2018) and Poland (Wróblewska, 2017). This study is based on data collected in the context of the British REF2014 but it advances a description of the ‘discourse of impact’ that can be generalized and applied to other national and international contexts.

Although impact evaluation is a new practice, a body of literature has been produced on the topic. This includes policy documents on the first edition of REF in 2014 (HEFCE, 2015; Stern, 2016) and related reports, be it commissioned (King’s College London and Digital Science, 2015; Manville et al., 2014, 2015) or conducted independently (National co-ordinating center for public engagement, 2014). There also exists a scholarly literature which reflects on the theoretical underpinnings of impact evaluations (Gunn and Mintrom, 2016, 2018; Watermeyer, 2012, 2016) and the observable consequences of the exercise for academic practice (Chubb and Watermeyer, 2017; Chubb et al., 2016; Watermeyer, 2014). While these reports and studies mainly draw on the methods of philosophy, sociology and management, many of them also allude to changes related to language.

Several publications on impact drew attention to the process of meaning-making around the notion of ‘impact’ in the early stages of its existence. Manville et al. flagged up the necessity for the policy-maker to facilitate the development of common vocabulary to enable a broader ‘cultural shift’ (2015, pp. 16, 26. 37–38, 69). Power wrote of an emerging ‘performance discourse of impact’ (2015, p. 44) while Derrick (2018) looked at the collective process of defining and delimiting “the ambiguous object” of impact at the stage of panel proceedings. The present paper picks up these observations bringing them together in a unique discursive perspective.

Drawing from linguistic pragmatics and Foucauldian discourse analysis, the paper presents shifts related to the introduction of ‘impact’ as element of evaluation in four stages. These are, in chronological order: (1) the ‘problematisation’ of the notion of ‘impact’ in policy and its appropriation on a local level, (2) the creation of an impact infrastructure to orchestrate practices around impact, (3) the consolidation of a new genre of writing—impact case study, (4) academics’ uptake of the notion of impact and its progressive inclusion in their professional positioning.

Each of these stages is described using theoretical concepts grounded in empirical data. The first stage has to do with the process of ‘problematization’ of a previously non-regulated area, i.e., the process of casting research impact as a ‘problem’ to be addressed and regulated by a set of policy measures. The second stage took place when in rapid response to government policy, new procedures and practices were created within universities, giving rise to an impact ‘infrastructure’ (or ‘apparatus’ in the Foucauldian sense). The third stage is the emergence of a crucial element of the infrastructure—a new genre of academic writing—impact case study. I argue that engaging with the new genre and learning to write impact case studies was key in incorporating ‘impact’ into scholars’ narratives of ‘academic identity’. Hence, the paper presents new practices of ‘subjectivation’ as the fourth stage of incorporation of ‘impact’ into academic discourse. The four stages of the introduction of ‘impact’ into academic discourse are mutually interlinked—each step paves the way for the next.

Of the described four stages, only stage three focuses a classical linguistic task: the description of a new genre of text. The remaining three take a broader view informed by sociology and philosophy, focusing on discursive practices i.e., language used in social context. Other descriptions of the emergence of impact are possible—note for instance Power’s four-fold structure (Power, 2015), at points analogous to this study.

Theoretical framework and data

This study builds on a constructivist approach to social phenomena in assuming that language plays a crucial role in establishing and maintaining social practice. In this approach ‘discourse’ is understood as the production of social meaning—or the negotiation of social, political or cultural order—through the means of text and talk (Fairclough, 1989, 1992; Fairclough et al., 1997; Gee, 2015).

Linguistic pragmatics and Foucauldian approaches to discourse are used to account for the changes related to the rise of ‘impact’ as element of evaluation and discourse on the macro and micro scale. In looking at the micro scale of every-day linguistic practices the analysis makes use of linguistic pragmatics, in particular concepts of positioning (Davies and Harré, 1990), stage (Goffman, 1969; Robinson, 2013), metaphor (Cameron, et al., 2009; Musolff, 2004, 2012), as well as genre analysis (Swales, 1990, 2011). Analyzing the macro scale, i.e., the establishment of the concept of ‘impact’ in policy and the creation of an impact infrastructure, it draws on selected concepts of Fouculadian governmentality theory (crucially ‘problematisation’, ‘apparatus’, ‘subjectivation’) (Foucault, 1980, 1988, 1990; Rose, 1999, pp. ix–xiii).

While the toolbox of linguistic pragmatics is particularly useful in analyzing linguistic aspects of the datasets, Foucault’s governmental framework helps bring together findings from the two datasets in a broader analysis, allowing more general conclusions on the practices of governing (academic) subjects within evaluation frameworks. Both pragmatic and Foucauldian traditions of discourse analysis have been productively applied in the study of higher education contexts (e.g., Fairclough, 1993, Gilbert and Mulkey, 1984, Hyland, 2009, Myers, 1985, 1989; for an overview see Wróblewska and Angermuller, 2017).

The analysis builds on an admittedly heterogenous set of concepts, hailing from different traditions and disciplines. This approach allows for a suitably nuanced description of a broad phenomenon—the discourse of impact—studied here on the basis of two different datasets. To facilitate following the argument, individual theoretical and methodological concepts are defined where they are applied in the analysis.

Data

The studied corpus consists of two datasets: a written and oral one. The written corpus includes 78 impact case studies (CSs) submitted to REF2014 in the discipline of linguisticsFootnote 1. Linguistics was selected as a discipline straddling the social sciences and humanities (SSH). SSH are arguably most challenged by the practice of impact evaluation as they have traditionally resisted subjection to economization and social accountability (Benneworth et al., 2016; Bulaitis, 2017).

The CSs were downloaded in pdf form from REF’s website: https://www.ref.ac.uk/2014/. The documents have an identical structure, featuring basic information: name of institution, unit of assessment, title of CS and core content divided into five sections: (1) summary of impact, (2) underpinning research, (3) references to the research, (4) details of impact (5) sources to corroborate impact. Each CS is about 4 pages long (~2400 words). The written dataset (with a word-count of 173,474) was analyzed qualitatively using MAX QDA software with a focus on the generic aspect of the documents.

The oral dataset is composed of semi-structured interviews with authors of the studied CSs (n = 20) and other actors involved in the evaluation, including two policy-makers and three academic administratorsFootnote 2. In total, the 25 interviews, each around 60 min long, add up to around 25 h of recordings. The interviews were analyzed in two ways. Firstly, they were coded for themes and topics related to the evaluation process—this was useful for the description of impact infrastructure presented in step 2 of analysis. Secondly, they were considered as a linguistic performance and coded for discursive devices (irony, distancing, metaphor etc.)—this was the basis for findings related to the presentation of one’s ‘academic self’ which are the object of fourth step of analysis. The written corpus allows for an analysis of the functioning of the notion of ‘impact’ in the official, administrative discourse of academia, looking at the emergence of an impact infrastructure and the genre created for the description of impact. The oral dataset in turn sheds light on how academics relate to the notion of impact in informal settings, by focusing on metaphors and pragmatic markers of stage.

The discourse of impact

Problematization of impact

The introduction of ‘impact’, a new element of evaluation accounting for 20% of the final result, was seen as a surprise and as a significant change in respect to the previous model of evaluation—the Research Assessment Exercise (Warner, 2015). The outline of an approach to impact evaluation in REF was developed on the government’s recommendation after a review of international practice in impact assessment (Grant et al., 2009). The adopted approach was inspired by the previously-created (but never implemented) Australian RQF framework (Donovan, 2008). A pilot evaluation exercise run in 2010 confirmed the viability of the case-study approach to impact evaluation. In July 2011 the Higher Education Council for England (HEFCE) published guidelines regulating the new assessment (HEFCE, 2011). The deadline for submissions was set for November 2013.

In the period between July 2011 and November 2013 HEFCE engaged in broad communication and training activities across universities, with the aim of explaining the concept of ‘impact’ and the rules which would govern its evaluation (Power, 2015, pp. 43–48). Knowledge on the new element of evaluation was articulated and passed down to particular departments, academic administrative staff and individual researchers in a trickle-down process, as explained by a HEFCE policymaker in an account of the run-up to REF2014:

There was no master blue print! There were some ideas, which indeed largely came to pass. But in order to understand where we [HEFCE] might be doing things that were unhelpful and might have adverse outcomes, we had to listen. I was in way over one hundred meetings and talked to thousands of people! (…) [The Impact Agenda] is something that we are doing to universities. Actually, what we wanted to say is: ‘we are doing it with you, you’veFootnote 3 got to own it’.

Int20, policymaker, example 1Footnote 4

Due to the importance attributed to the exercise by managers of academic units and the relatively short time for preparing submissions, institutions were responsive to the policy developments. In fact, they actively contributed to the establishment and refinement of concepts related to impact. Institutional learning occurred to a large degree contemporarily to the consolidation of the policy and the refinement of the concepts and definitions related to impact. The initially open, undefined nature of ‘impact’ (“there was no master blue-print”) is described also in accounts of academics who participated in the many rounds of meetings and consultations. See example 2 below:

At that time, they [HEFCE] had not yet come up with this definition [of impact], not yet pinned it down, but they were trying to give an idea of what it was, to get feedback, to get a grip on it. (…) And we realised (…) they didn’t have any more of an idea of this than we did! It was almost like a fishing expedition. (…) I got a sense very early on of, you know, groping.

Int1, academic, example 2

The “pinning down” of an initially fuzzy concept and defining the rules which would come to govern its evaluation was just one aim of the process. The other one was to engage academics and affirm their active role in the policy-making. From an idea which came from outside of the British academic community (from the the government, the research councils) and originally from outside the UK (the Australian RQF exercise), a concept which was imposed on academics (“it is something that we are doing to universities”) the Impact Agenda was to become an accepted, embedded element of the academic life (“you’ve got to own it”). In this sense, the laboriousness of the process, both for the policy-makers and the academics involved, was a necessary price to be paid for the feeling of “ownership” among the academic community. Attitudes of academics, initially quite negative (Chubb et al., 2016, Watermeyer, 2016), changed progressively, as the concept of impact became familiarized and adapted to the pre-existing realities of academic life, as recounted by many of the interviewees, e.g.,:

I think the resentment died down relatively quickly. There was still some resistance. And that was partly academics recognising that they had to [take part in the exercise], they couldn’t ignore it. Partly, the government and the research council has been willing to tweak, amend and qualify the initial very hard-edged guidelines and adapt them for the humanities. So, it was two-way process, a dialogue.

Int16, academic, example 3

The announcement of the final REF regulations (HEFCE, 2011) was the climax of the long process of making ‘impact’ into a thinkable and manageable entity. The last iteration of the regulations constituted a co-creation of various actors (initial Australian policymakers of the RQF, HEFCE employees, academics, impact professionals, universities, professional organizations) who had contributed to it at different stages (in many rounds of consultations, workshops, talks and sessions across the country). ‘Impact’ as a notion was ‘talked into being’ in a polyphonic process (Angermuller, 2014a, 2014b) of debate, critique, consultation (“listening”, “getting feedback”) and adaptation (“tweaking”, “changing”, “amending hard-edged guidelines”) also in view of the pre-existing conditions of academia such as the friction between the ‘soft’ and ‘hard’ sciences (as mentioned in example 3). In effect, impact was constituted as an object of thought, and an area of academic activity begun to emerge around it.

The period of defining ‘impact’ as a new, important notion in academic discourse in the UK, roughly between July 2011 and November 2013, can be conceptualized in terms of the Foucauldian notion of ‘problematization’. This concept describes how spaces, areas of activity, persons, behaviors or practices become targeted by government, separated from others, and cast as ‘problems’ to be addressed with a set of techniques and regulations. ‘Problematisation’ is the moment when a notion “enters into the play of true and false, (…) is constituted as an object of thought (whether in the form of moral reflection, scientific knowledge, political analysis, etc.)” (Foucault, 1988, p. 257), when it “enters into the field of meaning” (Foucault, 1984, pp. 84–86). The problematization of an area triggers not only the establishment of new notions and objects but also of new practices and institutions. In consequence, the areas in question become subjugated to a new (political, administrative, financial) domination. This eventually shapes the way in which social subjects conceive of their world and of themselves. But a ‘problematisation’, however influential, cannot persist on its own. It requires an overarching structure in the form of an ‘apparatus’ which will consolidate and perpetuate it.

Impact infrastructure

Soon after the publication of the evaluation guidelines for REF2014, and still during the phase of ‘problematisation’ of impact, universities started collecting data on ‘impactful’ research conducted in their departments and recruiting authors of potential CSs which could be submitted for evaluation. The winding and iterative nature of the process of problematization of ‘impact’ made it difficult for research managers and researchers to keep track of the emerging knowledge around impact (official HEFCE documentation, results of the pilot evaluation, FAQs, workshops and sessions organized around the country, writings published in paper and online). At the stage of collecting drafts of CSs it was still unclear what would ‘count’ as impact and what evidence would be required. Hence, there emerged a need for specific procedures and specialized staff who would prepare the REF submissions.

At most institutions, specific posts were created for employees preparing impact submissions for REF2014. These were both secondment positions such as ‘impact lead’, ‘impact champion’ and full-time ones such as impact officer, impact manager. These professionals soon started organizing between themselves at meetings and workshops. Administrative units focused on impact (such as centers for impact and engagement, offices for impact and innovation) were created at many institutions. A body of knowledge on impact evaluation was soon consolidated, along with a specific vocabulary (‘a REF-able piece of research’, ‘pathways to impact’, ‘REF-readiness’ etc.) and sets of resources. Impact evaluation gave raise to the creation of a new type of specialized university employee, who in turn contributed to turning the ‘generation of impact’, as well as the collection and presentation of related data into a veritable field of professional expertize.

In order to ensure timely delivery of CSs to REF2014, institutions established fixed procedures related to the new practice of impact evaluation (periodic monitoring of impact, reporting on impact-related activities), frames (schedules, document templates), forms of knowledge transfer (workshops on impact generation or on writing in the CS genre), data systems and repositories for logging and storing impact-related data, and finally awards and grants for those with achievements (or potential) related to impact. Consultancy companies started offering commercial services focused on research impact, catering to universities and university departments but also to governments and research councils outside the UK looking at solutions for impact evaluation. There is even an online portal with a specific focus on showcasing researchers’ impact (Impact Story).

In consequence, impact became institutionalized as yet another “box to be ticked” on the list of academic achievements, another component of “academic excellence”. Alongside burdens connected to reporting on impact and following regulations in the area, there came also rewards. The rise of impact as a new (or newly-problematised) area of academic life opened up uncharted areas to be explored and opportunities for those who wished to prove themselves. These included jobs for those who had acquired (or could claim) expertize in the area of impact (Donovan, 2017, p. 3) and research avenues for those studying higher education and evaluation (after all, entirely new evaluation practices rarely emerge, as stressed by Power, 2015, p. 43). While much writing on the Impact Agenda highlights negative attitudes towards the exercise (Chubb et al., 2016; Sayer, 2015), equally worth noting are the opportunities that the establishment of a new element of the exercise opened. It is the energy of all those who engage with the concept (even in a critical way) that contributes to making it visible, real and robust.

The establishment of a specialized vocabulary, of formalized requirements and procedures, the creation of dedicated impact-related positions and departments, etc. contribute to the establishment of what can be described as an ‘impact infrastructure’ (comp. Power, 2015, p. 50) or in terms of Foucauldian governmentality theory as an ‘apparatus’Footnote 5. In Foucault’s terminology, ‘apparatus’ refers to a formation which encompasses the entirety of organizing practices (rituals, mechanisms, technologies) but also assumptions, expectations and values. It is the system of relations established between discursive and non-discursive elements as diverse as “institutions, architectural forms, regulatory decisions, laws, administrative measures, scientific statements, philosophical, moral and philanthropic propositions” (Foucault, 1980, p. 194). An apparatus servers a specific strategic function—responding to an urgent need which arises in a concrete time in history—for instance, regulating the behavior of a population.

There is a crucial discursive element to all the elements of the ‘impact apparatus’. While the creation of organizational units and jobs, the establishment of procedures and regulations, participation in meetings and workshops are no doubt ‘hard facts’ of academic life, they are nevertheless brought about and made real in discursive acts of naming, defining, delimiting and evaluating. The aim of the apparatus was to support the newly-established problematization of impact. It did so by operating on many levels: first of all, and most visibly, newly-established procedures enabled a timely and organized submission to the upcoming REF. Secondly, the apparatus guided the behavior of social actors. It did so not only through directive methods (enforcing impact-related requirements) but also through nurturing attitudes and dispositions which are necessary for the notion of impact to take root in academia (for instance via impact training delivered to early-career scholars).

Interviewed actors involved in implementing the policy in institutions recognized their role in orchestrating collective learning. An interviewed impact officer stated:

My feeling is that ultimately my post should not exist. In ten or fifteen years’ time, impact officers should have embedded the message [about impact] firmly enough that they [researchers] don’t need us anymore.

Int7, impact officer, example 4

A similar vision was evoked by a HEFCE policymaker who was asked if the notion of impact had become embedded in academic institutions:

I hope [after the next edition of REF] we will be able to say that it has become embedded. I think the question then will be “have we done enough in terms of case studies? Do we need something very much lighter-touch?” “Do we need anything at all?”—that’s a question. (…) If [impact] is embedded you don’t need to talk about it.

Int20, policy-maker, example 5

Rather than being an aim in itself, the Impact Agenda is a means of altering academic culture so that institutions and individual researchers become more mindful of the societal impacts of their research. The instillment of a “new impact culture” (see Manville et al., 2014, pp. 24–29) would ensure that academic subjects consider the question of ‘impact’ even outside of the framework of REF. The “culture shift” is to occur not just within institutions but ultimately within the subjects—it is in them that the notion of ‘impact’ has to become embedded. Hence, the final purpose of the apparatus would be to obscure the origins of the notion of ‘impact’ and the related practices, neutralizing the notion itself, and giving a guise of necessity to an evaluative reality which in fact is new and contingent.

The genre of impact case study as element of infrastructure

In this section two questions are addressed: (1) what are the features of the genre (or what is it like?) and (2) what are the functions of the genre (or what does it do? what vision of research does it instil?). In addressing the first question, I look at narrative patterns, as well as lexical and grammatical features of the genre. This part of the study draws on classical genre analysis (Bhatia, 1993; Swales, 1998)Footnote 6. The second question builds on the recognition, present in discourse studies since the 1970s’, that genres are not merely classes of texts with similar properties, but also veritable ‘dispositives of communication’. A genre is a means of articulation of legitimate speech; it does not just represent facts or reflect ideologies, it also acts on and alters the context in which it operates (Maingueneau, 2010, pp. 6–7). This awareness has engendered broader sociological approaches to genre which include their pragmatic functioning in institutional realities (Swales, 1998).

The genre of CS differs from other academic genres in that it did not emerge organically, but was established with a set of guidelines and a document template at a precise moment in time. The genre is partly reproductive, as it recycles existing patterns of academic texts, such as journal article, grant application, annual review, as well as case study templates applied elsewhere. The studied corpus is strikingly uniform, testifying to an established command of the genre amongst submitting authors. Identical expressions are used to describe impact across the corpus. Only very rarely is non-standard vocabulary used (e.g., “horizontal” and “vertical” impact rather then “reach” and “significance” of impact). This coherence can be contrasted with a much more diversified corpus of impact CSs submitted in Norway to an analogous exercise (Wróblewska, 2019). The rapid consolidation of the genre in British academia can be attributed to the perceived importance of impact evaluation exercise, which lead to the establishment of an impact infrastructure, with dedicated employees tasked with instilling the ‘culture of impact’.

In its nature, the CS is a performative, persuasive genre—its purpose is to convince the ‘ideal readers’ (the evaluators) of the quality of the underpinning research and the ‘breadth and significance’ of the described impact. The main characteristics of the genre stem directly from its persuasive aim. These are discussed below in terms of narrative patterns, and grammatical and lexical features.

Narrative patterns

On the level of narrative, there is an observable reliance on a generic pattern of story-telling frequent in fiction genres, such as myths or legends, namely the Situation-Problem–Response–Evaluation (SPRE) structure (also known as the Problem-Solution pattern, see Hoey, 1994, 2001 pp. 123–124). This is a well-known narrative which follows the SPRE pattern: a mountain ruled by a dragon (situation) which threats the neighboring town (problem) is sieged by a group of heroes (response), to lead to a happy ending or a new adventure (evaluation). Compare this to an example of the SPRE pattern in a sample impact narrative from the studied corpus:

Mosetén is an endangered language spoken by approximately 800 indigenous people (…) (SITUATION). Many Mosetén children only learn the majority language, Spanish (PROBLEM). Research at [University] has resulted in the development of language materials for the Mosetenes. (…) (RESPONSE). It has therefore had a direct influence in avoiding linguistic and cultural loss. (EVALUATION).

CS40828Footnote 7

The SPRE pattern is complemented by patterns of Further Impact and Further Corroboration. The first one allows elaborating the narrative, e.g., by showing additional (positive) outcomes, so that the impact is not presented as an isolated event, but rather as the beginning of a series of collaborations, e.g.,:

The research was published in [outlet] (…). This led to an invitation from the United Nations Environment Programme for [researcher](FURTHER IMPACT).

CS22177

Patterns of ‘further impact’ are often built around linking words, such as: “X led to” (n = 78)Footnote 8, “as a result” (n in the corpus =31), “leading to” (n = 24), “resulting in” (n = 13), “followed” (“X followed Y”–n = 14). Figure 1 below shows a ‘word tree’ for a frequent linking structure “led to”. The size of the terms in the diagram represents frequencies of terms in the corpus. Reading the word tree from left to right enables following typical sentence structures built around the ‘led to’ phrase: research led to an impact (fundamental change/development/establishment/production of…); impact “led to” further impact.

Fig. 1: Word tree with string ‘led to' prepared with MaxQDA software.
figure 1

Word tree with string ‘led to'. This word tree with string ‘led to’ was prepared with MaxQDA software. It visualises a frequent sentence structure where research led to impact (fundamental change/ development/ establishment/ production of…) or otherwise how impact “led to” further impact.

Full size image

The ‘Further Corroboration’ pattern provides additional information which strengthens the previously provided corroborative material:

(T)he book has been used on the (…) course rated outstanding by Ofsted, at the University [Name](FURTHER CORROBORATION).

CS36979

Grammatical and lexical features

Both on a grammatical and lexical level, there is a visible focus on numbers and size. In making the point on the breadth and significance of impact, CS authors frequently showcase (high) numbers related to the research audience (numbers of copies sold, audience sizes, downloads but also, increasingly, tweets, likes, Facebook friends and followers). Adjectives used in the CSs appear frequently in the superlative or with modifiers which intensify them: “Professor [name] undertook a majorFootnote 9 ESRC funded project”; “[the database] now hosts one of the world’s largest and richest collections (…) of corpora”; “work which meets the highest standards of international lexicographical practice”; “this experience (…) is extremely empowering for local communities”, “Reach: Worldwide and huge”.

Use of ‘positive words’ constitutes part of the same phenomenon. These appear often in the main narrative on research and impact, and even more frequently in quoted testimonials. Research is described in the CSs as being new, unique and important with the use of words such as “innovative” (n = 29), “influential” (n = 16), “outstanding” (n = 12), “novel” (n = 10), “excellent” (n = 8), “ground-breaking” (n = 7), “tremendous” (n = 4), “path-breaking” (n = 2), etc. The same qualities are also rendered descriptively, with the use of words that can be qualified as boosters e.g., “[the research] has enabled a complete rethink of the relationship between [areas]”; “vitally important [research]”.

Novelty of research is also frequently highlighted with the adjective “first” appearing in the corpus 70 timesFootnote 10. While in itself “first” is not positive or negative, it carries a big charge in the academic world where primacy of discovery is key. Authors often boast about having for the first time produced a type of research—“this was the first handbook of discourse studies written”…, studied a particular area—“This is the first text-oriented discourse analytic study”…, compiled a type of data—“[We] provid[ed] for the first time reliable nationwide data”; “[the] project created the first on-line database of…”, or proven a thesis: “this research was the first to show that”…

Another striking lexical characteristic of the CSs is the presence of fixed expressions in the narrative on research impact. I refer to these as ‘impact speak’. There are several collocations with ‘impact’, the most frequent being “impact on” (n = 103) followed by the ‘type’ of impact achieved (impact on knowledge), area/topic (impact on curricula) or audience (Impact on Professional Interpreters). This collocation often includes qualifiers of impact such as “significant”, “wide”, “primary”,“secondary”, “broader”, “key”, and boosters: great, positive, wide, notable, substantial, worldwide, major, fundamental, immense etc. Impact featured in the corpus also as a transitive verb (n = 22) in the forms “impacted” and “impacting”—e.g., “[research] has (…) impacted on public values and discourse”. This is interesting, as use of ‘impact’ as a verb is still often considered colloquial. Verb collocations with ‘impact’ are connected to achieving influence (“lead to..”, “maximize…”, “deliver impact”) and proving the existence and quality of impact (“to claim”, “to corroborate” impact, “to vouch for” impact, “to confirm” impact, to “give evidence” for impact). Another salient collocation is “pathways to impact” (n = 14), an expression describing channels of interacting with the public, in the corpus occasionally shortened to just “pathways” e.g., “The pathways have been primarily via consultancy”. This phrase has most likely made its way to the genre of CS from the Research Councils UK ‘Pathways to Impact’ format introduced as part of grant applications in 2009 (discontinued in early 2020).

On a syntactic level, CSs are rich in parallel constructions of enumeration, for instance: “(t)ranslators, lawyers, schools, colleges and the wider public of Welsh speakers are among (…) users [of research]”; “the research has benefited a broad, international user base including endangered language speakers and community members, language activists, poets and others”; [the users of the research come] “from various countries including India, Turkey, China, South Korea, Venezuela, Uzbekistan, and Japan”. Listing, alongside providing figures, is one of the standard ways of signaling the breadth and significance of impact. Both lists and superlatives support the persuasive function of the genre. In terms of verbal forms, passive verbs are clearly favored and personal pronouns (“I, we”) are avoided: “research was conducted”, “advice was provided”, “contracts were undertaken”.

Vision of research promoted by the genre of CS

Impact CS is a new, influential genre which affects its academic context by celebrating and inviting a particular vision of successful research and impact. It sets a standard for capturing and describing a newly-problematized academic object. This standard will be a point of reference for future authors of CSs. Hence, it is worth taking a look at the vision on research it instills.

The SPRE pattern used in the studied CSs favors a vision of research that is linear: work proceeds from research question to results without interference. The Situation and Problem elements are underplayed in favor of elaborate descriptions of the researchers’ ‘Reactions’ (research and outreach/impact activities) and flattering ‘Evaluations’ (descriptions of effects of the research and data supporting these claims). Most narratives are devoid of challenges (the ‘Problem’ element is underplayed, possible drawbacks and failures in the research process are mentioned sporadically). Furthermore, narratives are clearly goal-oriented: impact is shown as included in the research design from the beginning (e.g., impact is frequently mentioned already in section 2 ‘Underpinning research’, rather than the latter one ‘Details of the impact’). Elements of chance, luck, serendipity in the research process are erased—this is reinforced by the presence of patterns of ‘further proof’ and ‘further corroboration’. As such, the bulk of studied CSs channel a vision of what is referred to in Science Studies as ‘normal’ (deterministic, linear) science (Kuhn, 1970, pp. 10–42). From a purely literary perspective this makes for rather dull narratives: “fairy-tales of researcher-heroes… but with no dragons to be slain” (Selby, 2016).

The few CSs which do discuss obstacles in the research process or in securing impact stand out as strikingly diverse from the rest of the corpus. Paradoxically, while apparently ‘weakening’ the argumentation, they render it more engaging and convincing. This effect has been observed also in in an analogous corpus of Norwegian CSs which tend to problematize the pathway from research to impact to a much higher degree (Wróblewska, 2019, pp. 34–35).

The lexical and grammatical features of the CSs—the proliferation of ‘positive words’, including superlatives, and the adjective “first”— contribute to an idealization of the research process. The documents channel a vision of academia where there is no place for simply ‘good’ research—all CSs seem based on ‘excellent’ and ‘ground-breaking’ projects. The quality of research underpinning impact is recognized in CSs in a straightforward, simplistic way (quotation numbers, peer reviewed papers, publications in top journals, submission to REF), which contributes to normalizing the view of research quality as easily measurable. Similarly, testimonials related to impact are not all equal. Sources of corroboration cited in CSs were carefully selected to appear prestigious and trustworthy. Testimonials and statements from high-ranking officials (but also ‘celebrities’ such as famous intellectuals or political leaders) were particularly sought-after. The end effect reinforces a solidified vision of a hierarchy of worth and trustworthiness in academia.

The prevalence of impersonal verbal forms suggests an de-personalized vision of the research process (“work was conducted”, “papers were published”, “evidence was given…”), where individual factors such as personal aspirations, constraints or ambitions are effaced. The importance given to numbers contributes to a strengthening of a ‘quantifiable’ idea of impact. This is in line with a trend observed in academic writing in general – the inflation of ‘positive words’ (boosters and superlatives) (Vinkers et al., 2015). This tendency is amplified in the genre of CS, particularly in its British iteration. In a Norwegian corpus claims to excellence of research and breadth and significance of impact were significantly more modest (Wróblewska, 2019, pp. 28–30).

The genre of impact CS is a core binding component of the impact infrastructure: all the remaining elements of this formation are mutually connected by a common aim – the generation of CSs. While the CS genre, together with the encompassing impact infrastructure, is vested with a seductive/coercive force, the subjects whose work it represents and who produce it take different positions in its face.

Academics’ positioning towards the Impact Agenda

Academics position themselves towards the concept of impact in many explicit and implicit ways. ‘Positioning’ is understood here as performance-based claims to identity and subjectivity (Davies and Harré, 1990, Harré and Van Langenhove, 1998). Rejecting the idea of stable “inherent” identities, positioning theorists stress how different roles are invoked and enacted in a continuous game of positioning (oneself) and being positioned (by others). Positioning in academic contexts may take the form of indexing identities such as “professor”, “linguist”, “research manager”, “SSH scholar”, “intellectual”, “maverick” etc. (Angermuller, 2013; Baert, 2012, Hamann, 2016, Hah, 2019, 2020). Also many daily interactions which do not include explicit identity claims involve subject positioning, as they carry value judgments, thereby also evoking counter-statements and colliding social contexts (Tirado and Galvaz, 2008, pp. 32–45).

My analysis draws attention to the process of incorporating impact into academic subjectivities. I look firstly at the mechanics of academics’ positioning towards impact: the game of opposite discursive acts of distancing and endorsement. Academics reject the notion of ‘impact’ by ironizing, stage management and use of metaphors. Conversely, they may actively incorporate impact into their presentation of academic ‘self’. This discursive engagement with the notion of impact can be described as ‘subjectivation’, i.e., the process whereby subjects re(establish) themselves in relation to the grid of power/knowledge in which they function (in this case the emergent ‘impact infrastructure’).

Distancing

The relatively high response rate of this study (~50%) and the visible eagerness of respondents to discuss the question of impact suggest an emotional response of academics to the topic of impact evaluation. Yet, respondents visibly struggled with the notion of ‘impact’, often distancing themselves from it through discursive devices, the most salient being ironizing, use of metaphors and stage management.

Ironizing the notion of impact

In many cases, before proceeding to explain their attitude to impact, interviewed academics elaborated on the notion of impact, explaining how the notion applied to their discipline or field and what it meant for them personally. This often meant rejecting the official definition of impact or redefining the concept. In excerpt 6, the interviewee picks up the notion:

Impact… I don’t even like the word! (…) It sounds [like] a very aggressive word, you know, impact, impact! I don’t want to impact! What you want, and what has happened with [my research] really is… more of a dialogue.

Int21, academic, example 6

Another respondent brought up the notion of impact when discussing ethical challenges arising from public dissemination of research.

When you manage to go through that and navigate successfully, and keep producing research, to be honest, that’s impact for me.

Int9, academic, example 7

An analogous distinction was made by a third respondent who discussed the effect of his work on an area of professional activity. While, as he explained, this application of his research has been a source of personal satisfaction, he refused to describe his work in terms of ‘impact’. He stressed that the type of influence he aims for does not lend itself to producing a CS (is not ‘REF-able’):

That’s not impact in the way this government wants it! Cause I have no evidence. I just changed someone’s view. Is that impact? Yes, for me it is. But it is not impact as understood by the bloody REF.

Int3, academic, example 8

These are but three examples of many in the studied corpus where speakers take up the notion of impact to redefine or nuance it, often juxtaposing it with adjacent notions of public engagement, dissemination, outreach, social responsibility, activism etc. A previous section highlighted how the definition of impact was collectively constructed by a community in a process of problematization. The above-cited examples illustrate the reverse of this phenomenon—namely, how individual social actors actively relate to an existing notion in a process of denying, re-defining, and delimiting.

These opposite tendencies of narrowing down and again widening a definition are in line with the theory of the double role of descriptions in discourse. Definitions are both constructions and constructive—while they are effects of discourse, they can also become ‘building blocks’ for ideas, identities and attitudes (Potter, 1996, p. 99). By participating in impact-related workshops academics ‘reify’ the existing, official definition by enacting it within the impact infrastructure. Fragments cited above exemplify the opposite strategy of undermining the adequacy of the description or ‘ironizing’ the notion (Ibid, p.107). The tension between reifying and ironizing points to the winding, conflictual nature of the process of accepting and endorsing the new ‘culture of impact’. A recognition of the multiple meanings given to the notion of ‘impact’ by policy-makers, academic managers and scholars may caution us in relation to studies on attitudes towards impact which take the notion at face value.

Metaphors

Respondents nuanced the notion of impact also through the use of metaphors. In discourse analysis metaphors are seen in not just as stylistic devices but as vehicles for attitudes and values (Mussolf, 2004, 2012). Many of the respondents make remarks on the ‘realness’ or ‘seriousness’ of the exercise, emphasizing its conventional, artificial nature. Interviewees admitted that claims made in the CSs tend to be exaggerated. At the same time, they stressed that this was in line with the convention of the genre, the nature of which was clear for authors and panelists alike. The practice of impact evaluation was frequently represented metaphorically as a game. See excerpt 9 below:

To be perfectly honest, I view the REF and all of this sort of regulatory mechanisms as something of a game that everybody has to play. The motivation [to submit to REF] was really: if they are going to make us jump through that hoop, we are clever enough to jump through any hoops that any politician can set.

Int14, academic, example 9

Regarding the relation of the narratives in the CSs to truth see example 10:

[A CS] is creative stuff. Given that this is anonymous, I can say that it’s just creative fiction. I wouldn’t say we [authors of CSs] lie, because we don’t, but we kind of… spin. We try to show a reality which, by some stretch of imagination is there. (It’s) a truth. I’m not lying. Can it be shown in different ways? Yes, it can, and then it would be possibly less. But I choose, for obvious reasons, to say that my external funding is X million, which is a truth.

Int3, academic, example 10

The metaphors of “playing a game”, “jumping through hoops” suggest a competition which one does not enter voluntarily (“everybody has to play it”) while those of “creative fiction”, “spinning”, presenting “a truth” point to an element of power struggle over defining the rules of the game. Doing well in the exercise can mean outsmarting those who establish the framework (politicians) by “performing” particularly well. This can be achieved by eagerly fulfilling the requirements of the genre of CS, and at the same time maintaining a disengaged position from the “regulatory mechanism” of the impact infrastructure.

Stage management

Academics’ positioning towards impact plays out also through management of ‘stage’ of discursive performance, often taking the form of frontstage and backstage markers (in the sense of Goffman’s dramaturgy–1969, pp. 92–122). For instance, references to the confidential nature of the interview (see example 10 above) or the expression “to be perfectly honest” (example 9), are backstage markers. Most of the study’s participants have authored narratives about their work in the strict, formalized genre of CS, thereby performing on the Goffmanian ‘front stage’ for an audience composed of senior management, REF panelists and, ultimately, perhaps “politicians”, “the government”. However, when speaking on the ‘back stage’ context of an anonymous interview, many researchers actively reject the accuracy of the submitted CSs as representations of their work. Many express a nuanced, often critical, view on impact.

Respondents frequently differentiate between the way they perceive ‘impact’ on different ‘levels’, or from the viewpoint of their different ‘roles’ (scholar, research manager, citizen…). One academic can hold different (even contradictory) views on the assessment of impact. Someone who strongly criticizes the Impact Agenda as an administrative practice might be supportive of ‘impact’ on a personal level or vice versa. See the answer of a linguist asked whether ‘impact’ enters into play when he assesses the work of other academics:

When I look at other people’s work work as a linguist, I don’t worry about that stuff. (…) As an administrator, I think that linguistics, like many sciences, has neglected the public. (…) At some point, when we would be talking about promotion (…) I would want to take a look at the impact of their work. (…) And that would come into my thinking in different times.

Int13, academic, example 11

Interestingly, in the studied corpus there isn’t a simple correlation between conducting research which easily ‘lends itself to impact’ and a positive overall attitude to impact evaluation.

Subjectivation

The most interesting data excerpts in this study are perhaps the ones where respondents wittingly or unwittingly expose their hesitations, uncertainties and struggles in positioning themselves towards the concept of impact. In theoretical terms, these can be interpreted as symptoms of an ongoing process of ‘subjectivation’.

‘Subjectivation’ is another concept rooted in Foucauldian governmentality theory. According to Foucault, individuals come to the ‘truth’ about their subjectivity by actively relating to a pre-existent set of codes, patterns, rules and rituals suggested by their culture or social group (Castellani, 1999, pp. 257–258; Foucault, 1988, p. 11). The term ‘subjectivation’ refers to the process in which an individual establishes oneself in relation to the grid of power/knowledge in which they function. This includes actions subjects take on their performance, competences, attitudes, self-esteem, desires etc. in order to improve, regulate or reform themselves (Dean, 1999, p. 20; Lemke, 2002; Rose, 1999, p. xii).

Academics often distance themselves from the assessment exercise, as shown in previous sections. And yet, the data hints that having taken part in the evaluation and engaged with the impact infrastructure was not without influence on the way they present their research, also in nonofficial, non-evaluative contexts, such as the research interview. This effect is visible in vocabulary choices—interviewees routinely spoke about ‘pathways to impact’, ‘impact generation’, ‘REF-ability’ etc. ‘Impact speak’ has made its way into every-day, casual academic conversations. Beyond changes to vocabulary, there is a more deep-running process—the discursive work of reframing one’s research in view of the evaluation exercise and in its terms. Many respondents seemed to adjust the presentation of their research, its focus and aims, when the topic of REF surfaced in the exchange. Interestingly, such shifts occurred even in the case of respondents who did not submit to the exercise, for instance because they were already retired, or because they refused to take part in it. For those who have submitted CSs to REF, the effect of having re-framed the narrative of their research in this new genre often had a tremendous effect.

Below presented is the example of a scholar who did not initially volunteer to submit a CS, and was reluctant to take part when she was encouraged by a supervisor. During the interview the respondent distanced herself from the exercise and the concept of impact through the discursive devices of ironizing, metaphors, stage management, and humor. The respondent was consistently critical towards impact in course of the interview. Therefore the researcher expected a firm negative answer to the final question: “did the exercise affect your perception of your work?”. See excerpt 13 below for her the respondent’s somewhat surprising answer.

Do you know what? It did, it did, it did. Almost a kind of a massive influence it had. Maybe this is the answer that you didn’t see coming ((laughing)). (…) It did [have an influence] but maybe from a different route as for people who were signed up for [the REF submission] from the outset. (…) When I saw this [CS narrative] being shaped up and people [who gave testimonies] I kind of thought: goodness me! And there were other moving things.

Int21, academic, example 13

Through the preparation of the CS and particularly through familiarizing herself with the underpinning testimonials, the respondent gained greater awareness of an area of practice which was influenced by her research. The interviewee’s attitude changed not only in the course of the evaluation exercise, but also—as if mirroring this process—during the interview. In both cases, elements which were up to that moment implicit (the response of end-users of the work, the researcher’s own emotional response to the exercise and to the written-up narrative of her impact) were made explicit. It is the process of recounting one’s story in a different framework, according to other norms and values (and in a different genre) that triggers the process of subjectivation. This example of a change of attitude in an initially reluctant subject demonstrates the difficulty in opposing the overwhelming force of the impact infrastructure, particularly in view of the (sometimes unexpected) rewards that it offers.

Many respondents found taking part in the REF submission—including the discursive work on the narrative of their research—an exhausting experience. In some cases however, the process of reshaping one’s academic identity triggered by the Agenda was a welcome development. Several interviewees claimed that the exercise valorized their extra-academic involvement which previously went unnoticed at their department. These scholars embraced the genre of CS as an opportunity to present their impact-related activities as an inherent part of their academic work. One academic stated:

At last, I can take my academic identity and my activist identity and roll them up into one.

Int11, academic, example 14

Existing studies have focused on situating academics’ attitudes towards the Impact Agenda on a positive-negative scale (e.g., Chubb et al., 2016), and studied divergences depending on career stage or disciplinary affiliation etc. (Chikoore, 2016; Chikoore and Probets, 2016; Weinstein et al., 2019). My data shows that there are many dimensions to each academic’s view of impact. Scholars have complex (sometimes even contradictory) views on ‘impact’ and the discursive work in incorporating impact into a coherent academic ‘self’ is ongoing. While an often overwhelming ‘impact infrastructure’ looms over professional discursive positioning practices, academic subjects are by no means passive recipients of governmental new-managerial policies. On the contrary, they are agents actively involved in accepting, rejecting and negotiating them on a local level—both in front-stage and back-stage contexts.

Looking at the front stage, most CSs seem compliant in their eagerness to demonstrate impact in all its breadth and significance. The documents showcase large numbers and data once considered trivial in the academic context (Facebook likes, Twitter followers, endorsement of celebrities…) and faithfully follow the policy documents in adopting ‘impact speak’. Interviews with academics paint a different picture: the respondents may be playing according to the rules of the evaluation “game”, but they are playing consciously, often in an emotionally detached, distanced manner. Other scholars adjust to the regulations, but not in the name of compliance, but in view of an alignment between the goals of the Agenda and their personal ones. Finally, some academics perceive the evaluation of impact as an opportunity to re-position themselves professionally or re-claim areas of activity which were long considered non-essential for an academic career, like public engagement, outreach and activism.

Concluding remarks

The initial, dynamic phases of the introduction of impact to British academia represent, in terms of Foucauldian theory, the phase of ‘emergence’. This notion draws attention to the moment when discursive concepts (‘impact’, ‘impact case study’…) surface and consolidate. It is in these terms that the previously non-regulated area of academic activity will be thereon described, assessed, evaluated. New notions, definitions, procedures related to impact and the genre of CS will continue to circulate, emerging in other evaluation exercises, at other institutions, in other countries.

The stage of emergence is characterized by a struggle of forces, an often violent conflict between opposing ideas—“it is their eruption, the leap from the wings to centre stage” (Foucault, 1984, p. 84). The shape that an emergent idea will eventually take is the effect of clashes of these forces and it does not fully depend on any of them. Importantly, emergence is merely “the entry of forces” (p. 84), and “not the final term of historical development” (p. 83). For Foucault, a concept, in its inception, is essentially an empty word, which addresses the needs of a field that is being problematized and satisfies the powers which target it. A problematization (of an object, practice, area of activity) is a response to particular desires or problems—these constitute an instigation, but do not determine the shape of the problematization. As Foucault urges “to one single set of difficulties, several responses can be made” (2003, p. 24).

With the emergence of the Impact Agenda, an area of activity which has always existed (the collaboration of academics with the non-academic world) was targeted, delimited and described with new notions in a process of problematization. The notion of ‘impact’ together with the genre created for capturing it became the core of an administrative machinery—the impact infrastructure. This was a new reality that academics had to quickly come to terms with, positioning themselves towards it in a process of subjectification.

The run-up to REF2014 was a crucial and defining phase, but it was only the first stage of a longer process—the emergence of the concept of ‘impact’, the establishment of basic rules which would govern its generation, documentation, evaluation. Let’s recall Foucault’s argument that “rules are empty in themselves, violent and unfinalized; they are impersonal and can be bent to any purpose. The successes of history belong to those who are capable of seizing these rules”… (pp. 85–86). The rules embodied in the REF guidelines, the new genre of CS, the principals of ‘impact speak’ were in the first instance still “empty and unfinalized”. It was up to those subject to the rules to fill them with meaning.

The data analyzed in this study shows that despite dealing with a new powerful problematization and functioning in the framework of a complex infrastructure, academics continue to be active and highly reflective subjects, who discursively negotiate key concepts of the impact infrastructure and their own position within it. It will be fascinating to study the emergence of analogous evaluation systems in other countries and institutions. ‘Impact infrastructure’ and ‘genre’ are two excellent starting points for an analysis of ensuing changes to academic realities and subjectivities.