Introduction

There is a persistently popular view of research methods as neutral, technical tools that researchers can pick up and use to learn about a reality ‘out there’ (Schwartz-Shea and Yanow, 2011; Moon et al., 2019). In such a view, developing competency in a given method is primarily about learning to apply the relevant technical routines and procedures to obtain reliable results (e.g., King et al., 1994; Sutherland et al., 2018). Ethical issues are considered an important but relatively minor aspect of methods training, primarily concerning the avoidance of direct harm to research participants, while political considerations are usually framed as what policy-makers and practitioners might (not) do with the research findings rather than as part of the practice of the methods themselves (e.g., Newing, 2011). In recent decades, growing recognition of complexity has led to the rise of mixed methods approaches in the social sciences, where researchers are encouraged to combine methods—often differentiated along a qualitative-quantitative spectrum—to gain a more comprehensive understanding of a given phenomenon (Johnson and Onwuegbuzie, 2004; Tashakkori et al., 2021) and to better address complex societal challenges (Mertens et al., 2015). This latter transformative aim has been especially advocated in applied fields such as health, education, and our own field of sustainability science, where mixing methods is also encouraged across disciplines in interdisciplinary research (Klein, 2008; Barry and Born, 2013; Wehrden et al., 2017) and in collaboration with societal actors in transdisciplinary and co-production research (Albrecht et al., 1998; Lang et al., 2012; Norström et al., 2020). It is commonly suggested that different methods each provide ‘a piece of the puzzle,’ which together add up to a clearer picture of a single shared reality (Fig. 1A).

Fig. 1: Changing understandings of research methods.
figure 1

A It illustrates the view that different research methods each provide their own ‘puzzle pieces,’ which can all be fitted together to provide a clearer picture of reality. B It adds complexity to this view, by highlighting the presence of different methodological traditions with differing and sometimes incommensurable assumptions. C It adds further complexity still by suggesting that these differing traditions do not simply describe different realities but also inevitably participate in and help to create the realities they describe. Illustration by Elsa Wikander/Azote.

Full size image

Yet closer attention to methods muddies this attractively simple view. The methodological literature increasingly emphasises that the use of methods is conditioned by and reflects variable, inherently contestable philosophical assumptions about the nature of reality (ontology), how that reality can be known (epistemology), and notions of good or ethical practice (axiology) (Fay, 1996; Abbott, 2004; Tashakkori et al., 2021). These assumptions shape the ways that researchers identify topics, formulate questions, choose and use methods, and report results. Differences in such assumptions produce distinctive methodological approaches—including, for example, positivist, interpretivist and critical realist—that cut across methods and disciplines (Moon and Blackman, 2014). These different approaches each produce their own ethical standards, practices and considerations that guide the use of methods and interactions between researchers and participants (informing decisions about, for example, participant anonymity and informed consent). The inherently contestable character of the philosophical assumptions that underpin different methodological approaches introduces an unavoidably value-based or ‘political’ aspect to the choice of which assumptions to adopt (Lélé and Norgaard, 2005). Nevertheless, these assumptions often remain hidden for researchers working within a particular methodological approach, as they constitute tacit knowledge implicitly reinforced through years of training and education (Schwartz-Shea and Yanow, 2011).

If these philosophical assumptions are applied consistently across methods or are shared within a research group they often aid mixed-methods research (even if they remain hidden to the researchers involved). For example a single researcher might adopt similar assumptions in their use of interviews and surveys, or an interdisciplinary team in their use of controlled behavioural experiments and agent-based models (Schwartz-Shea and Yanow, 2011; Wijermans et al., 2022). However, differences in these assumptions can create significant “cognitive-epistemic, social and organisational, communicative, and technical challenges” for researchers engaged in mixed-methods research, causing stress, conflict, or even complete breakdowns in collaboration (Lang et al., 2012: p. 37; Lélé and Norgaard, 2005). Consequently, there has been increasing attention to the “key competencies” and expertise necessary for researchers to effectively navigate the challenges of mixing methods in inter-, transdisciplinary and co-production research (Wiek et al., 2011: p. 203; Brundiers et al., 2021; Bammer et al., 2020). For example, Haider et al. (2018: p. 191) propose “methodological groundedness” (a thorough knowledge of one methodological approach) and “epistemological agility” (the ability to recognise and work with those adopting different approaches) as key competencies for researchers in inter- and transdisciplinary sustainability science. To stay with the jigsaw puzzle analogy, these insights raise the possibility that different methodological approaches might not all be contributing parts to one shared puzzle, but possibly to their own, sometimes radically different puzzles (Fig. 1B).

To muddy the waters further still, there has been increasing recognition in recent decades that the various social scientific methodological approaches and methods do not simply provide descriptions of differing realities, but actively help to make and bring these realities into being (Giddens, 1990; Flyvbjerg, 2001; Law and Urry, 2004; Law, 2015; de la Cadena and Blaser, 2018). For example, Osborne and Rose (1999) describe how the methodological innovation of the representative sample, along with changing ideas about the role of citizens, helped to create the notion of “public opinion” which, reinforced through regular surveys and opinion polls, has become a reality of contemporary democratic governance. Likewise, social scientific concepts as diverse as “the rational actor” and “institutional racism” have not only shaped policies but also human self-understandings and relationships, which have subsequently become targets for further study, intervention and transformation (e.g., Bregman, 2020, MacKenzie et al., 2008; Murji, 2007). Performative understandings of research methods, building on these insights and often inspired by complexity thinking, have suggested that if methods help to produce or “enact” different possible realities, then it follows that researchers face a degree of choice in the realities they work to strengthen (Mol, 1999; Law and Urry, 2004; Law, 2009). Such understandings both deepen and link the significance of the ethical and political dimensions of research methods: the differing ethical practices that guide interactions between researchers and participants are fundamentally linked to the differing social effects that methods help to produce and the realities they help to build (this is what Wingrove, qtd. in Schwartz-Shea (2006: p. 209), calls the “world-affecting” character of methods). We refer to these interlinked dimensions as the ethical-political dimensions of research methods. Returning one final time to the puzzle analogy, performative social science suggests that the different puzzles built by researchers inevitably participate in social life, helping to strengthen some possible realities and weaken others (Fig. 1C).

In sum, the multiple, contested and performative character of research methods highlights their inherent ethical-political dimensions, raising the ever-present question for the researcher, “what are the social effects of the puzzles I am contributing to?” It is important for all researchers to be be aware of the ethical-political dimensions of research methods so that they can better understand the interplay between their assumptions and intentions, their methodological practices, and the social effects of their research. However, it is especially important for researchers engaged in mixed methods, inter- and transdisciplinary, and co-production research for two main reasons: firstly, such researchers are more likely to encounter methodological differences, and therefore need to be able to recognise and negotiate the ethical-political dimensions of these differences; secondly, such researchers often aim to contribute to societal challenges and social change, so need to be able to think through the ethical-political consequences of their research. We therefore propose that the ability to recognise and negotiate the ethical-political dimensions of research methods should be considered a key competency of mixed methods, inter- and transdisciplinary, and co-production research. In the context of transdisciplinary sustainability research, this competency sits at the intersection of “values-thinking” and “integration” competencies, helping researchers to navigate the value-laden, ethical and political aspects of integrative research and problem-solving processes (Wiek et al., 2011; Brundiers et al., 2021; Redman and Wiek, 2021). It can also be considered a central aspect of “co-productive agility”—the ability to work reflexively across different knowledges to develop shared approaches to complex and contested social challenges (Chambers et al., 2022: p. 3). Identifying and negotiating these ethical-political dimensions can be difficult for those not used to thinking about the ethical-political significance of their methodological practices, but doing so can lead to unexpected alliances and novel, even possibly transformative, approaches to knowledge and action (Chambers et al., 2022).

In this paper, we—a team comprising one researcher working with an interpretive approach (Simon) and one with a behavioural approach (Caroline)—draw on our personal experiences leading a transdisciplinary sustainability research project to highlight the importance of recognising and negotiating the ethical-political dimensions of research methods. We begin by describing our mixed method research design, including photovoice and controlled behavioural experiments, and then how the preparation of several ethics applications produced dilemmas that forced us to confront the latent ethical-political dimensions of bringing these methods together. We then describe the practices and skills we found useful in navigating these ethical-political dimensions and arriving at provisional solutions. In so doing we provide a rare practical example of negotiating the ethical-political dimensions encountered in the real-time of an integrative, mixed-methods transdisciplinary project (e.g., Buizer et al., 2015; Cockburn and Cundill, 2018). We conclude by providing some pointers for evaluators in assessing the ethical-political rigour of mixed methods, inter- and transdisciplinary, and co-production research, and discuss how the competency to navigate the ethical-political dimensions of research methods might be better supported within research projects, graduate training programmes and research organisations.

Mixing research methods in the New Normal project

Within mixed methods, inter- and transdisciplinary, and co-production research there is significant debate about how graduate students should be trained: should they be encouraged to freely mix approaches and methods from the beginning of their training, or should they be encouraged to deeply learn a particular methodological approach and associated methods first, before then seeking to work with others (Haider et al., 2018)? More by chance and intuition than design, the authors of this paper followed the latter approach. We both conducted our PhDs in inter- and transdisciplinary sustainability science: Caroline followed a behavioural approach, using insights from behavioural economics, psychology, common-pool resources, and complex adaptive systems to explore human behaviour and collective action in relation to drastic environmental change (Schill et al., 2015; Schill, 2017), and Simon adopted an interpretive approach, using insights from interpretive policy analysis, discourse analysis, and critical social theory to explore contextual meanings and practices in natural resource management (West, 2016; West et al., 2019). As we finished our PhDs, we became intrigued by the similarities and differences between our two approaches. We felt we were both interested in the same broad topic—the complexity of human behaviour in relation to sustainability issues—but were working with very different assumptions and methodological practices. On the one hand, we were aware that within our inter- and transdisciplinary research community there was an expectation that researchers should seek to mix methods, yet on the other we were aware that the stereotypical view in the methodological literature was that behavioural (positivist) and interpretive approaches sat on either side of a great divide (Table 1). Nevertheless, we were also conscious that we identified to varying extents with these stereotypical views, and had a strong sense that these divisions were not as clear and rigid as they were often portrayed.

Table 1 The ‘stereotypical view’ of differences between interpretive and behavioural (positivist) methodological approaches. Adapted from Evered and Reis Louis (1981).
Full size table

We therefore began to develop a joint project that would explore and critically interrogate the possibility of integrating, mixing, or otherwise drawing on both behavioural and interpretive approaches to address sustainability challenges. In terms of methods, Caroline was interested in continuing her use of controlled behavioural experiments, and Simon was keen to use the photovoice method (Table 2). We arrived at the provisional idea that enacting the photovoice method first could help to generate the focus and framing for the experiments. Our reasons were partly ethical and partly epistemological: this sequencing would enable participants to ensure that the experiment would be relevant and sensitive to social-ecological context, while also helping to address issues of external validity. From this methodological starting point we then looked for a sustainability challenge to explore, arriving at the idea of exploring human responses to abrupt environmental change. The drivers and impacts of abrupt environmental change are a prominent research topic at our research centre (e.g., Rocha et al., 2015), yet human responses are less well understood. Finally, we looked for places or communities that might be interested in partnering with us in the research, considering factors such as community experiences of ecological change, language spoken, and the existence of local research partners. A contact—now a collaborator in the project—put us in touch with an Iñupiat village in northern Alaska (the name of the village is anonymised in this paper). This collaborator had conducted their PhD research on climate change adaptation in the village and thought that village authorities might be interested in further exploring their responses to the abrupt ecological changes brought about by climate change, including melting permafrost and sea ice, and changing subsistence patterns.

Table 2 Ideal-typical descriptions of photovoice and controlled behavioural experiments (with illustrative references).
Full size table

We met with village representatives, presented our initial ideas, and emphasised that while we came with a particular set of interests, we wanted to ensure that any research would be meaningful, useful and conducted in partnership with the community, and welcomed input on setting the terms and agenda of the project. Our proposal was welcomed, and the village representatives provided feedback and encouraged us to submit a funding proposal. At this stage we envisaged that photovoice would enable participants to share their experiences and stories of ecological change, which would then help to frame and inform the experiments as a way of exploring how the community might respond to different scenarios of ecological change. At the same time, we were aware that these ideas might evolve, and were interested in actively interrogating the ways in which we were attempting to integrate or mix methods, so we decided to make personal video diaries and include a communications expert specialising in videography to reflect on and document the project as it unfolded. We articulated the following guiding research questions:

  • What environmental changes matter most to northern Alaskan communities?

  • What effects are these changes having on subsistence lifestyles?

  • How might northern Alaskan communities act when faced with abrupt change in a key resource?

We tell this story of the origins of the project because it highlights that research projects, methods and questions are never ‘neutral,’ but emerge—right at the beginning—from a mélange of interests, commitments, agendas and chance events (Schwartz-Shea, 2006).

We gathered our ideas into a project proposal titled: Living with the ‘new normal’: exploring human responses to abrupt environmental change in the Arctic using behavioural and interpretive social science (the ‘New Normal’ project). The project was funded, and after two further meetings with the village authorities to plan the fieldwork (including the creation of a local Steering Committee to oversee the research composed of village elders), we were directed to complete an ethics application for the research according to the village research guidelines. In addition, we were required to complete an ethics application for our home research institution and, because the project was deemed to involve handling of “sensitive personal information” according to the EU-wide General Data Protection Regulation (GDPR), we were also required to submit ethics forms to the Swedish Ethical Review Board. These three ethics applications emphasised accountability to different audiences: to the Iñupiat rules and laws governing the conduct of research in the specific village we would be working with, to the ethical standards of Stockholm University (including the norms and expectations of our respective methodological communities), and to European conventions around data management and privacy. In the following sections, we first explore the dilemmas we encountered in integrating or ‘mixing’ the different ethical practices guiding the interactions between researchers and participants in photovoice and controlled behavioural experiments, before then showing how these ethical dilemmas are fundamentally linked to the social effects and realities enacted by each method.

Encountering ethical dilemmas in mixing photovoice and controlled behavioural experiments

In single-method research, ethical challenges arise through efforts to apply the ethical standards and assumptions within a given methodological approach to the particularities of a given topic and context (e.g., Ifcher and Zarghamee, 2016). When bringing multiple methods together, additional challenges are generated through the interaction of different methodological practices that may entail quite different standards of ethical behaviour. In this section we describe two such dilemmas that we encountered in developing our ethics applications for the New Normal project, drawing from our personal reflections captured in video diaries and facilitated dialogues (for more detail on these diaries and dialogues, see section ‘Developing the expertise to effectively negotiate the ethical-political dimensions of research methods’).

A priori vs. responsive research design

In controlled behavioural experiments the underlying ontological assumption is that there is an external reality that exists independently of the observer. The epistemological challenge is to design and conduct an experiment (consisting of ‘control’ and treatment conditions) that captures the essence of a given phenomenon and allows the researcher to test the hypothesis of interest by manipulating certain variables, with the aim of developing objective (minimised bias) and generalisable knowledge about the phenomenon. Therefore, the variables of interest, research design and procedure need to be established a priori, prior to data collection, and are known at the time of writing an ethics application. Within this set of assumptions and practices, it is considered ethical to carefully check the research design with scientific experts in terms of potential risks to participants prior to data collection. By contrast, in photovoice the underlying ontological assumption is usually that reality is generated in the interaction between the observer and the observed, and the epistemological challenge is to generate a rich and contextually valid understanding of the multiple realities experienced by research participants. Consequently, the research design is usually developed iteratively and collaboratively by the researcher with their participants, and while researchers might be able to state the general topic and principles guiding the research at the outset, it is almost impossible to give an exact account of the procedures prior to fieldwork. Within the assumptions and practices of photovoice, it is considered ethical to carefully negotiate the research design and potential risks with participants throughout the duration of the project. In both controlled behavioural experiments and photovoice, it is considered ethical to adhere to the principles of free, prior and informed consent and to ensure that participants know that participation is entirely voluntary, that they may withdraw at any time without giving a reason and without consequence, and can ask questions and raise concerns at any time.

We found that the contrasting ethical practices within experimental and photovoice methods concerning a priori vs. responsive research designs raised significant challenges in terms of developing an integrative design that could satisfy the different accountabilities at play in the various ethics applications. We had decided in the initial planning meetings for the project that the insights developed through photovoice would inform the design of the experiment. The participatory nature of photovoice would help us to meet the requirements in the village research guidelines that the community would be treated as a partner in study design, data collection, interpretation and publication of research, and that the experiment would be locally appropriate and relevant. Nevertheless, such a design made it difficult to respond to the requirements of the Swedish Ethical Review Board that we specify basic experimental design issues such as “how many people will be engaged in the research?” and “Please detail the techniques/treatments that will be followed.” Indeed, Simon began to feel uncomfortable as he realised that adhering to ‘his’ commitments in photovoice would directly affect Caroline’s ability to adhere to ‘hers’ in the experiment. As he reflected at the time, “I wonder if Caroline is struggling with that […] my answers to her questions about research design are always, ‘ah yeah well we’ll wait and see what happens! See what happens when we arrive [in the village], see what people want to do.’ Whereas I think that for experiments you actually have to [specify] what you’re going to do, far in advance of doing it.” These concerns were well-founded. Caroline reflects, “[the lack of detail] pushed me totally out of my comfort zone, and I started to fear that this would compromise the ethical integrity of the experiments.” Indeed, although waiting to establish the research design made good ethical sense in order to make the research appropriate and meaningful for the community (and to satisfy the photovoice criteria), it would also potentially put participants at risk in terms of participating in an experiment that hadn’t been properly scrutinised.

Anonymity vs. credit for knowledge contributions

In behavioural experiments, the epistemological assumption is that participants (representing a particular population) are contributing evidence through their decisions and behaviours rather than contributing knowledge per se. It is thought that these decisions and behaviours may be influenced or biased if participants know that the decisions they make will be made public. Moreover, there is a concern that connecting the identities of participants to their choices in the experiment risks exposing participants to negative repercussions in their community. It is typically considered ethical practice to guarantee participants anonymity, which implies that all decisions made by participants will be anonymous too. In direct contrast, photovoice rests on the basis that participants are contributing deeply personal knowledge and experiences to the research. There is often an explicit aim to publicly elevate these experiences and perspectives to bring about greater public recognition and understanding, and to contribute to positive policy or social change. These aims raise a complicated set of ethical considerations about agency, informed consent and possible social repercussions. Therefore it is considered ethical practice for researchers to explore together with participants the pros and cons of anonymity in particular situations, and provide participants the option to be named and credited for their work if they wish (however, it is important to note here that interpretive research more generally, as opposed to photovoice in particular, often does proceed on guarantees of anonymity).

The integration of these contrasting practices around anonymity vs. credit for knowledge contributions also posed significant challenges. We had initially intended for the photovoice participants to be part of the planning and implementation of the experiments, and to then conduct detailed post-experimental interviews in which participants would be able to reflect on their choices and behaviour within the experiment. Yet by integrating the photovoice, experiments and interviews in this way, we also blurred understanding of what we were asking participants to contribute to the research. The village research guidelines require that Iñupiat elders should have the option to be credited for their knowledge contributions, but if the participants who conducted a post-experimental interview were credited, this would violate the principle that their identity should not be linked with their behaviour in the experiment. As Caroline reflected: “So in my world, everything would be anonymous […] This is something you promise to participants. Moreover, this is really something that is so core [to a behavioural approach] I would not question it. It can also have very real consequences, because I would be terrified that—you know, if you put people from the community in a social dilemma in the experiment, and the outcome is regarded as something as bad and one could identify individuals that contributed to that bad outcome, that would just be a disaster because you would interfere with the relationships [in the community].” At the same time, Simon reflected: “The main thing we are taught in participatory action research is that it is ethical practice to provide the opportunity for participants to be credited for their contributions; but I now completely see that guaranteeing anonymity makes very good ethical sense from a behavioural approach. It is really tricky to figure out what to do.” The tension was that if participants received credit for their knowledge contributions in the post-experimental interviews, they would potentially be put at risk of harm through their specific contributions in the experiment becoming publicly known.

Surfacing the ethical-political dimensions of research methods

In seeking to resolve these two ethical dilemmas—a priori v. responsive research design, and anonymity v. credit for knowledge contributions—it was initially tempting for us to view each set of methodological practices as ‘correct’ within their own aims and frame of reference. This is what Schwartz-Shea (2006: p. 325) refers to as the “both-and” approach to methodological pluralism, and is consistent with a view that different methods each add their own “part of the puzzle” (Fig. 1A). From this view, the ethical dilemmas we faced might be resolved by attempting to respect each set of ethical standards and the integrity of each method on its own terms, for instance by paying careful attention to things like timing and coordination between each set of practices. However, while such a “both-and” approach might be possible within a more multi-disciplinary design—it is easy to imagine a project where an interpretive and behavioural researcher each performs their method separately to the other before then combining the data in some way—it was much more difficult to take such an approach within a more integrative design where each practice impinges on or affects the performance of the other. In addition, whether applied to single discipline or multi-, inter-, transdisciplinary or co-production research, the “both-and” approach ignores the manner in which methodological assumptions and their associated ethical practices are not only different, but also in some cases inherently oppositional. The oppositional nature of these assumptions and practices raises a set of more political questions about their differential effects in society and the different realities they help to enact. As Wingrove notes (qtd. in Schwartz-Shea, 2006: p. 313), “to assume that one or the other [approach] can or should simply accept one another is perhaps to not take seriously enough what they both take the political (=world-affecting) stakes to be.”

Both of our ethical dilemmas arose from the different ways that our methodological practices enact the relationships between researchers and participants, that in turn generate distinctive yet interlinked realities and social effects (Law and Urry, 2004). Positivist experimental practices tend to enact the researcher as an external expert and participants as discrete individuals that represent a specific population. Participants are configured as actors who make decisions and behave in certain ways, but who are not fully aware of their reasons for doing so; it is the job of the researcher to observe and analyse the behaviour of participants using statistical analysis in order to uncover general and objective behavioural patterns and regularities (Schwartz-Shea, 2006). The performance of positivist research practices more generally, repeated on a large scale and linked in with many other practices (including, for instance, particular practices of policy-making) generates the possibility of a field of expert, objective knowledge that can be used to inform governance based on notions of equal treatment (Law, 2009). Indeed, positivist research practices have been central to notions of equality and representative democratic systems within Western liberal democracies (Ezrahi, 1990). Yet it has also been argued—especially by those working with interpretive approaches—that positivist practices risk contributing to the instrumental control and manipulation of society and can produce technocratic forms of governance (Bevir and Blakely, 2018). By contrast, interpretive practices of photovoice enact researchers as facilitator-discussants, while participants are still enacted as discrete individuals but as members of distinctive historical and socio-cultural communities. Participants are configured as sense-makers and story-tellers, as knowledgeable experts of their own lives; the expertise of the researcher lies in eliciting and helping participants to express their expertise. The performance of interpretive practices more generally generates the possibility of a variety of legitimate knowledges and, emerging partly in critique of positivistic practices, have been central to notions of emancipation and deliberative democracy (Hajer and Wagenaar, 2003). Yet they have, in turn, also been challenged by those working with positivist approaches, who have raised concerns about their potential to undermine collective institutions in liberal democracies and have associated them with ‘post-truth’ governance (e.g., Fukuyama, 2018; Wight, 2018).

This deliberately broad and stereotypical sketch of the different yet interconnected realities generated by our methodological approaches begins to indicate that any resolution to our ethical dilemmas will also inevitably contain social and political dimensions. Indeed, the political stakes of these dilemmas are particularly high when conducting research with and in Indigenous communities (Tuhiwai Smith, 1999). Iñupiat knowledge practices produce governance systems and lived realities that differ substantially to Western traditions (interpretive and positivist), for example constituting Iñupiat people relationally rather than as discrete individuals (Inuit Circumpolar Council-Alaska, 2015; Topkok, 2015). Iñupiat ways of life continue to be threatened, restricted and harmed by colonialism, where Western research (of all types) plays a central role enacting Iñupiat communities in terms legible to Western institutions and settler-colonial liberal-democratic states (Heleniak and Napper, 2021). Consequently, the primary research aim of many Indigenous peoples in Alaska in recent decades has been to renew and revitalise Indigenous methodologies to strengthen Indigenous lives, lands and cultures (Hogan and Topkok, 2015; Zanotti et al., 2020). At the same time, however, Indigenous peoples have also articulated the importance of engaging in (and improving) Western research practices that can help to enhance self-determination (Reid et al., 2021), for instance through contributions to community planning (Raymond-Yakoubian and Daniel, 2018).

Such histories and imperatives highlight that that there is no way to remain neutral in resolving the ethical dilemmas we faced in the New Normal project: constructing an a priori or responsive research design, or choosing to maintain anonymity or provide credit to participants, are not just technical choices but also political acts with social consequences. They raise the question: “what are the possible consquences for participants and the community in general of being enacted in particular ways through our methods?” There are no easy or general answers to such questions—while photovoice may seem on first glance more responsive to protocols around providing credit for knowledge contributions, there are equally good reasons for Indigenous peoples to engage in positivist research practices, for example to aid in the construction of appropriate categories and indicators relevant to Indigenous communities (e.g., Smylie and Firestone, 2015), demonstrate the importance of Indigenous practices in terms recognisable within Western knowledge and governance systems (Yibarbuk et al., 2001), or to generate complementary knowledge about social or environmental change affecting Indigenous peoples (Reid et al., 2021). Rather, these questions raise the importance of careful and respectful deliberation among all those implicated in a research project in each particular context (Inuit Circumpolar Council, 2022). In light of an Iñupiat village choosing to partner with us in the New Normal project, we were compelled to improve our abilities to recognise and negotiate the ethical-political dimensions of our research methods and the implications of bringing them together.

Through these various considerations, it became clear that negotiating the ethical-political dimensions of mixing methods is not simply about balancing the practices of different methods, but also about interrogating the ethics of their very existence in each particular research context. For example, an Indigenous elder may feel ethically compelled to resist practices that they feel perpetuate colonialism; an interpretivist may feel ethically compelled to resist practices that they feel lead to the instrumental control of citizens; and a positivist may feel ethically compelled to resist practices that they feel reduce trust in collective institutions. Indeed, as we have engaged with these ethical-political dimensions we have sensed ourselves shifting from a simple “both-and” approach to mixing methods, towards what Schwartz-Shea (2006: p. 326) refers to as “reflexive” and what Barry and Born (2013: p. 12) describe as an “agonistic-antagonistic” approach. Such an approach has required that we nurture an ongoing critical dialogue in the New Normal project where we can mutually interrogate our aims, intentions and positionalities in our research, the ways we engage with participants, and the (variable) effects of our research in society. The use of the word ‘antagonistic’ might suggest at first glance that such dialogue offers only a route to conflict. However, a growing body of experience in mixed methods, inter- and transdisciplinary, and co-production research suggests that, if left untouched, these dimensions will either become the ‘elephant in the room’ that prevents effective collaboration in any case (Chambers et al., 2022) or will be surpressed in ways that restrict truly transformative contributions to society (Turnhout et al., 2020). Indeed, others have suggested that explicitly mobilising tensions can act as a heuristic source of discovery that can generate new forms of knowledge and action (Abbott, 2004; Pohl et al., 2021). We have found that explicitly surfacing the ethical-political dimensions of research methods has created a space for reflection and greater mutual understanding that has generated fertile ground for collaboration despite the significant methodological differences between us.

Developing the expertise to effectively negotiate the ethical-political dimensions of research methods

Explicitly surfacing and negotiating the ethical-political dimensions of mixing methods is difficult and potentially uncomfortable. In our experience, researchers are often more at ease when methodological integration, and indeed academic work in general, is framed in epistemological rather than ethical-political terms. Some traditions, including interpretive social science, have spent more time working on the ethical-political aspects of research than others and can be expected to ‘lead the way’ in these negotiations (e.g., Haraway, 1988; Harding, 1992). Yet at the same time, in our experience at least, interpretive researchers can be well-practised in critiquing other knowledge practices but less so in critically reflecting on their own. Genuine reflexivity is a difficult task for everyone and needs to be nurtured and applied equally across traditions and approaches. In this section, we describe some of the practices we have engaged in within the New Normal project, the skills we have developed through these practices, and how these skills have helped us to negotiate the ethical-political dimensions of our methods and devise provisional solutions to the dilemmas we have encountered (Fig. 2).

Fig. 2: Developing the competency to negotiate the ethical-political dimensions of research methods.
figure 2

In the New Normal project we engaged in practices including reading groups in philosophy, ethics and social studies of science, personal video diaries, and group facilitated dialogues. These practices helped us to develop skills in reflexivity and accountability, deliberation and practical wisdom, which in turn enhanced our competency to negotiate the ethical political dimensions of our research methods. Illustration by Elsa Wikander/Azote.

Full size image

Practices

Reading groups in philosophy, ethics and social studies of science

While in some academic traditions this may seem a statement of the obvious, reading philosophy, ethics and social studies of science is often neglected in problem-driven research, where such reading may be viewed as either unnecessary or an indulgence given the scale of the real-world challenges the research seeks to address (O’Rourke and Crowley, 2013). While we have some sympathy for such views, we have found that when working across different methodological approaches it is important to make an explicit space for discussion about the foundational assumptions, aims and purposes of the approaches involved, including mutual critiques (e.g., Guba and Lincoln, 1994; Abbott, 2004), as these are central to the possible social outcomes and impacts of research (e.g., Hogan and Topkok, 2015). We refer to social studies of science (also known as Science and Technology Studies, or STS) as well as philosophy and ethics because this literature explicitly explores the social contexts and effects of science, and in treating science as a social practice has helped us to make better comparisons between our approaches (e.g., Jasanoff, 2004; Barry and Born, 2013).

Personal video diaries

Video diaries and journals have been proposed as useful methods for nurturing reflection in inter-, transdisciplinary and co-production research (Norström et al., 2020; Montana et al., 2020), yet practical examples of such usage are rare (Pender and Jansen, 2020). In the New Normal project we have built in regular personal video diaries as a core research practice, to document, reflect on and evaluate our work together. We have each recorded at least 1-2 video diaries a month to address the following prompts: “What am I finding difficult in the project at the moment?” “What do I think is going well?” “What is difficult about working in this way?” “What is easy?” and “What do I feel excited/nervous about?” We decided not to plan our video diary entries in advance but to respond to the prompts spontaneously and freely, with the aim of maintaining the opportunity to surprise ourselves and enable the often implicit and emotional aspects of research to surface. We found that, combined with the reading groups, these diaries became a space where the ethical-political dimensions of our methods began to surface. For example, we both recorded additional video diaries to reflect on the challenges we faced in putting our ethics applications together and further explored these challenges in consequent facilitated dialogue sessions.

Group facilitated dialogues

While the video diaries have been a useful tool for our personal reflection, we felt that if they remained personal they would only indirectly contribute to group collaboration. We therefore decided that, at regular intervals throughout the project, we would watch and discuss each others video journals. Given the personal nature of the diaries, however, we felt that these discussions might be difficult to conduct on our own—for example, we might offend one another, or conversely, refrain from difficult yet possibly fruitful discussions for fear of disrupting our collaboration. We therefore engaged a dedicated communications expert as part of the project to help facilitate and document these discussions and conduct follow-up interviews. We have found that a facilitator has been crucial for us to be able to work through the differing epistemological, ontological and axiological assumptions in our approaches and the ethical-political dimensions these raise—affirming recent efforts to use facilitated philosophical dialogues to improve problem oriented inter- and transdisciplinary research (O’Rourke and Crowley, 2013; Hubbs et al., 2021; Hertz and Mancilla-Garcia, 2019). On the one hand, the presence of a facilitator has provided the space and the expertise for us to engage in difficult conversations that we might have otherwise ‘swept under the carpet,’ and on the other has helped us to identify issues and tensions that we would not have been able to identify on our own.

Skills

Reflexivity and accountability

Through the reading groups, diaries and facilitated dialogues, we have strengthened our skills in practising reflexivity: the ability to reflect critically on our methodological choices, our aims and intentions in our research, and the kinds of social effects and realities that our research might help to bring about (Montana et al., 2020). While reflexivity emphasises our own individual self-awareness of these issues, the notion of accountability draws attention to the broader social contexts and collectives in which we conduct research: raising questions about who we are and should be accountable to in making our methodological choices (Kenney, 2015). For example, the creation of the local Steering Committee to oversee our research design helped to strengthen our accountability to Iñupiat elders in the partnering village. Crucially, building our abilities to practice reflexivity and accountability has helped us to recognise that “[no] one set of research questions, methods, theories or approaches are automatically on side of justice” (Schwartz-Shea, 2006: p. 329).

Deliberation

Deliberation refers to the ability to engage in dialogue with “those who think otherwise,” in good faith and with an openness about one’s own commitments and willingness to have them challenged—while also seeking points of connection and common ground where possible (Verran, 2014: p. 527; Li and Wagenaar, 2019). Through our deliberation in the New Normal project we have come to the realisation that—despite sizeable methodological differences—we share a general critical-reflexive sensibility or orientation towards our methodological practices. Much of the methodological literature would suggest that such an orientation is the preserve of interpretive approaches, but we found that it was possible to apply a supposedly interpretive critical-reflexive sensibility (concerned with the contigent meaning-making of researchers and participants) to the construction and interpretation of controlled behavioural experiments (see also, e.g., Abbott, 2004; Macknight and Medvecky, 2021). Moreover, we have found that we also share a broad political positioning in our research: we both aim to challenge what we perceive as simplistic and often (socially and ecologically) harmful assumptions about human behaviour that percolate in mainstream science, policy and management. These shared commitments do not guarantee ‘plain sailing’ or the absence of tensions and conflict, but they have provided common ground from which to negotiate our ethical-political dilemmas.

Practical wisdom

Practical wisdom or judgement refers to the ability to negotiate situations of multiple and contested values, and come to decisions or ways forward that contribute well to the situation and set of relationships involved in a particular course of action (Schwartz and Sharpe, 2010). As Flyvbjerg (2001: p. 55) notes when discussing the origins of practical wisdom in the Aristotelian concept of phronesis, it is “mainly talking about ethics in relation to social and political praxis, that it, the relationship you have to society when you act” [italics in original]. Practical wisdom therefore involves bringing our previously noted skills of reflexivity and deliberation to bear on the practical necessity of acting in the situation at hand (Fischer, 2007; Cameron, 2018). For example, ethical clearance needs to be obtained before fieldwork, and so we had to at least provisionally resolve our ethical dilemmas one way or another by a particular date (without the benefit of hindsight or perfect information).

Negotiating the ethical-political dimensions of our methods

We drew on all of these practices and skills in our efforts to provisionally resolve our two ethical dilemmas prior to fieldwork. For the first dilemma, a priori vs. responsive research design, we decided to re-emphasise the ‘responsiveness to context’ that we had intended at the outset of the project because we felt that in this situation, given the history of extractive and colonial research practices in Indigenous communities, the ethical-political imperative was to ensure that our partnering village would be able to set the agenda for the research and the terms on which they would like to be engaged within the photovoice and experimental methods. In practice, this meant that in the ethics applications we described the broad types of experiments that we might undertake, and then explained the methodological and ethical reasons why we could not be more specific at this point.

For the second dilemma, anonymity vs. credit for knowledge contributions, we decided to proceed with the assumptions that data from the experiments would be collected in anonymous form in order to reduce the ethical risks of making names public, and that post-experimental interviews would be credited but not linked to specific decisions in the experiment. However, because we did not yet know the precise form the experiments and post-experiment interviews would take (given our emphasis on a responsive design), we decided to discuss the appropriate route to take with our local Steering Committee. In practice, then, this required us to defer the decision, and to explain our (ethical-political) reasons for doing so in the ethics application.

It is important to note that we do not describe these choices here because we believe them to be particularly good or ethical per se—the very nature of the ethical-political dimensions of research means that others may have made different choices, and that our own may be legitimately contested. Rather, the point is to demonstrate the practices and skills that we have drawn on in our efforts to better grapple with the ethical-political dimensions of mixing methods in transdisciplinary research and to come to more explicitly reasoned and careful ways forward.

Evaluating and strengthening the ethical-political rigour of mixed methods, inter- and transdisciplinary, and co-production research

Given that we are suggesting that the ability to recognise and negotiate the ethical-political dimensions of research methods should be considered a key competency in mixed methods, inter- and transdisciplinary, and co-production research, it follows that navigating these dimensions well should be considered an indicator of research quality. How might evaluators of mixed methods, inter- and transdisciplinary, and co-production research projects, graduate training programmes and research organisations assess the rigour with which the ethical-political dimensions of research methods are being addressed, and how might this rigour be strengthened? In this section, we provide some initial pointers for evaluating and strengthening this competency.

Negotiating the ethical-political dimensions of mixing methods is a continuous process, so evaluators ought to be looking at patterns of interaction and practices over the lifespan of a given initiative. For example, we have highlighted the value of reading groups in philosophy, ethics and social studies of science, personal video diaries, and facilitated dialogues, but there will be many other possible practices, including more unconventional practices like role-plays where researchers can adopt and act-out alternative perspectives (Boyd et al., 2015; Haider et al., 2018). Such practices echo the suggestions made by Lang et al. (2012: p. 34) for the transdisciplinary project design principle “mitigate conflict constellations,” including reflexive meetings, open discussion forums and mediated negotiations. Yet the simple existence of such practices is not enough because, as we have shown, the ethical-political dimensions of research are often implicit. Inspired by other practical efforts to surface implicit methodological commitments (including Eigenbrode et al., 2007; Hogan and Topkok, 2015; Hertz and Mancilla-Garcia, 2019, and Simon’s participation in the course ‘Indigenous Knowledges and Epistemologies’ at Charles Darwin University, Australia) we suggest that evaluators might look for the use of prompts that attempt to make ethical-political dimensions explicit. We provide the following examples intended to prompt reflection on the relations between ontology, epistemology, axiology and the social effects of research methods:

  • Ontology: What does ‘reality’ mean to you? Can you give an example of something that is ‘real’ and something that is not? What are some characteristics of the phenomena you wish to investigate? Do you know of or can you imagine other possible answers to these questions among your research group?

  • Epistemology: How is it possible to gain reliable knowledge about the phenomena you wish to investigate? How do you engage with research participants? What do you ask your participants to do in your research, and why is this considered valuable/necessary? Can you imagine other possible answers to these questions among your research group?

  • Axiology: What does ‘ethical practice’ mean to you? What ethical issues and questions do you encounter in your research? Can you imagine other possible answers to these questions among your research group?

  • Social effects: How do you think the knowledge produced through your methods might contribute to society? What audiences do you hope to reach with your research? How would you describe your own personal political beliefs and values, and how might these relate to your research methods and methodological approach? Can you imagine other possible answers to these questions among your research group?

The ability of teams to collaboratively reflect and reason around these kinds of questions might be assessed through activities such as periodic focus groups, interviews and diaries. Over the course of a project, researchers should improve their abilities to:

  • Articulate some of the ontological, epistemological and axiological assumptions contained within the methods used, and discuss how these differ to other possible options and why these are (in)appropriate for their particular study context.

  • Reflect on the methodological choices that have been made throughout the project, in terms of their effects on participants and the broader (social) contexts of the research, and how these might have been improved or done differently.

  • Reflect on some of the ethical-political issues raised over the course of the project, both in relation to the immediate empirical context of the research and in society more generally.

The rigour with which a team has negotiated the ethical-political dimensions of mixed method research cannot be completely assessed in terms of pre-determined tick boxes, but rather in terms of the cultivation of certain sensibilities or orientations to knowledge production that should perculate throughout a project. This might be assessed by looking for the presence of guiding logics in research processes and outputs, for example the “agonistic-antagonistic” logic described by Barry and Born (2013: p. 12) in which assumptions are continually made explicit, discussed, and potentially transformed.

The responsibilities for nurturing greater ethical-political rigour in mixing methods should be distributed across project teams, graduate training programmes and research organisations. We are well aware from our own experiences of the many challenges faced by graduate students and researchers seeking to learn how to engage productively in inter- and transdisciplinary research, and do not seek to simply load additional responsibilities onto individuals. The practices, prompts and evaluative criteria we provide above can be integrated into foundational methods courses offered in graduate training programmes (e.g., Hogan and Topkok, 2015). For example, in the context of transdisciplinary sustainability research recent PhD courses have aimed to equip students with abilities to negotiate the ethical-political dimensions of engaging in transformative research, mirroring the themes raised in our prompts above (Student, 2022). Research organisations might further strengthen the ethical-political rigour of mixed methods research by, for example, hosting regular seminars where researchers and research groups are encouraged to share and discuss the ethical-political dilemmas they are facing, or by holding facilitated reflective workshops during annual staff retreats (e.g., Hubbs et al., 2021). Strengthening ethical-political rigour ought to be seen as part of the core business of mixed methods, inter- and transdisciplinary, and co-production research aimed at solving societal challenges.

Conclusion

In this paper we have highlighted that understandings of research methods are changing: away from the perception that methods are neutral, technical tools, and towards recognition that methods are shaped by inherently contestable assumptions about the nature of reality, knowledge, and good or ethical practice, and furthermore that methods actively participate in the realities they describe. These shifting understandings highlight the unavoidably ethical-political dimensions of research methods: that the methodological practices that guide the interactions between researchers and participants are inextricably linked to the variable social effects and realities that the methods help to bring about. We have argued that these ethical-political dimensions are important for all researchers to recognise and be prepared to negotiate, but particularly so for those engaged in mixed methods, inter- and transdisciplinary, and co-production research. On the one hand, such researchers are more likely to encounter ethical-political differences among collaborators and therefore need to be prepared to recognise and negotiate these differences, and on the other such researchers often explicitly aim to bring about social change and transformation and need to be able to think carefully about the ethical-political consequences of their research.

We have drawn on our own experiences to suggest that the ability to negotiate the ethical-political dimensions of research methods should be considered a key competency of mixed methods, inter- and transdisciplinary, and co-production research. We have provided some initial ideas for practices, skills, and evaluative criteria that can support the development of this competency, and direct the reader to a growing literature on these and related aspects for more ideas (e.g., Haider et al., 2018; Montana et al., 2020; Hubbs et al., 2021; Chambers et al., 2022). While the language of “navigating” or “negotiating” the ethical-political dimensions may seem at first glance to imply an ability to somehow rise above or transcend the particularities of any one set of commitments towards a consensus or integrative approach, we reiterate that any such integrative outcome will carry its own unavoidable and contestable ethical-political dimensions. Learning to negotiate the ethical-political dimensions of research methods entails recognising that you will never be free of them—they are an inescapable condition of producing knowledge in society, and require an ongoing commitment to asking the question: “what kind of world are my methodological practices (not) contributing to?”