Adding to the list of environmental challenges facing agriculture, COVID-19 and the demographics of age, migration and urbanization pose a serious threat to the sustainability of farm businesses and food security1. In particular, farm businesses across the world are struggling to fill vacancies and provide safe working conditions for labourers.

Autonomous robots could help address these immediate challenges2. Whilst their physical manifestation comprises hardware, such as a vehicle combined with manipulators, their autonomy is derived from sophisticated algorithms rooted in artificial intelligence. These algorithms fuse sensor data to enable control and real-time decision support. Autonomous robots can perform tasks collaboratively with humans (so-called co-bots) or on their own3. Apart from isolated on-farm examples, autonomous platforms with robotic mobility that fuse multiple technologies across a single fleet (for example, crop forecasting, planting, harvesting and packing) are not yet fully implementable and face substantial barriers. However, there is already adoption of static robotic milking technologies in the dairy sector and in-field deployment of tractor-mounted robotic manipulators to remove weeds and protect crops from pests and diseases2.

We know, however, that the history of agricultural innovation is littered with failure and slow adoption, and there are legal, ethical and social concerns associated with autonomous agriculture4,5. Potential challenges, opportunities and consequences of autonomous agriculture (Fig. 1) are interlinked and depend on how technologies are designed and implemented. Many of these aspects have been discussed in the burgeoning literature on the social and ethical impacts of digitalization in agriculture6,7. Empirical research remains limited for autonomous farming robotics; potential issues have largely been extrapolated from empirical research on smart farming technologies in general or the use of autonomous robots in other workplaces. Here, we identify examples of responsible innovation principles being implemented and indicate where more needs to be done.

Fig. 1: Overview of challenges, opportunities and potential consequences of autonomous agriculture.
figure 1

Positive (+), negative (–) and uncertain (+/–) consequences are indicated. Positive consequences denote opportunities to be harnessed, whereas negative consequences denote challenges to be overcome concerning the operationalization, adoption and/or deployment of innovations (see refs. 4,5 for more detail).

Full size image

Responsible innovation in agriculture and beyond

The most widely used framework for responsible innovation was proposed by Stilgoe and colleagues8 and involves four key components: anticipating the impacts of innovation; reflecting on one’s work and adapting accordingly (reflexivity); including a wide range of stakeholders in the design process; and responding to stakeholders’ concerns, ideas and knowledge by constructing appropriate institutional structures.

Guidance on responsible innovation — provided by funders such as the Engineering and Physical Sciences Research Council9, InnovateUK10 and the European Commission — encourages companies to be cognisant of their responsibility and committed to responsible research and innovation principles, by exploring the challenges that could arise from innovation and acting on their findings in a transparent, inclusive and timely manner. Despite frequent calls for companies to conduct a transparent and iterative process of responsible innovation, there is a lack of commitment to, or reporting of, the steps taken in technology development in the agriculture industry.

In the following sections we discuss how the four key components mentioned above can be operationalized to guide technology development in agriculture11, outlining key research needs. The examples referenced herein alongside the guidance from Stilgoe et al.8 and Eastwood et al.11 provide a good overview of techniques that can be used to apply responsible innovation principles.

Anticipation

With the objective of minimizing negative, unintended outcomes8, ‘anticipation’ involves identifying, predicting and exploring the potential short- and long-term consequences of future innovation across society and is therefore essential for the responsible development of autonomous robots.

Very little empirical anticipatory work for autonomous robots in farming has included a variety of stakeholders in the process, though a recent paper by Legun and Burch12 begins to describe a process of co-design in the context of robotic apple orchards in New Zealand. Empirical studies have otherwise been limited to the narrow use of foresight exercises in the form of technology use and acceptance surveys and farmer13 or public opinion surveys14 using online questionnaires and short interviews. Foresight is also used to elucidate future benefits and challenges associated with combining a technology with other methods, such as the Delphi technique (which relies on anonymous rounds of voting)15. Other anticipatory processes include ‘horizon scanning’ (scanning data sources to detect early developments16) and ‘socio-literary techniques’ (using science fiction as a tool to encourage dialogue about technology futures17, possibly through ‘Ag-Tech movie nights’18). A typical methodology in robotics and human–robotic interaction are ‘Wizard of Oz’ studies19, where autonomy is ‘fake’; robots are usually remote-controlled, anticipating the abilities they may have once fully implemented. Video studies are also often employed20, where participants are presented with recordings of robot behaviour and assess it from a third-person perspective.

One further method to consider is backcasting, which involves building an (ideal) future scenario, and working backwards to identify the steps needed to get to there. This is done in anticipatory governance approaches, for example. A key area for future research will be to use different anticipatory methods with diverse stakeholders specifically on the subject of autonomous robots in agriculture. Those included in the process of anticipation should be those directly affected by the adoption of robotics, including farmers, farm workers and consumers of food produced in that way. Including such a wide range of stakeholders will create a number of practical challenges related to power inequality (for example, farm managers versus farm workers) and language barriers (for example, migrant farm workers) and these will need to be managed sensitively by trained facilitators.

Reflexivity

Reflexivity entails “holding a mirror up to one’s own activities, commitments and assumptions, being aware of the limits of knowledge, and being mindful that a particular framing of an issue may not be universally held”8. Constant analysis and critique of one’s work among peers is an embedded practice of rigorous science. However, scientists and engineers typically carry out reflexivity and other responsible innovation practices behind closed doors, in the lab, and do not recognize these processes as reflexivity in responsible innovation terms21. Opening these conversations up to the public and acknowledging and listening to other actors can improve the quality of reflexivity.

Reflexivity in the realm of autonomous robots in agriculture has mostly come from the user-centred design process. Work to date in this space has recognized that robotic systems interacting with humans need to undergo an iterative development approach22, bringing together subjective user experience with actual system logs. After including stakeholders and seeking their information requirements and preferences for autonomous robots through surveys23, workshops24 and field experiments25,26,27, designers have altered prototypes and design paths to ensure that the robots work for the user. Yet, this is narrow reflexivity; it involves developers tweaking design based on user feedback, rather than conducting a fundamental analysis of the assumptions and values underlying the proposed solution or questioning if agricultural robotics is really the path we want to take as a society. We rarely carry out a deeper form of reflexivity, possibly missing alternative solutions.

The development of and engagement with best practice guidelines, codes of conduct and international standards is another form of reflexivity that can guide industry to conduct innovation in a responsible manner, although it is not always clear whether they continue to serve the purpose of reflexivity once adopted. In Australia, a code of practice for agricultural mobile field machinery with autonomous functions is currently under development to help guide safe working procedures in the field; this code of practice is intended to hold some legal weight. International standards for the use of autonomous robots such as ISO 10218 provide norms for worker safety when collaborating with robots in a structured, industrial environment. In ISO 10218, safety aspects such as tactile and pressure sensors, safe maximum speed, proximity sensors, human detection cameras, and emergency stop are described to ensure the safety of human–robot collaboration. Other relevant international standards include: ISO 18497 (design principles for safety with highly automated agricultural machines — operations of robots in-field are not covered); ISO 17757 (for use of autonomous machinery in mining); and ISO/SAE DIS 21434, currently under development (for cybersecurity in road vehicles). The agricultural industry can glean insights from these standards, however there is a necessity to further develop agriculture-specific standards and codes of practice that account for human–robot collaboration in flexible, unstructured environments such as in the field. Understanding how this might be done effectively, bringing together relevant stakeholders, is an important future area for research.

Inclusion

Concepts of inclusion are frequently limited to the ‘consideration’ of how stakeholders may be impacted or react to innovation by a limited group of experts28. Genuine inclusion should involve the participation of a full range of stakeholders. If we do not pursue methods for the substantive inclusion of a full range of actors, not just the usual suspects, and do not give due attention to power inequalities between stakeholders throughout the participatory process, then we risk reinforcing unequal participation under the guise of inclusivity. It may appear that increased participation from the start is time-consuming and resource-intensive, but user-centred design can prevent problems further down the line.

Within the development of autonomous robots in agriculture, inclusion has mostly taken the form of consultation and sometimes collaboration, involving feedback from farmers and farm workers on the technical side of robot development. Simulation experiments29,30 and field-based workshops23 have allowed farmers and farm workers to test the usability of a technology. Researchers have used task scenarios, observations, and participant feedback to feed into prototype development. The social sciences have developed a number of participatory methods that allow substantive inclusion, such as citizen juries and deliberative workshops, and a greater selection of these should be brought to bear for inclusion surrounding autonomous agriculture31.

Stakeholders identified in the PAS 440 Responsible Innovation framework developed for InnovateUK10 include co-developers; markets, customers and end-users; regulators and standards bodies; NGOs representing civil society stakeholders; and individual citizens likely to be affected. Beyond the usual suspects, it is important to engage with harder-to-reach stakeholders. Schillo and Robinson28 discuss the importance of engaging with historically marginalized groups. In the case of autonomous agriculture, this could involve small farmers (who may be pushed out of the industry by larger farmers with more capacity to adopt and adapt), organic farmers (whose farming strategy may be more difficult to align with autonomous robots focused on precision fertilization32), as well as farm workers (who could lose jobs as they are replaced by robots). Blok33 argues that stakeholder inclusion and participation can typically become reductive as it focuses on the cognitive approach to understanding the perspectives of stakeholders in a self-serving ‘immunization strategy’, where the goal is to convince others, prevent criticism and portray the company as having good intentions. We should ultimately ensure that we are undertaking substantive, rather than tokenistic inclusion.

The involvement of stakeholders should not be restricted to the exploration of consequences in terms of economic opportunity or technology acceptance, but include wider implications and society’s ‘grand challenges’. To date there are limited examples of this work: Pfeiffer et al.14 explored public opinion of digital farming technology through surveys and spontaneous associations; Kester et al.13 surveyed farmers’ views of the future of automation on topics such as perceived value, applications and expectations; and Baxter et al.26 asked fruit pickers questions regarding the impact of autonomous robots on their job security.

Responsiveness

Identifying potential consequences, reflecting on underlying assumptions, values and problem-solving processes, and including stakeholders in the innovation process can only lead to responsible innovation if newly gained insights are acted on. Actors should be reactive to new knowledge and ensure development is iterative. This could be in the form of adapting R&D projects or early design prototypes based on feedback from stakeholders. Other actions that result from new information could include adjusting business models, altering control or access to software, amending workers contracts and working conditions3, or refraining from developing a certain robot altogether if it is not desired by society.

Responsiveness is also important within institutional structures, which should respond promptly to new information, in policy, law and regulatory environment. Regulation can restrict innovation (for example, genetically modified crops in Europe), efficiency and competitive advantage, however legal structures will be important to ensure protection for users of autonomous robots and for clarifying the liability framework. Hence, regulation can act as both a barrier to and an enabler of adoption. Basu et al.5 describe the current legal frameworks, regulations and standards that are relevant to the development of autonomous robots in agriculture, as well as the gaps in areas such as data protection law, ethics of robot autonomy and artificial intelligence. Similarly to how the European Union embedded ‘privacy by design’ into its General Data Protection Regulation, others are calling for ‘equality by design’ in artificial intelligence regulation to safeguard against bias and discrimination that may inadvertently be engrained in technology and machine learning34. There are examples of ‘technological redlining’ as well as technological limitations of measurement such as unequal object detection or lower quality heart rate measurement for those with darker skin34. A lack of transparency with algorithms, machine learning and artificial intelligence — the ‘black box problem’ — can lead to bias and discrimination issues within machine learning becoming further entrenched and replicated. Regulatory oversight of equality by design34 is key to ensure that programmers address any bias and discrimination that may be produced in algorithms, ultimately ensuring that technology treats users fairly.

Conclusion

Addressing the social, legal and ethical implications of autonomous robots is arguably a greater challenge than the development of the technology itself. More research is needed to ensure that anticipation, reflexivity and inclusion efforts are turned into responsive action on the ground. As highlighted in this paper, most empirical work for the development of autonomous agriculture has been focused on the technical aspects of robot operation with some level of inclusion and reflexivity to ensure improvement of technical performance. Little published work has gone beyond this to use methods that allow for substantive inclusion and deeper reflexivity on the subject. Yet, if society decides that autonomous robotics for farming is the way to go, then practising responsible innovation in their development is vitally important to prevent future controversy, implementation delays and negative consequences. Ultimately, the success or failure of autonomous robots in agriculture will not rest on the limits of our technical enterprise, but on our ability to involve society, learn from it and respond appropriately.