The challenge of making moral machines

In the waning days of 2020, Timnit Gebru, an artificial intelligence (AI) ethicist at Google, submitted a draft of an academic paper to her employer. Gebru and her collaborators had analysed natural language processing (NLP), and specifically the data-intensive approach of training NLP artificial intelligences (AIs). Such AIs can accurately interpret documents produced by humans, and respond naturally to human commands or queries.

In their study, the team found the process of training a NLP AI requires immense resources and creates a considerable risk of embedding significant bias into the AI. That bias can lead to inappropriate or even harmful responses. Google was skeptical of the paper’s conclusions, and was displeased that Gebru had submitted it to a prominent conference. The company asked Gebru either to retract the paper or remove any mention of Google affiliations. Gebru refused the terms. Within a day, she learned that she no longer had a job.

Gebru’s sudden ouster raised serious questions about the transparency, accountability and safety of AI development, particularly in private companies. It also crystalized concerns about AI algorithms that had been bubbling along for years.

Whether embedded in a natural-language processor or a medical diagnostic, AI algorithms can carry unintentional biases, and those biases can have real-world consequences. The manipulation of the Facebook algorithm to impact the 2016 United States presidential election is one frequently cited example. As another, Aimee van Wynsberghe, an AI ethicist at the University of Bonn in Germany, cites an abortive effort by Amazon to use an AI-based recruiting tool. The tool, which was tested between 2014 and 2017, drew the wrong lessons from the company’s past hiring patterns.

“When they put it in practice, they found that the algorithm would not select women for the higher-level positions, only for lower-level ones,” says van Wynsberghe.

Yet the development of AI continues to accelerate. The market for AI software is expected to reach US$63 billion in 2022, according to Gartner Research, and that is on top of 20% growth in 2021. Already commonplace in online tools such as recommendation or optimization engines and translation services, higher impact AI applications are on the horizon, particularly in large sectors like energy, include those in transportation, healthcare, manufacturing, drug development and sustainability.

Given the size and number of opportunities, the enthusiasm for AI solutions can obscure risks associated with them. As Gebru found, AIs have the potential to cause real harm. If humans can’t trust the very machines meant to help them, the true promise of the technology may never be fulfilled.

Smarter by the day

Although many AIs are programmed directly by humans, most modern implementations are built on artificial neural networks. The algorithms analyse data to identify and extract patterns, essentially ‘learning’ about the world as they go. The interpretations of these data guide the next step of analysis, or inform decisions made by the algorithm.

Artificial neural networks analyse data collaboratively in a manner roughly analogous to the neurons in the human brain, explains Jürgen Schmidhuber, director of KAUST in Saudi Arabia. He developed a foundational neural network framework known as ‘long short-term memory’ (LSTM) in the late 1990s.

“In the beginning, the learning machine knows nothing – all the connections are random,” he says. “But then over time, it makes some of the connections stronger and some of them weaker, until the whole thing can do interesting things.”

Nature Outlook: Robotics and artificial intelligence

Such training is a characteristic of LSTM and other approaches to neural networks, and it’s a reason those AIs have become so popular. An AI that learns to learn has the potential to develop novel solutions to extremely difficult problems. The FII Institute THINK initiative, for example, is pursuing a multi-pronged roadmap for AI development to explore healthcare applications such as drug discovery and epidemic control, as well as sustainability-oriented efforts to monitor and protect forest and marine ecosystems – all of which lend themselves to AI applications.

But training can build bad habits as easily as good ones. As Gebru found with NLP AIs, very large and improperly curated data sets can amplify rather than rectify human biases in an AI’s decision-making process. Sandra Wachter, a researcher specializing in data ethics at the University of Oxford in the United Kingdom, highlights the example of diagnostic software tools designed to detect signs of skin cancer through image analysis, which fare poorly on black- or brown-skinned individuals because they were primarily trained on data from Caucasian patients. “It might be misdiagnosing you in a way that could actually have harmful consequences for your health and might even be lethal,” she says.

Similar training data problems have plagued IBM’s AI-driven Watson Health platform, and the company recently moved to divest itself of this technology after years of struggling with poor diagnostic performance and ill-advised treatment recommendations.

Such cases beg the question: Who is to blame when an algorithm does not work as designed? Answers may be easy to reach when an AI’s conclusions are objectively wrong, as in certain medical diagnostics. But other situations are much more ambiguous.

For years, Facebook enabled companies to target their advertising based on algorithmically derived information that allowed the platform to infer a user’s race, an option now discontinued. “Black people wouldn't be able to see certain job advertisements, or advertisements for housing or financial services, for example,” says Wachter. “But those people didn’t know about it.”

The victims of discrimination might have a claim in the courts after the fact. But the best solution is to pre-empt the introduction of destructive bias in the first place with ethical AI design.

Rules for robots

The idea of imbuing machines with ethics is not new. Author Isaac Asimov penned his Three Laws of Robotics when thinking of androids more than 75 years ago, and all three of his laws raise ethical considerations. In the research labs around the world, science fiction is now edging towards reality as researchers grapple with how to embed ethics into AI.

Current work entails identifying sets of internal guidelines that would be compatible with human laws, norms, and moral expectations, and could serve to keep AIs from making harmful or otherwise inappropriate decisions. Van Wynsberghe pushes back against the idea of calling such AI systems ‘ethical machines’ per se. “It’s like a sophisticated toaster,” she says. “This is about embedding ethics into the procedure of making the machines.”

In 2018, the Institute of Electrical and Electronics Engineers (IEEE), a non-profit organization headquartered in New York City, US, convened an interdisciplinary group of hundreds of experts from around the world to hash out some of the core principles underlying ‘ethically aligned design’ for AI systems. Bertram Malle, a cognitive scientist specializing in human-robot interaction at Brown University in Providence, Rhode Island, US, who co-chaired one of the effort’s working groups, says, “We can’t just build robots that are ‘ethical’ – you have to ask ethical for whom, where and when.” Accordingly, the ethical framework for any given AI, Malle says, should be developed with close input from the communities of people with which they will ultimately be interacting.

A recent law review article from Wachter’s team highlighted some of this complexity. After assessing a variety of metrics designed to assess the level of bias in an AI system, her team determined that 13 out of 20 failed to meet the legal guidelines of the European Union’s non-discrimination law.

“One of the explanations is because the majority, if not all, of those bias tests were developed in the US… under North American assumptions,” she says. This work was conducted in collaboration with Amazon, and the company has subsequently adopted an improved bias-testing system based on the open-source toolkit that resulted from the study.

A trustworthy AI system also requires a measure of transparency, where users can get a clear sense of how an algorithm arrived at a particular decision or outcome. This can be tricky, given the ‘black box’ complexity and proprietary nature of many AI systems, but is not an insurmountable problem. “Building systems that are completely transparent is both unrealistic and unnecessary,” says Malle. “We need to have systems that can answer the kinds of questions that humans have.”

That has been another priority for Wachter’s team, which uses a strategy called ‘counterfactual explanation’ to probe AI systems with different inputs in order to determine which factors lead to which outcomes. She cites the example of interrogating diagnostic software with different metabolic parameters to understand how the algorithm determines that a patient has diabetes.

Ethics for all

If embedding ethics and transparency into AI is a difficult problem, the ethical and transparent development of AI, by humans, could be even more challenging. Private companies like Google, Facebook, Baidu and Tesla account for a large portion of overall AI development, while new start-ups seem to emerge on a weekly basis. Ethical oversight in such settings can vary considerably.

“We see glimmers of hope, where [companies] have hired their own ethicists,” van Wysnberghe says. “The problem is that they’re not transparent about what the ethicists are doing, what they’re learning – it’s all behind non-disclosure agreements.” The firing of Gebru and other ethicists highlights the precariousness of allowing companies to police themselves.

But there are potential solutions. To overcome the opacity of private AI development, for example, van Wynsberghe advocates the notion that companies could collectively sponsor an independent ethical review organization to act analogously to the institutional review boards that supervise clinical trials. In this approach, corporations would collectively fund a board of ethicists to take on rotating ‘shifts’ at the companies to oversee work. “So you’d have this kind of flow of information and shared experiences and whatnot, and the ethicists are not dependent on the company for their paycheck,” she says. “Otherwise, they’re scared to speak up.”

New legal frameworks could help as well, and Wachter believes that many companies are likely to welcome some guidance rather than operating in an environment of uncertainty and risk. “Now examples are being put on the table that concretely tell them what it means to be accountable, what it means to be bias-free, and what it means to protect privacy,” she says.

The European Union currently leads the way, with an ‘AI Act’ that provides a detailed framework for the risk-based assessment of where AI systems can be deployed safely and ethically. China is also implementing strict regulations designed to prevent AI-based exploitation of or discrimination against users – although these same regulations could also provide a vehicle for further censorship of online speech.

Above all, automation should not be seen as a universal solution and the collective good, for all humans not just AI developers, should always be a consideration. Malle favours a focus on systems that complement rather than replace human expertise in areas such as education, healthcare and social services. For example, AI could help overextended teachers to get a better handle on students who need more individual attention or are struggling in particular areas of the curriculum. Or AI could take care of routine tasks in the hospital ward, so that nurses can better focus on the specific needs of their patients.

The goal should be to amplify what can be achieved with available human intellect, expertise and judgement – not to take those out of the equation altogether. “I really see opportunities in the domains where we really don’t have enough humans or not enough trained humans,” Malle says. “Let’s think about domains of need first.”