Close-up of a person writing code on a laptop.

AI tools are competing with humans to write code.Credit: Getty

Artificial intelligence (AI) researchers have been impressed by the skills of AlphaCode, an AI system that can often compete with humans at solving simple computer-science problems. Google sister company DeepMind, an AI powerhouse based in London, released the tool in February and has now published its results in Science1, showing that AlphaCode beat about half of humans at code competitions.

And in the past week or so, social-media users have been mesmerized by the ability of another chatbot, called ChatGPT, to produce occasionally meaningful-sounding (and sometimes sublimely ridiculous) mini-essays on request — including short computer programs. But these state-of-the-art AIs can perform only rather limited tasks, and researchers say they are far from being able to replace human programmers.

ChatGPT, the latest version of a natural-language system by software company OpenAI of San Francisco, California, was released on 30 November. Both ChatGPT and AlphaCode are ‘large language models’ — systems based on neural networks that learn to perform a task by digesting massive amounts of existing human-generated text. In fact, the two systems use “virtually the same architecture”, says Zico Kolter, a computer scientist at Carnegie Mellon University in Pittsburgh, Pennsylvania. “And while there are of course minor differences in the training and execution, the main difference, if there is any, is that they are simply trained upon different data sets, and thus for different tasks.”

Whereas ChatGPT is a general-purpose conversation engine, AlphaCode is more specialized: it was trained exclusively on how humans answered questions from software-writing contests. “AlphaCode was designed and trained specifically for competitive programming, not for software engineering,” David Choi, a research engineer at DeepMind and a co-author of the Science paper, told Nature in an e-mail.

Human needs

Researchers have pointed out that much of the work that goes into a large software-engineering project — say, designing a web browser — involves understanding the needs of humans who are going to use it. These are difficult to describe with the simple, machine-readable specifications that an AI can use to produce code.

Are your students using AI to write papers? Take Nature’s poll

Nature wants to discover the extent of concerns about the impact of artificial-intelligence tools on education and research integrity, and how research institutions are dealing with them. Take our poll here.

Kolter says that it’s unclear whether it will ever be possible for machines to generate large-scale software systems from scratch. But “my best guess is that tools like these that can generate portions of a program will likely become ‘second-nature’ kind of tools for programmers”, he says.

“We hope that further research will result in tools to enhance programmer productivity and bring us closer to a problem-solving AI,” Choi says.

Kolter adds that there are already some AI tools good enough to make programmers’ jobs easier, such as one called Copilot, a code-autocompletion service introduced last year by code repository GitHub and based on OpenAI technology.