Introduction

Technological progress has been exponential in modern times: Powered flight progressed from the Wright Brothers’ Flyer to the Apollo Moon Landing in less than 70 years, and Moore’s Law recently celebrated its 50th anniversary of continued doubling of computing technology every 2 years. Advances in materials have enabled much of our technological progress,1 but the process of materials development itself remains slow, often 20 years from invention to use.2 One reason is that current methods of research are labour intensive. Humans drive most aspects of research, from initial hypothesis generation to experimental design, analysis, interpretation and iterative hypothesis refinement.

Many materials problems are complex and high dimensional. The number of experiments needed to capture the relevant phenomena is often overwhelmingly large and slows research. To address this, automation has been applied to research processes via high-throughput and combinatorial techniques in the fields of life sciences3–7 and materials research.8–16 In addition to automation, computation is being exploited to speed research. The Materials Genome Initiative17 and related Integrated Computational Materials Science and Engineering efforts18 have highlighted the need for computational approaches to more effectively make use of the explosion in data generation. However, neither automation nor Integrated Computational Materials Science and Engineering addresses the iterative nature of the research process. Each new set of experiments brings new knowledge to incorporate into the design of subsequent experiments. Here the human involvement in analysis and experimental design for high-throughput approaches remains a bottleneck.3 The solution to these problems is to team human researchers with autonomous research robots that can capture and incorporate new experimental knowledge in an iterative learning loop, enabling us to traverse complex experimental parameter spaces.19

Autonomy is a topic of great current interest, and is an extension of automation; an autonomous system is capable of changing its behaviour in response to a changing environment. Traditional examples include driverless cars,20 spacecraft21 and unmanned aerial vehicles.22,23 In autonomous systems, models of the environment are built up and decisions are exercised based on sensory data. In the context of scientific research, autonomy can be applied to systems where available instrumentation is employed as sensory input. Autonomy has already impacted the life sciences, exemplified by excellent work in the study of gene functions24–26 and drug discovery.27,28 However, autonomy has yet to deeply impact materials discovery and research. Here we demonstrate our Autonomous Research System (ARES), which we believe is the first of its kind to combine robotics, artificial intelligence (AI), data science and in situ techniques towards the vastly complex realm of materials development. In particular, we applied ARES towards the synthesis of single-walled carbon nanotubes (CNTs).

CNTs have unparalleled properties, combining strength and stiffness with extreme electrical and thermal conductivity. These properties are being exploited in nanoelectronic and plasmonic devices, lightweight conductive cables, composites and ballistic materials, as well as biological and chemical sensors.29,30 But the synthesis of CNTs remains poorly controlled and poorly understood, despite two decades of intensive effort. The defective, impure material that generally results from typical growth methods also lacks the diameter and helicity control needed for electronics, and the poor yield and product variability increases cost and impedes transition of this important material. One of the biggest challenges to controlling growth is the overwhelmingly large experimental parameter space (e.g., temperature, pressure, gas composition). Here we report the successful demonstration of autonomy applied to the experimental study of CNT synthesis, wherein ARES autonomously traversed high-dimensional parameter space, varying multiple growth parameters between each experiment with the goal of learning to control CNT growth rate. This work paves the way for ARES to be expanded to other materials and processes, and signifies a fundamental change in the process of research by teaming human and robot researchers to attack vastly more complex and potentially impactful problems.

Results

With ARES, we successfully built and demonstrated an autonomous research system capable of learning to grow CNTs by developing an AI planner and linking it to an automated growth reactor with in situ characterisation used as feedback control to iteratively define, conduct and analyse experiments. ARES continuously improved its ability to target growth rates over a series of hundreds of experiments. Each robotically controlled experiment consisted of synthesising CNTs by a chemical vapour deposition (CVD) process where the ARES AI planner supplied the growth conditions—temperature, pressure, and partial pressures of ethylene, hydrogen and water vapour.

A unique feature of ARES is the use of a laser to individually heat small (10 μm) pillars of silicon pre-seeded with catalyst15,31–34 for the CVD reaction (see experimental details and Supplementary Figure S1 for a schematic diagram of the instrumentation). We chose cobalt as the growth catalyst (2 nm film on a 10 nm thick alumina support), which was deposited uniformly onto the micro-pillars. Each pillar was thermally isolated on a wafer of thousands of pillars, making it in effect an independent micro-reactor. This enabled hundreds of experiments to be conducted in series by moving from one pillar to the next, with the ability to change all the experimental input conditions (i.e., temperature, pressure and gas composition). In addition, because of our unique configuration, ARES conducted its series of experiments without the need to exchange substrates from the growth chamber, thus enabling such high experimental throughput with minimal human intervention. Recently we demonstrated rapid automated experimentation with rates of up to 100 experiments per day (compared to 1 per day for conventional methods) for a series of 534 experiments in multi-dimensional parameter space.15

The ARES heating laser also served as the excitation source for Raman spectroscopy, enabling in situ acquisition of spectra during CNT growth. The characteristic Raman peak from CNTs, called the G band, corresponds to tangential vibrations of the carbon atoms, and its intensity is representative of the yield.31 ARES uses the increase in intensity of the G band with time during each experiment to determine the CNT growth rate, νexp (taken as the maximum experimentally observed growth rate, see Supplementary Figure S2). The experimental growth rate was used as the signal for the ARES feedback loop.

Our overall objective for ARES was to autonomously learn to control the growth rate of CNTs using AI and closed-loop feedback over many experimental iterations. And so, building on our previous work, we implemented an AI planner that proposed new experimental growth conditions based on an analysis of a database of prior experiments, iteratively improving its ability to predict growth rates. The AI experimental planner was comprised of a random forest model35 with growth conditions exercised through a genetic algorithm.36 This combination was chosen for its ability to capture nonlinear relationships between input and output variables within disjoint design spaces,37 which we deemed appropriate for the complexity of nanotube growth. It is currently implemented in a customised version of the Lockheed Martin Nanotechnology Material Data Mining, Modeling & Management (NMD-M3) software tool.38 Before the first autonomous experiments, an initial set of 84 experiments was conducted to provide a database of prior knowledge needed for the random forest planner to build its first model. These experiments were designed to span the growth parameter input space in a grid style and were executed in automated mode, i.e., with pre-planned conditions supplied by the user but executed without user intervention.

Once the initial database was established, ARES performed a series of more than 600 experiments in autonomous mode, where the AI planner generated input conditions for each experiment. For each experimental iteration the AI planner received an objective growth rate from the user. It then analysed the database of prior experiments and generated new experimental growth conditions expected to achieve a predicted growth rate, νpred, that targeted the user-supplied objective growth rate. Because the database incompletely spans the experimental parameter space, the predicted growth rate can differ from the objective, a known effect arising from domain applicability.39 After each experiment the database of prior experimental results was updated with the resultant experimental growth rate, and the planner refined its model representation to reflect the latest information. Over the course of the experimental campaign the user periodically modified the objective growth rate to cause ARES to probe a broader span of the experimental parameter space, which increased its domain coverage. To test for convergence between the predicted and experimentally achieved growth rates, series of experiments were grouped into tasks of 29–94 experiments. The objective growth rate was held constant for later tasks, but varied within some earlier tasks (see Supplementary Table S1).

The results of the convergence test are shown in Figure 1a, which compares the experimental growth rates to the growth rates predicted by ARES. Note that as ARES gained more experience, i.e., as the cumulative number of experiments increased, the spread between experimental and predicted growth rates became smaller, thus demonstrating experimental convergence. We quantified convergence by normalising the difference between experimentally obtained and predicted growth rates: Δ= ν exp − ν pred ν pred in Figure 1b. The mean values of Δ, μΔ and their s.d.s., σΔ, for each task enabled a statistical treatment of the data to analyse the trend towards convergence. As ARES learned to target growth rates, the mean difference from the prediction trended towards zero, implying that it successfully predicted the experimental growth rate. Moreover, the s.d. reduced to approximately 30%. To understand the context of these s.d. values, we conducted a series of experiments using the same input conditions, and analysed the statistical spread in the experimental growth rate. We found that the intrinsic variability in the system, which we termed the noise floor, over 20–30 experiments ranged from 20 to 30%, which is similar to the value found by Oliver et al. for their automated CNT growth furnace.14 By the end of the experimental campaign the variability in the experimental values matched the noise floor. Thus we conclude that ARES was able to target growth rates to the degree of variability intrinsic to our system. Hereafter we use ‘on-target’ to refer to experiments whose growth rates matched the predicted growth rates within the variability of the system.

Figure 1
figure 1

Demonstration of ARES learning to target CNT growth rates. (a) Experimental and predicted growth rates. (b) Mean (μΔ)and s.d. (σΔ) of the normalised difference between experimental and predicted growth rates. Initially the experimental rates were scattered far from the predicted (i.e., large σΔ and μΔ deviating from zero). As ARES learned over hundreds of experimental iterations the experimental rates converged (shaded area) to the predicted ones. This resulted in μΔ trending towards zero, and σΔ trending towards 30%, the noise floor of the system.

Full size image

Having achieved experimental convergence we endeavoured to analyse the results with ex situ characterisation and data mining. We extracted a subset of experiments that achieved the targeted growth rate. Scanning electron microscopy imaging after growth confirmed that experimental growth rates measured in situ were commensurate with CNT yield. Figure 2 shows the results of growth experiments that achieved on-target rates of 500, 3,000 and 16,000 s−1, and demonstrates that the density of CNTs in the scanning electron microscope images is proportional to the experimentally observed growth rates.

Figure 2
figure 2

SEM images demonstrating correlation between growth rate and yield of CNTs. On-target experiments with growth rates near the predicted ones of (a) 500, (b) 3,000 and (c) 16,000 (s−1). The time over which the nanotubes grew was approximately the same. The amount of nanotubes in the images increased in proportion to the growth rate. Scale bars: 500 nm.

Full size image

We then analysed the progression of experimental conditions selected by ARES that led to growth rate convergence. In each iteration, the initial sampling of growth conditions from the genetic algorithm was inherently stochastic, yielding a ranked list of suggested experiments where predictions closest to the objective were prioritised. In order to avoid proposing the same experiment repeatedly, suggestions from using the genetic algorithm were further filtered based on proximity to existing data (assessed by Euclidian distance), to prioritise selection of ‘different’ experiments. In an early task, when models were trained from a limited and sparse data-set, this strategy led to emphasis on a broad range of parameter choices (Figures 3a and b) and resulted in few experiments matching the predictions: Only 8% were on-target. In a later task (Figures 3c and d) models were trained from a three times larger data-set. The resulting improvement in models’ fidelity led to better match between predictions and experiments (68% of on-target experiments) and a narrow range of experimental parameters selected by the filtering algorithm. The narrow ranges of parameter choices in Task 10 are significant, representing convergence on a set of growth conditions that can be used as a growth recipe.

Figure 3
figure 3

Variability in the experimental parameter space with learning. Experimental conditions chosen by ARES in Task 3, before convergence (a, b), and in Task 10, after convergence (c, d), are compared over four experimental parameters (temperature, water concentration, and H2 and C2H4 partial pressures). Red dots represent successful, on-target experiments. (a, b) Before convergence ARES sampled a wide range of growth conditions, and only 8% of experiments were on-target. (c, d) After convergence ARES sampled a narrow range of growth conditions, with 68% on-target experiments, demonstrating its ability to autonomously optimise multiple experimental parameters.

Full size image

We then analysed the entire database of experiments for insights into growth kinetics. The average water/ethylene ratio in Task 10 is 1.6×10−3 (Supplementary Figure S3a), which is comparable to the 1×10−3 reported to maximise CNT yield and growth rate by Futaba et al.40,41 We also found that the rate dependence on the temperature (Supplementary Figure S3b) corresponds to the well-established Puretzky model42 with activation energy Ea=1.1±0.3 eV, which is within the 0.6–1.5 eV range reported in the literature and is interpreted as the activation energy of the precursor decomposition reaction on the catalyst surface.42–44

The data mining results highlight ARES’ ability to search across a complex, multi-dimensional parameter space. All of these growth parameters are known to significantly affect CNT growth, and typically require extensive experimentation and serial optimisation by conventional methods.40,41 The ability of the ARES planner to optimise multiple experimental parameters simultaneously and converge on the predictions highlights its efficacy in solving complex materials research problems that challenge human researchers using conventional, non-autonomous research processes.

Discussion

We believe ARES is the first demonstration of an autonomous research system in the area of materials science. The ability of ARES to learn to control the growth rates of CNTs in multi-dimensional parameter space and over a range of rates is an inroad to the complex and vexing challenge of controlled nanotube synthesis. It also opens broad areas of applicability to other materials research problems, including other CVD reactions, additive manufacturing, metal oxidation, etc. In general, the advantage of the ARES method lies in the application of artificial intelligence and autonomy to research problems that are automatable, have large, complex experimental parameter spaces, and can be probed by in situ characterisation techniques to enable closed-loop feedback. These advantages come at the expense of the time needed for hardware and software development. However, as ARES development expands to more materials development problems, we expect the time and resources needed to adapt it to a particular research problem will progressively reduce. The iterative closed-loop approach contrasts with high-throughput/combinatorial techniques in important ways—they do not have autonomous feedback, and have slow iteration rates.3 In addition, the broadcast nature of library generation results in large numbers of uninformative experiments.

Beyond faster and more efficient experimentation, ARES changes the size, complexity and risk of problems undertaken by researchers who operate under constrained resources. Because ARES enables more challenging experimental campaigns we expect ARES to be a disruptive tool for the research process itself. The immediate future for ARES includes the planner development to enable targeting of multiple objectives as well as expanding the diversity of AI planning algorithms. Random forest/genetic algorithm was just the first learning algorithm we used. ARES enables the application of a variety of AI learning approaches, with the distinct advantage of its iterative method driving the continuous updating and refinement of the AI planner. We will also incorporate more human-interpretable prior knowledge in the form of physical and chemical models, simulations and materials.

ARES also has implications for the role of human researchers and human-machine interaction, which is a topic of interest in the autonomous robotics community. In our case, humans chose the experimental area (CNT growth), the feedback signal and objective (maximum growth rate), seeded the database to initialise the AI planner, and periodically modified the objectives in response to the observed experimental progress. As we continue to develop autonomous research methods, ARES creates a need to explore and define human-machine interaction for generalised research processes, including goal definition, experimental path planning, progress towards goals, human-machine trust (e.g., awareness and reporting of the system and environmental state), and oversight. In a future where repetitive, mundane tasks are efficiently dispatched by ARES, and AI analysis of high-dimensional parameter spaces is done iteratively, researchers will be freed to work to their strengths, performing creative, insightful and contextual tasks.

Materials and methods

The ARES instrument diagram is shown in Supplementary Figure S1. CNTs were grown in a cold-wall CVD chamber installed on a two-axis motion stage above an inverted Raman microscope (Eclipse-Ti, Nikon, Mellville, NY, USA). The growth wafer consisted of 5×5 arrays of silicon pillars spaced 50 μm apart on a thermally insulating SiO2 sublayer, which were 10 μm tall and 10 μm in diameter (Supplementary Figure S1 inset). The pillars were coated with a 10 nm ALD (atomic layer deposition) alumina support layer, on which a 2 nm Co catalyst film was deposited by electron beam evaporation. Each pillar constituted an independently addressable micro-reactor, which was heated by a 532 nm laser (Verdi V6, Spectra-Physics, Santa Clara, CA, USA) that doubled as the Raman excitation source through a 50× extra-long working distance objective lens. Raman spectra were collected from a ~5 μm size spot illuminated by the excitation laser. The small thermal mass of the pillar combined with the low thermal conductivity of the SiO2 sublayer enabled heating to reaction temperatures within a fraction of a second when laser power was varied in the 0–2 W range. The CVD chamber pressure was measured by a capacitance pressure gauge (MKS) and regulated by a throttle valve (MKS) exhausting into a vacuum pump. H2, C2H4 and Ar feedstock gases ware metered in varying ratios via mass flow controllers (MKS, Andover, MA, USA). Water vapour was introduced and controlled through a throttle valve (MKS), and its concentration was kept constant via a PID control loop with a dew point sensor (Shaw, Bradford, UK) as a feedback. The temperature-induced shift of the Si stokes and anti-stokes Raman bands (±520 cm−1) was used to calculate the growth temperature according to the following equation:

(1) Δ ω ( T ) = C [ 1 + 2 e ℏ ω 0 / 2 k B T − 1 ] + D [ 1 + 3 e ℏ ω 0 / 3 k B T − 1 + 3 ( e ℏ ω 0 / 3 k B T − 1 ) 2 ]

Here kB is Boltzmann’s constant, ħ is Plank’s constant, and ω0, C and D are constants with the values 528, −2.96 and −0.174 cm−1, respectively.45 The accuracy of the temperature measurement was estimated to be within ~10–20 °C.

The two-axis motion stage, microscope focus (z), mass flow controllers, pressure controller, water controller, laser, shutters and spectrometer were all simultaneously controlled by custom software developed for 64-bit Windows 7 in C#/.NET 4.0 using Windows Visual Studio 2010. In a typical series of experiments, the planner was first given an objective. The database of the previous experiments containing inputs (temperature, pressure, gas flow rates, etc.) and outputs (kinetic parameters) was transferred to the planner. The planner converted flow rates and pressure into partial pressures of the constituent gases, built a model correlating inputs and outputs, and generated experimental conditions and predicted growth rate. The first pillar was positioned under the laser and the growth atmosphere was set to the intended composition. The laser power was increased in less than a second, and in-situ Raman spectra were acquired in 5 s intervals, with each spectrum processed in real time to obtain temperature (from Si bands’ shifts) and CNT G- and D-band areas. Nanotube nucleation and growth was detected by the appearance and increase in intensity of G- and D-bands seen at ~1,590 and ~1,350 cm−1 (Supplementary Figure S1a). The G-band area was normalised by the laser power, by the Bose thermal occupation factor [ 1 − exp ( ℏ ω / k B T ) ] − 1 (ω is G-band frequency, T is temperature), and by the area of the Si stokes peak at 520 cm−1 obtained with 200 mW laser power and 30 s exposure that served as a proxy for the spectrometer throughput. Normalisation was necessary in order to compare G-band areas and rates across different experiments, and resulted in the rate measured in units of s−1. Upon experiment completion (300 s typical time) the normalised G-band area versus time dependence (‘growth curve’) was automatically fit with equation (2) (Supplementary Figure S2b), and the resulting kinetic parameters were written into the database along with experimental inputs. The last room-temperature spectrum (Supplementary Figure S2c) was acquired with 200 mW power and 30 s exposure:

(2) G ( t ) = G max ( 1 − 1 + θ 0 1 + exp ( ( t − t 0 ) / τ ) ) .

The updated database was transferred to the planner, and the cycle was repeated. This allowed up to 25 experiments in autonomous mode (without human intervention), updating the database after each experiment to build an improved model.

Electron microscopy characterisation was done in FEI Sirion SEM (FEI Company, Inc., Hillsboro, OR, USA) equipped with a field-emission gun and a through-lens detector (TDI) detector, at 4 keV accelerating voltage. Since pillars were on a silicon dioxide sublayer, charging could be severe. As a result, nanotubes present as sparse films (Figure 2a) often appeared in inverse contrast (dark lines on bright background). Thick nanotube mats (Figures 2b and c) tend to eliminate charging.