Introduction

The timeline for materials discovery, development, and deployment is slow and resource intensive, requiring 10–20 years of research and development to bring a product to market1. The process also relies on human intuition and trial and error to traverse vast multivariate synthesis domains. Integrating machine learning (ML) and artificial intelligence (AI) algorithms into the materials development cycle may drastically accelerate the process, reduce the role of human judgement, and alter the paradigm of materials research. Carbon nanotubes (CNTs) represent an illustrative example of the materials development cycle. CNTs have been at the forefront of nanotechnology since their discovery in 19912, yet their adoption into industrial applications remains sparse3. Their mechanical, thermal, and electrical properties far exceed those of conventional engineering materials3, such that the technological potential of CNTs could be disruptive in applications ranging from structural materials4,5, digital electronics6,7,8, flexible sensors9,10,11, power transmission12, and thermal devices13,14,15, among others. As an example, CNTs exhibit an elastic modulus in excess of 1 TPa16,17 but are compliant enough to be tied into a knot and spun into yarns18. Their thermal conductivity exceeds 5000 W/mK19 which is approximately 12 times greater than that of copper, at a mass density that is 4 times less. Single-walled CNTs have an electron mobility that is in excess of 100,000 cm2/Vs20 and can operate at frequencies greater than 1 GHz21. Scaling the physical properties of one CNT to large populations remains a significant challenge. When populations of CNTs are concurrently synthesized to form vertically oriented CNT forests, however, their ensemble properties are drastically reduced. Understanding and controlling the process–structure–property paradigm for CNT forests is a long-standing barrier to the widespread adoption of CNTs in many applications.

CNT forests are routinely synthesized using chemical vapor deposition (CVD) methods. The typical areal density of CNT forests ranges from an order of 109 CNTs/cm2 for multi-walled CNT (MWNT) forests22, to greater than 1013 CNTs/cm223 for single-walled CNT (SWNT) forests. The uncontrolled self-assembly of CNT forests generates an open-cell, foam-like structural morphology seen in the scanning electron microscope (SEM) images of Fig. 1. The wavy, entangled morphology is thought to degrade the ensemble properties of CNT forests, sometimes by many orders of magnitude. As an example, the typical modulus of CNT forests is on the order of 10–100 MPa24,25,26,27,28, a full 4–5 orders of magnitude less than the elastic modulus of individual CNTs. Similarly, the experimentally realized thermal conductivity of CNT forests is on the order of 80 W/mK29, far less than the 5000 W/mK measured for an individual CNT. While the existing performance gap between predicted and observed properties is vast, it serves as an indicator that CNT forests may be designed to operate within a larger technological performance envelope if the CNT forest assembly processes could be better understood and controlled. Realizing properties within this envelope will require the development of new tools that can identify how CNT forest synthesis attributes correlate with forest structure and ensemble physical properties.

Fig. 1: SEM images of carbon nanotube forests.
figure 1

The structural morphology of a CNT forests may exhibit relatively aligned CNTs (a) or wavy and entangled CNTs (b) depending on synthesis conditions. Scale bars represent 500 nm.

Full size image

To efficiently navigate within a high-dimensional synthesis space, researchers have recently constructed a high-throughput, autonomous research robot to deterministically synthesize isolated SWNTs. The Autonomous REsearch Systems (ARES) uses a Raman spectrometer laser to provide heat and simultaneously characterize growing SWNTs in situ30,31. The ARES system utilizes ML algorithms to learn from previous experimental results and uses an AI planner to sequence experiments to reach an experimental objective. In a particularly compelling demonstration, the ARES system autonomously determined experimental parameter sets required to achieve specified SWNT growth rates in fewer than 100 experimental iterations31. The ARES approach shows that integrating AI and ML, with high-throughput experimentation can facilitate experimental parameter space navigation within a process–structure domain that is not well understood by researchers. The interactions between large populations of concurrently growing CNTs in forests add complexity not present during the synthesis of isolated SWNTs, and ARES-style experimental campaigns interrogating CNT forests have not yet been reported but offer a promising approach.

In the current study, a time-resolved finite element method (FEM) CNT forest simulation tool32,33 is used as a high-throughput virtual laboratory to examine the synthesis–structure–property design loop of CNT forests. Images of each CNT forest morphology were obtained at the end of their simulated synthesis. A mechanical compression simulation was used to obtain mechanical properties33,34,35. Deep learning (DL) neural networks, trained and tested using FEM physics-based simulated images, were developed to predict the class label for the CNT synthesized images36. Subsequently, CNT forest physical and growth attributes like stiffness and buckling load are predicted using another ML random forest (RF) regression algorithm. A total of 63 unique CNT forest synthesis classes were established based on combinations of CNT diameter, population growth rate variability, and CNT areal density. From these classes, a combined pool of 22,106 FEM simulated synthesis and compression experiments were performed generating one image per experiment to create a pool of data for training and testing the ML models. The physics-based FEM simulations provided precise ground-truth data that would otherwise be very difficult, if not impossible, to obtain using physical experimentation techniques. The ML techniques achieved greater than 91% classification accuracy of physical and growth attributes, with an R2-regression fit of 0.96 and 0.94 for buckling and stiffness predictions, respectively. The image-based R2 coefficient of determination, and root-mean-square error (RMSE) values matched or exceeded those obtained using a classification-based linear regression analysis in which all input physical attributes were precisely known a priori and no image data was considered. The results demonstrate that image-based ML algorithms using simulated SEM imagery are able to detect unique visual structural morphological characteristics of CNT forests to provide precise, individualized predictions.

Results

Simulating CNT forest synthesis and mechanical compression

The typical CVD parameters available for CNT forest synthesis include catalyst composition, catalyst thickness, buffer layer composition, buffer layer thickness, substrate temperature, gas temperature, catalyst conditioning, hydrocarbon gas composition, carrier gas composition, synthesis pressure, and synthesis time, among others. Each of these parameters can influence the resulting diameter distribution, areal density, growth rate, and CNT catalyst lifetime. For example, increasing the porosity of an alumina buffer layer (controlled by deposition methodology) was shown to drastically increase the growth rate, lifetime, and density of CNTs when using an Fe catalyst thin film37. Likewise, trace amounts of carbon can increase CNT number density by reducing oxidized catalyst nanoparticles38,39. Because of the cost and time required to explore the available synthesis parameter space, researchers typically operate within a small parameter range that has produced acceptable CNT growth in the past. In fact, time and cost constraints are disincentives to thoroughly explore the vast parameter space. For the 11 synthesis parameters mentioned above, a full combinatorial experimental campaign consisting of just three quantized levels per parameter, without replicates, would consist of 177,147 experiments. At a relatively aggressive pace of five experiments per day, this experimental campaign would require almost a 100 years to complete. Characterization of the resulting CNT forests would require additional time that may exceed the time required for synthesis.

Here, a mechanical finite element simulation was used as a high-throughput virtual laboratory to synthesize and mechanically compress thousands of unique CNT forests. DL algorithms classified the CNT forest physical attributes and predicted the CNT forest mechanical properties based solely on the simulated CNT forest imagery. DL algorithms typically require a very large amount of training data which is not available in our case. The DL algorithms bootstrap onto the physics-based simulation to predict the complex synthesis–structure–property relationships of CNT forests, and can be viewed as an advanced data augmentation method to provide more training data, where data, ground-truth, or both are very sparse like in molecular, cellular, and tissue imaging40,41,42,43. Diverse CNT forests could then be generated using combinations of seven CNT population densities, three CNT diameters, and three levels of population growth rate variability. The CNT linear density varied from 50 to 200 CNTs per 10 μm simulation span. These CNT densities correspond to 2.5 × 109 to 4 × 1010 CNT/cm2, consistent with reports for MWNT forests39. Seven unique CNT density levels were selected to span this large span of observed results. The levels of CNT diameter (10, 16, and 22 nm) and CNT forest growth rate coefficients of variability (3%, 6%, and 9%) were also selected based on available data for MWNT forest synthesis data44. Each unique combination of these parameters was considered to represent a labeled class or category and used for supervised learning. We designate this complementary suite of digital AI/ML tools for CNT material discovery as CNTNet.

Each CNT forest simulation uses homogeneous CNT diameters selected from one of three values and a stochastic assignment of growth rate and orientation angle. Figure 2 displays a representative set of simulated CNT forest morphology images from each of the 63 forest classes. While synthesis is confined to a two-dimensional (2D) plane, the resultant CNT waviness and bundling resemble physical CNT forests. In this way, the resultant CNT forest morphology approximates a slice from larger 3D CNT forest. Each block of images is sorted based on the CNT density. The lowest and highest CNT densities, denoted as D50 and D200 (corresponding to 50 and 200 CNTs per 10 μm span, respectively), are enlarged and shown in the third row of the figure. Some trends may readily be identified by the naked eye, particularly when observing the lowest and highest density examples. Increasing CNT linear density decreased the white space in each image, while the wavelength and amplitude of CNT waviness also decreased. An increased growth rate coefficient of variation within a forest decreased the vertical orientation of individual CNTs within a forest, leading to an increased CNT–CNT contact density and decreased wavelength along the length of the forest. Importantly, all CNTs were plotted with the same line thickness, regardless of CNT diameter. This decision was motivated by the likely resolution limits of an SEM, which at modest magnifications would be unable to distinguish between the small variations in CNT diameters presented here. As a result, all diameter classifications were inferred based upon variations in the CNT forest morphology rather than by direct diameter acquisition. Because the bending stiffness of hollow cylinders increases with the fourth power of diameter, the CNTs within larger-diameter CNT forests have less curvature than smaller-diameter CNT forests.

Fig. 2: Representative images from all 63 simulated CNT forest classes.
figure 2

Each block of nine sub-types is grouped by CNT linear density, ranging between D50 and D200 (seven values), representing a density of 50 and 200 CNTs per 10 μm simulation span, respectively. Each block of nine images is arranged in an order consistent with the key provided on the right side of the middle row of images. Here, R indicates the outer radius in nm (three values), and G is the growth rate coefficient of variation (three values). The image blocks for D50 (red border) and D200 (green border) are expanded in the third row to show greater image texture details. The horizontal domain is 10 μm for each simulation. Note that each image has a replicated texture block in the horizontal direction to demonstrate periodic boundary conditions.

Full size image

Each of the CNT forests used for classification were mechanically compressed to determine their buckling load and elastic stiffness. To the best of our knowledge, this study of 22,106 CNT forests represents the most extensive evaluation of CNT forest mechanics to date. A representative CNT forest undergoing compression, and the resultant force–displacement response, is shown in Fig. 3. During compression, the CNT forest morphology deformed and buckled at the top and bottom-most regions of the CNT forest throughout the initial stages of compression, before densification. Coordinated CNT buckling that originated near the top surface of the CNT forest grew in extent with increased compressive strain, while a single buckle near the bottom of the forest persisted without obvious growth. Similar collective buckling behavior has been observed experimentally24,26,45,46. The forest stiffness (N/m) was determined from the tangent loading slope evaluated at a compressive displacement of 1.5 μm from the top surface of the forest. The initial 0.5 μm of compression after first contact deformed only the free ends of the tallest CNTs, as shown by the small and erratic loads at the onset of compression in Fig. 3(c), and was not characteristic of the overall forest stiffness. At 1.5 μm compression, by contrast, contact was established with the entire span of the CNT forest. The buckling load was determined by imposing a line with a slope equal to the CNT forest stiffness and an offset of 0.4 μm of compression. The buckling load was defined as the point at which applied load and the offset stiffness line (shown in red in Fig. 3(c)) intersected. The typical compression simulation was terminated after these characteristic metrics were obtained. An extended compression experiment in Fig. 3(d) is showing the typical elastic, plateau, and densification regions that are characteristic of CNT forests.

Fig. 3: Simulated CNT forest compression.
figure 3

An expanded CNT forest compression demonstration showing the simulated CNT forest morphology (a) at the beginning of compression and (b) after 20 μm compression. c The full load–displacement curve shows the typical linear-elastic regime at low displacement, an extended plateau, and a densification regime. d A magnified view of the loading curve shown in (c) illustrates the evaluation of forest stiffness and buckling load.

Full size image

Nested box plots in Fig. 4 display the stiffness and buckling load representative of all CNT forests. Axes are plotted using a logarithmic scale to show the power-law scaling that exists between forest attributes and the corresponding mechanical response. Within each plot, the mean value is plotted as a solid dot, the shaded box represents the upper and lower quartile of the data, while the vertical whiskers represent 1.5 times the interquartile range. The dual x-axes are sorted by nesting CNT linear density and CNT outer radius in ascending order. Note that the inner CNT radius is 70% that of the outer radius in all simulations. The various population growth rate coefficients of variation are plotted by color within the box plots. Based on the box plots, the buckling load and stiffness are positively correlated with all CNT attributes, with a ranked sensitivity (from smallest to greatest) of population growth rate variation, diameter, and linear density. The variation of stiffness and buckling load spans >3 orders of magnitude for the combination of parameters varied in the current study.

Fig. 4: Nested box plots of simulated CNT forest mechanical properties as a function of class.
figure 4

The (a) buckling load and (b) elastic stiffness for 63 distinct classes of CNT forests vary by orders of magnitude within the simulated parameter space. The shaded box represents the upper and lower quartile, while the vertical whiskers represent 1.5 times the interquartile range. The bottom grouping, ranging from 50 to 200 represents the quantity of CNTs, the upper grouping represents CNT outer radius of 5, 8, and 11 nm.

Full size image

To determine the scaling relationships associated with each CNT forest attribute, and to serve as a baseline for later ML predictions, a linear regression analysis of CNT forest buckling load and stiffness was performed relative to the input CNT forest physical parameters. The analysis inherently presumed a foreknowledge of the CNT forest characteristics of CNT density, CNT diameter, and population growth rate variation and was not informed by CNT forest morphology images. We stress that the precise knowledge of these CNT forests attributes would be difficult or impossible to obtain with physical experiments, while simulation provided access to the exact values. The values within each CNT attribute class were normalized by the modal value for the regression analysis to ensure that each parameter varied by similar magnitude. A linear regression analysis based on the log10 parameter values was then conducted. The analysis showed that Ef ~ ρ1.9, Ef ~ D1.34, and Ef ~ COV0.13, where Ef, ρ, D, and COV represent CNT forest stiffness, density, diameter, and population growth rate coefficient of variation, respectively. Similarly, the CNT forest buckling load, σy, scaled as σy ~ ρ1.94, σy ~ D1.01, and σy ~ COV0.25.

Well-established scaling relationships exist for ideal open-cell foams that relate elastic modulus, E, yield strength, σy, and relative density, ρ. For ideal 2D open-cell foams and honeycombs, E ~ ρ2 and σy ~ ρ2, while isotropic 3D foams, including isotropic CNT foams47, exhibit E ~ ρ3 and σy ~ ρ3 scaling relations. Alumina-coated CNT forests have exhibited scaling on the order of scale as E ~ ρ2.8 and σy ~ ρ2.948, slightly less than that predicted by open-cell foam predictions. In a similar fashion, our scaling factors of Ef ~ ρ1.9 and σy ~ ρ1.9 are slightly less than the ideal value of 2 for these parameters with 2D open-cell foams, providing confidence in the simulation output. We anticipate that scaling the existing simulation to three dimensions will reflect the scaling relationships of 3D open-cell foams, as observed experimentally48.

The predicted buckling load and stiffness resulting from the regression, plotted relative to their ground-truth values, may be found in Fig. S1. Note that the linear regression model using the known values of physical attributes (without using image properties) provides a single prediction for each of the 63 classes. A statistical linear regression model produced an R2 value of 0.94 for both buckling load and forest stiffness, indicating that buckling load and CNT forest stiffness was predicted relatively well conditioned upon knowledge of the CNT physical attributes. The linear regression model had a root-mean-squared error (RMSE) of 0.22 and 0.20 for stiffness and buckling load, respectively, for the same data. While this linear regression approach is straightforward when using synthetic simulation because all CNT forest simulation parameters are prescribed, for real-world experiments the determination of CNT diameter, density, and growth rate variation is time consuming, cost prohibitive, and may be unrealistic to obtain with physical experiments for even modest parametric studies. These difficulties motivate the physics-based simulation and image-based prediction capabilities afforded by the DL techniques.

CNTNet classification and prediction results

Quantifying the complex and entangled CNT forest morphology after synthesis is itself a difficult task. Because of the relatively large SEM depth of field and because adjacent CNTs frequently bundle into ropes during synthesis, even rudimentary CNT forest measurements (such as diameter distribution and CNT areal density) are difficult using SEM analysis alone. Typical metrics to describe CNT forest alignment include the Herman’s orientation factor (HOF) obtained by small-angle X-ray scattering (SAXS)22,49, scanning electron microscopy (SEM) imagery38, or hand-crafted image-based descriptors27. Of these, SAXS is perhaps the most descriptive quantitative technique for CNT forest characterization, as it provides measurements of CNT alignment, diameter distribution, and areal density. This technique requires a high-intensity synchrotron photon source to penetrate the forest, and as such, it is not readily available. The SAXS measurement also acquires population-averaged metrics that may be difficult to correlate to individual CNT–CNT interactions. Nevertheless, SEM-based morphology characterization is currently the most frequent methodology for CNT forest characterization. As such, we designed a DL framework to learn from physics-based simulated CNT forest morphology images that are similar in scale and resolution to SEM imagery.

DL architectures are a powerful approach for pattern representation learning, and are being widely adopted in many fields including natural object image classification50, aerial scene classification51, feature tracking in wide-area motion imagery52, 3D point cloud classification53, vessel segmentation54, and malaria diagnosis55. Recent work evaluated using hand-crafted texture features like joint adaptive median binary patterns56,57 combined with random forest ML to identify the CNT class label for 12 classes36. In this work, we take advantage of recent developments in DL architectures to increase the number of CNT classes from 12 to 63, improve the classification accuracy, and add a ML approach for the prediction of CNT physical attributes and mechanical properties using simulated growth imagery. The CNTNet classifier provides a discrete class label output. We use the last feature layer, before the output layer, in the CNTNet classifier as the independent feature vector. We use RF regression trees with the 4096-D deep image-based features learned by the deep neural network (DNN) classifier in order to provide continuous real-valued estimates of the dependent mechanical properties. By employing this scheme, mechanical property predictions benefit from the descriptive power of learned deep features combined with the accuracy and robustness of random forest to noise and over-fitting by using an ensemble of diverse trees. As shown in Fig. 5, the proposed CNTNet classification and regression framework consists of a pipeline of several modules. The CNT forest images generated using the physics-based growth simulation model are used as input to train a VGG-19 DNN58 to predict CNT forest attributes (structural classification module) and mechanical properties (property regression module). The finite element simulation generates CNT forest structural morphology images, along with mechanical stiffness, and buckling load values. The CNTNet classifier DNN accepts CNT forest images as input and generates predictions of the CNT class labels along with the feature space embedding for CNT forest attributes and the mechanical properties of stiffness and buckling load. In this way, the CNTNet capability spans the entire synthesis–structure–property paradigm for CNT forest synthesis and virtual screening.

Fig. 5: Schematic of CNTNet modules.
figure 5

CNTNet is a modular image-based machine learning and control framework for enabling artificial intelligence-driven classification of CNT forest synthesis attribute groups and predictions of CNT forest material properties. The top structure classification module uses the VGG-19 deep learning network for classifying CNT forests into one of 63 CNT classes based on combinations of synthesis attributes generating different CNT forest simulated SEM image classes. The structure classification network captures morphological texture properties and was trained using transfer learning with initialization using ImageNet weights. The bottom property regression module uses random forest regression trees to learn CNT forest buckling load and stiffness by mapping the 4096-D feature image representation descriptor using 2D regression vector functions.

Full size image

Using CNTNet the overall classification accuracy for 63 classes was 91.0%, indicating that all three attributes of CNT density, radius, and growth rate coefficient of variation were correctly predicted with high accuracy on the test images. The feature descriptor embedding from the CNTNet classification module is then used for the CNTNet regression task to predict the CNT forest physical properties. Unlike the classification-based traditional linear regression model in the previous section, for the CNTNet RF regression predictor, the image feature descriptor embedding alone was used as the model input, without any knowledge of CNT forest attributes provided to the decision trees. The CNTNet-predicted buckling load and elastic stiffness relative to the known ground-truth values are shown in Fig. 6(c), (d). Note that the solid line represents an ideal 1:1 correlation between CNTNet prediction and ground-truth parameters from the physics-based simulation. For both stiffness and buckling load, the predicted mechanical properties lie close to the ground-truth values for ranges of values that vary by >3 orders of magnitude with an R2 value of 0.96 for buckling load and 0.94 for stiffness, which is better than the R2 values of 0.94 obtained for the classification-based traditional regression with more information known a priori. The RMSE values produced for the CNTNet-predicted stiffness and buckling load are 0.21 and 0.17, respectively. These values are an improvement over the respective RMSE values of 0.22 and 0.20 using the previously discussed statistical linear regression predictions which utilized the numerical values of CNT density, diameter, and growth rate variation as inputs.

Fig. 6: CNTNet classification and regression accuracy.
figure 6

a Confusion matrix for 63 CNT classes with the ordering based on seven CNT densities and nine sub-types. b t-SNE visualization of the 63 CNT forest classes grouped by the seven CNT population densities for better visualization. The separation of the density groups using the VGG-19 feature embedding sub-space indicates that the CNT forest density is classified with high accuracy. CNTNet regression prediction for (c) buckling load and (d) stiffness compared to the simulated ground-truth values. Data are colored according to the CNT forest linear density.

Full size image

The superior accuracy of the CNTNet regression predictions has practical implications. As stated previously, the structural characterization of CNT forests using methods such as SAXS is resource intensive and prohibitive for a high volume of samples. Image acquisition of CNT forests using SEM, however, is readily available to most researchers. The CNTNet results demonstrate that property predictions using image data may be superior to algorithms in which all CNT forest physical attributes are known a priori. Such a result might not be surprising when considering that image data contains embedded information that is unique to each CNT forest. Whereas the classification-based regression model based upon physical attributes generates a single-value prediction for all forests with matching attributes, image-based regression detects subtle differences within CNT forest morphology to generate unique predictions even for forests that share similar attributes. Moreover, the images themselves may contain sufficient information to infer the population-based physical attributes provided by methods such as SAXS, as demonstrated by the CNTNet classification results.

Data visualization can be used to understand the CNTNet classification and regression performance in a qualitative manner. The multi-class confusion matrix in Fig. 6(a) shows the misclassification accuracy between class Cq and Cr arranged by CNT density, in the form of a matrix with color intensity mapping to the percentage of predicted values which fall within a known ground-truth class label. The axes of the confusion matrix represent the predicted class label and the known ground-truth classification. The diagonal entries in the confusion matrix represent the respective class accuracies, while all other entries represent a misclassification. Marginal confusion matrices are provided as Supplementary material which show the classification accuracy as a function of only CNT density, only CNT radius, or only population growth rate variation. In these matrices, the classification of a given parameter level is assumed to be correct even if the other independent parameters are not correctly predicted. The lowest prediction accuracy in each of these quantities was 99.97% for CNT density, 88.99% for CNT radius, and 97.88% for growth rate coefficient of variation. The CNT density was predicted with 100% accuracy for forests having between 50 and 150 CNTs per 10 μm span. The high degree of accuracy of the CNTNet classification module demonstrates that DL techniques can readily identify subtle differences in the wavy texture morphology of CNT forests arising from the relatively small changes in CNT attributes.

The t-distributed stochastic neighbor embedding (t-SNE)59 can be used to visualize the class clusters projected in a low-dimensional space suitable for qualitative understanding of the class separability and inter-class mixing. This is important for the CNT classification task due to the large number of 63 classes with subtle texture differences between some classes. t-SNE reduces the dimensionality of the 4096-D feature descriptor embedding vectors learned by the VGG-19 DL network to generate a 2D plot as shown in Fig. 6(b). A principal component analysis (PCA)60 was first used to reduce the feature space dimensionality of the classification embeddings from 4096 to 50 before projecting their values to a 2D space using t-SNE. The 63 CNT classes are grouped into seven density classes shown in different colors. The seven CNT density classes are well separated in the t-SNE sub-space, indicating that CNTNet has learned a feature embedding manifold that is highly discriminatory. The t-SNE distributions sorted by diameter and growth rate variation are provided in the Supplementary material and show overlapping clusters that supports our confusion matrix analysis. The misclassification rate around 9% is due to the overlap in the 9 combinations of diameter and growth rate variation within each CNT density cluster as shown in Fig. S5.

Discussion

It is striking that image-based mechanical property prediction, matched or exceeded statistical predictions in which all CNT forest attributes were known with certainty. An image-based AI/ML approach using physics-based image synthesis is particularly powerful and applicable because the physical attributes of CNT forests are rarely, if ever, known with certainty in physical experiments. The CNTNet classification–regression framework demonstrates that the feature descriptor information embedding using CNT forest structural morphology images is sufficient to characterize the relevant synthesis-related CNT attributes (CNT density, CNT diameter, and CNT growth rate variation) and ensemble properties of CNT forests with high accuracy without any other external knowledge. We anticipate that these image-based computer vision techniques, upon further refinement, can be translated to and augmented with experimentally acquired SEM images of CNT forests to rapidly characterize their physical attributes with similar high accuracy. We acknowledge that the current simulation model may only capture the first-order mechanisms of CNT forest self-assembly and mechanics, but we anticipate that data from emerging experimental techniques will continuously improve the fidelity of the simulation techniques.

An image analysis system designed to characterize SEM images of CNT forests must accommodate SEM-specific imaging characteristics including molecular structure variability, measurement noise and grayscale intensity variations through the depth of imaging. We are planning to develop a new module to generate photo-realistic images of simulated CNT forests that reflect SEM appearance characteristics. The simulated CNT forests themselves will also be generated using CNT diameter distributions consistent with specific synthesis criteria. The CNTNet classifier and CNTNet regressor will then be retrained using more realistic synthetic CNT imagery augmented with labeled SEM images as these become available. Previous work on DL-based realistic synthetic data generation systems61,62, general adversarial networks63,64, and synthetic microscopy image generation approaches65,66 will inform our generation of more realistic synthetic CNT forest images, characterizing the generalization power of CNTNet and for translating the proposed pipeline to the analysis of SEM images.

High-throughput simulation is expected to play a major role in mapping the synthesis–structure–property relationships of CNT forests for virtual screening. The current study did not correlate the prescribed CNT synthesis attributes of diameter, areal density, and growth rate variation to specific process parameters such as catalyst composition, catalyst thickness, temperature, and gas flow rate; however, these correlative studies are being conducted, and limited empirical data exists within the literature. Precise determination of CNT catalyst kinetics is expected to accelerate in the coming years using in situ transmission electron microscope and SEM synthesis observations. As more complete experimental data is obtained, the relevant physical relationships between process parameters and CNT synthesis attributes can be integrated to improve the physics in the simulation framework and generate increasingly accurate simulations with little additional computational resources. The exploration of the nearly inexhaustible CNT forest parameter space will accelerate while becoming significantly less cost prohibitive.

Future advancements of CNTNet lie in the incorporation of experimental and numerical simulation data as separate information sources, but coordinated data streams. SEM-based in situ synthesis experiments are currently being conducted that will both validate the physics and kinetics of the finite element simulation and will provide a source of CNT forest SEM image inputs. The in situ experiments will provide an opportunity to approximate the ground truth and to precisely correlate simulation and experiment, which will allow evaluating the generalization capability of the CNTNet ML model. The implementation of 3D finite element code, analogous to the 2D code presented here, is ongoing and will produce CNT forest morphology imagery that is even more analogous to CNT forest SEM imagery. As quantification and understanding of increasingly complex CNT forest growth kinetics are understood and reported, simulation fidelity will continually increase. In the near future, we expect that a well-validated CNT forest simulation tool could be used to autonomously navigate the CNT forest synthesis parameter space to rapidly map the available combinations of CNT forest ensemble physical properties afforded by diverse CNT forests morphology, particularly those which have not been previously explored. These properties could then be confirmed and refined experimentally. Similarly, robust image-based characterization of CNT forest physical attributes will be enabled when a repository of well-characterized CNT forest SEM images becomes available. The tools developed and reported here are foundational to these efforts and establish that image-based DL can both classify CNT forests and predict their physical properties with high accuracy.

Methods

Physics-based CNT growth and compression simulation for ML

All finite element simulations were conducted using MATLAB 2018b software running a custom FEM code. CNT elements were treated as Euler-Bernoulli frame elements with a computational node at each end of the element, as described elsewhere33. The van der Waals interactions between adjacent CNT segments were simulated as linear-elastic spring elements. As neighboring CNTs contact, the van der Waals interactions may lead to the formation of bundles or ropes of multiple CNTs that often persist throughout the height of forests. Likewise, contacting CNTs may bend and form a wavy morphology. Each unique morphology is shaped by the mechanical equilibrium established between the contacting CNTs. Mechanical equilibrium was computed at each time step for all nodes within the CNT population before initiating a new set of elements at the base of existing CNTs at discrete time steps to simulate CNT growth. To accommodate the new CNT elements, the nodes that had previously resided at the bottom of the forest were displaced by a distance corresponding to the growth rate of each CNT and at an angle consistent with the orientation angle of the CNT. The stiffness matrix is regenerated at the beginning of each time step to account for the new orientation of each element, the formation of new CNT–CNT contact points, and the addition of new elements to the system. A horizontal growth span of 10 μm was selected for all simulations, with periodic boundary conditions at the horizontal extremes of the simulation domain to mitigate potential edge effects32. The population-averaged growth rate was 60 nm per time step for all simulations. Diverse CNT forest populations were achieved by selecting among seven CNT population densities (50–200 CNTs in increments of 25), three CNT outer radii (5, 8, 11 nm), and three values of CNT population growth rate coefficient of variation (3%, 6%, 9%), for a total of 63 distinct classes. The inner radius of each CNT was 70% that of the outer radius. Because the characteristics assigned to each CNT within a forest were chosen stochastically, each simulation produced a unique CNT forest, even within a common class. Each synthesis simulation was terminated once all CNTs in the population reached a height of 20 μm, and the growth image of the CNT forest morphology was saved for DL analysis.

Using the same simulation framework, the mechanical compression of each forest was simulated by vertically translating a rigid horizontal surface at 20 nm increments per time step. Each CNT node in contact with the simulated moving boundary was pinned, and the base node of each CNT was fixed, simulating a rigid substrate. Mechanical equilibrium was computed for each displacement step, and the compressive load was determined by summing the vertical reaction forces of all CNTs in contact with the moving surface. This force was equal in magnitude to the summed force transmitted to the simulated substrate. The plastic deformation of constituent CNTs was considered by employing the Brazier instability mechanism for thin-walled cylinders in which the circular cross-section of CNTs forms a flat kink67. For hollow CNTs, kinking of the original circular cross-section reduces the bending stiffness, and the cylinder can support less of a bending moment. The critical kinking moment can be expressed as,

$${M}_{{\mathrm{kink}}}=0.4683\frac{ED{t}^{2}}{\sqrt{1-{\nu }^{2}}}$$
(1)

for a thin-walled cylinder, where E is the CNT modulus, D is the CNT outer diameter, t is the CNT wall thickness, and ν is the Poisson’s ratio (0.17 for graphite). This critical limit was experimentally validated for CNTs using in situ TEM deflection and assuming a thickness of 2 walls where the maximum moment was generated67. The experimentally observed kinking moment was 25% greater than this approximation, as inner-most CNT walls acted to strengthen the deforming cross-section. In our simulation, the full wall thickness of the CNT was considered when computing the kinking moment of an element. When an element exceeds the critical kinking moment, its moment of inertia is reduced by a factor of 105 to approximate the limitation of the kinked cross-section to support mechanical moments. The elastic modulus of each kinked segment is reduced by a factor of 103. These values are estimates and characterize the inability of a kinked CNT to support additional load or mechanical moment.

CNTNet: DL and ML regression using a physics-based virtual laboratory for data generation

The theoretical understanding of CNT morphology, structure and physical properties is limited, and the experimental generation of a large collection of real SEM imagery that is essential for training DL networks is unavailable due to difficult material synthesis, high cost, and high resource requirements12. Therefore, we investigated the feasibility that a DL architecture trained using only simulated imagery from the physics-based simulation virtual laboratory model could be used for virtual screening68. Success using a virtual laboratory screening method would provide confidence for pursuing a computational automated materials design approach69 using SEM imagery from physical experiments to accelerate the guided search for optimal CNT materials with desirable properties.

We developed CNTNet a deep network that transfers the high-dimensional texture-based feature classification embeddings to predict CNT physical properties using ML-based RF regression. CNTNet is composed of two modules as shown in Fig. 5. The first structure classification module is the CNT image representation embedding deep architecture, which is learned using a classification task network. The second property regression module uses RF regression70,71,72,73 for CNT physical property prediction using supervision from the physics-based simulation system. Random forests provide a unified framework for manifold learning70, interpretability in the context of explainable AI74, better robustness to adversarial noise, and randomization in RF has been shown to be a powerful way to learn distances between images of never-seen objects75. We use the feature space embedding learned during K-way classification to boost the prediction power of the property regression module CNTNet. Once the deep network is trained on the visual appearance-based structure classification task, all the DL parameters or weights θC are frozen, so that the embedding representation learned through structure classification can be transferred to the property prediction module.

The CNTNet framework is motivated by the observation that CNT forest properties are strongly correlated with their class grouping (x-axes) and physical attributes of buckling load and stiffness, as depicted in Fig. 4. Using the structure classification DL module, the CNTNet extracts a rich high-dimensional feature descriptor that encodes the prior knowledge of CNT physical attributes in an embedding manifold, which can then be used to learn additional physical properties in the descriptor space, and leads to the observed strong performance of CNTNet for predicting the physical properties. The feature embedding provided by the VGG-19 network was used for both determining the CNT forest class label based on simulated growth attributes, and as input to a RF regression estimator to predict the mechanical properties.

The structure classification module in CNTNet uses physics-based simulated CNT forest structural imagery as input to predict the CNT attributes of CNT diameter, CNT population growth rate variation, and CNT density based on the 63 class groupings. Each class represents a unique (discrete) combination of these parameters. Given the morphology images Xi of CNT structures, Xi ∈ XCNT, where XCNT is the training set of CNT images. In this work, we used the VGG-19 CNN to learn the mapping function from CNT structure to CNT forest classes. We define \({f}_{{\theta }_{C}}(\cdot )\) as the mapping function from the input (image) space Xi to the class label space pi, parameterized by the classification network weights, θC, which is then followed by the \({{\mathrm{Softmax}}}(\cdot )\) non-linear operator, defined as:

$$\begin{array}{l}{p}_{i}={\mathrm{Softmax}}({f}_{{\theta }_{C}}({{{{\bf{X}}}}}_{i}))\\ {\mathrm{Softmax}}({z}_{i,k})=\frac{{e}^{{z}_{i,k}}}{\mathop{\sum }\nolimits_{j = 1}^{K}{e}^{{z}_{i,j}}},\,{{{\rm{for}}}}\,j=1,...K\end{array}$$
(2)

where Xi is the ith input CNT image, and pi is the associated CNTNet-predicted class label for Xi. After applying the Softmax operation to, zi,k, the output of the VGG-19 network estimates K real-valued class probabilities. Note that \({{\mathrm{Softmax}}}(\cdot )\) is a differentiable version of the maximum operator. During training we minimize the cross-entropy loss J defined as,

$$J=-\frac{1}{N}{{{{\rm{{{\Sigma }}}}}}}_{i = 1}^{N}\,{{{{\rm{{{\Sigma }}}}}}}_{k = 1}^{K}\,{y}_{i,k}{{\mathrm{log}}}\,({p}_{i,k})$$
(3)

where yi,k is the element in yi of kth class, and yi is the true label (1 for the target class and 0 for the other (K − 1) classes), pi,k is the predicted probability of label k for input Xi, and N is the mini-batch size during training. In our experiments, we used VGG-1958 architecture as the DL backbone network, and stochastic gradient decent (SGD) optimization to learn the optimal parameter space θC.

The classification embedding for the ith input (image) Xi is defined as \({{{{\bf{d}}}}}_{i}^{C}\in {{\mathbb{R}}}^{E}\), where E is the dimensionality of the classification embedding vector,

$${{{{\bf{d}}}}}_{i}^{C}={f}_{{\theta }_{C}}({{{{\bf{X}}}}}_{i})$$
(4)

where \({f}_{{\theta }_{C}}(\cdot )\) is the classification mapping to identify the CNT class type of the input image defined in Eq. (2). Next, we learn \({f}_{{\theta }_{R}}(\cdot )\) the unknown mapping function from the CNT structure classification space to the CNT physical property space that is parameterized by the RF regression variables θR. We define the predicted CNT physical property as the output vector \({\hat{{{{\bf{y}}}}}}_{i}\in {{\mathbb{R}}}^{2}\),

$${\hat{{{{\bf{y}}}}}}_{i}={f}_{{\theta }_{R}}({{{{\bf{d}}}}}_{i}^{C})$$
(5)

where one output dimension predicts CNT buckling load, and the other output predicts stiffness. The reason we learn a single RF regression to predict both properties of buckling load and stiffness jointly is due to their physically coupled relationship that may lie in the same manifold embedding. A figure showing the correlation between buckling load and stiffness may be found in the Supplementary file (Fig. S3). This approach is further strengthened by the general behavior of open-cell foams which consistently yield at an engineering strain of approximately 5%76.

In our experiments, 60% of simulated images were used for training, 20% for validation, and 20% for testing. For the CNT classification label assignment task, the pyTorch DL framework was used to train a VGG-19 backbone58 for image texture-based CNT structure categorization. For the physical property prediction task, we used the image texture embedding from the VGG-19 structural classification results to develop a RF regression model using the Scikit-Learn software77. The size of our generated images was 907 × 725 pixels. During training, the images are resized to 256 × 256 pixels initially, followed by the random crop augmentation to extract sub-images that were 224 × 224 pixels. Later, during validation and testing, we first resized images to 256 × 256 pixels, and then performed center cropping of each image to be the same 224 × 224 pixels. The hyperparameters for training the VGG-19 DL network used a stochastic gradient optimization approach with an initial learning rate value of 10−3, a weight decay of 10−4 and momentum of 0.9 for updating the gradient at each iteration, and a mini-batch size of 32 for gradient estimation. Common image data augmentation methods were used to enrich the amount and variety of training data, including random cropping to extract sub-images, and random flip with 0.5 probability. All experiments were conducted in 5-fold cross-validation settings. We trained the deep network for 200 epochs and used the classification accuracy on the validation data to select the best model parameters. The validated VGG-19 learned network model was then evaluated using the testing data to avoid overfitting. The 4096-dimension feature vector embedding from the last fully connected layer in the VGG-19 classification model was used to train the random forest regressor decision trees.

The RF regression module in CNTNet uses the RF approach which builds an ensemble of regression trees with randomized feature selection to learn the quantitative relationship between elastic stiffness and buckling load values. When training the regressor, the outputs are regressed to their \({{{\mathrm{log}}}\,}_{e}\) values. The Scikit-Learn library77 was used to learn the RF regression. In total, 1000 decision trees of varying depth were learned in the RF ensemble, all the 4096 features were considered at each node split, and with pure leaf nodes as the stopping condition during tree construction. The RF vector regressor is trained to predict two properties, buckling load and stiffness after a logarithmic transformation to handle the 5 orders of magnitude range in property values (see Figs. 6 and S3).

Evaluation metrics

To evaluate the performance of CNT forest classification, the standard overall accuracy (OA) metric was used,

$${\mathrm{OA}}=\frac{\mathop{\sum }\nolimits_{k = 1}^{K}{C}_{k,k}}{\mathop{\sum }\nolimits_{q = 1}^{K}\mathop{\sum }\nolimits_{r = 1}^{K}{C}_{q,r}}$$
(6)

where K is the number of classes, the K × K confusion matrix with diagonal terms being the correct classification probabilities and off-diagonal entries, Cq,r, are the misclassification probabilities between the ground-truth class q predicted to belong to class r.

To quantify the prediction of CNT forest mechanical properties, we use the coefficient of determination R2 and RMSE metrics. The R2 score is computed as,

$${R}^{2}(y,\hat{y})=1-\frac{\mathop{\sum }\nolimits_{i = 1}^{n}{({y}_{i}-\hat{{y}_{i}})}^{2}}{\mathop{\sum }\nolimits_{i = 1}^{n}{({y}_{i}-\bar{y})}^{2}}$$
(7)
$$\bar{y}=\frac{1}{n}\mathop{\sum }\limits_{i=1}^{n}{y}_{i}$$
(8)

where yi is the true CNT forest property value, \(\hat{{y}_{i}}\) is the predicted CNT forest property value, and n is number of samples. The RMSE metric between the true and predicted property values is computed as,

$${\mathrm{RMSE}}=\sqrt{\frac{1}{n}\mathop{\sum }\limits_{i=1}^{n}{({y}_{i}-\hat{{y}_{i}})}^{2}}$$
(9)