Introduction

There is now growing evidence showing that cells sense mechanical cues in the surrounding microenvironment to regulate their functions such as proliferation, differentiation, apoptosis, and pro-inflammation1,2,3,4,5,6. In response to the mechanical cues, cells often adjust their cytoskeletal tension such that many of the mechanical information are translated into a level of inherent cellular traction forces, and in turn into intracellular signals regulating the related functions3,7,8,9. Traction forces, thus related to various cell functions, are generated by the activity of nonmuscle myosin II and actin filaments that determine cellular contractility2,10,11,12. Because these proteins work downstream of diverse signaling pathways, it is often difficult to predict how the force may change upon perturbations to particular molecules such as gene mutations and drug administration. Thus, technologies allowing for efficiently evaluating the cellular traction force are expected to enhance comprehensive understanding of the force-related pathways.

We previously developed a wrinkle assay, a modified version of the method originally reported by Harris and colleagues13,14, in which the silicone substrates are spatially treated with uniform oxygen plasma to allow them to buckle upon the forces exerted by cells15,16,17.

As the individual wrinkles are lengthened with the increase in the forces18, the wrinkle length, detected for example by a machine learning approach19, can be used as a measure of the relative change in the force caused by perturbations, such as specific gene mutations. This technology is promising in that these experiments are performed easily to potentially enable a high-throughput analysis on the force-related pathways or drug screening. For instance, the force measurements with dozens of different drugs can be done simultaneously by implementing the wrinkle assay in a multiwell plate20. However, the interpretation of the wrinkle length was not necessarily straightforward in terms of quantitatively measuring the magnitude and direction of traction forces. Although the geometrical information of wrinkles21,22,23, such as wavelengths, would give an estimation for the force magnitude and direction, the geometry is still not enough to predict the local force distribution in a subcellular scale that is important to understand the cellular mechanotransduction and the morphological changes of cells. To overcome this limitation in quantification, here we describe a new machine learning system, wrinkle force microscopy (WFM), that converts the wrinkle information taken by a microscope into the actual cellular force distributions. For the initial training data, the cellular traction forces are obtained using the traction force microscopy (TFM), and we train the machine learning system with GAN (generative adversarial network) so that the network understands the way to convert the input microscope images to the force distributions from the training data. After sufficient training, the network can be utilized to predict the cellular forces just from the input images. The system would be a powerful tool to evaluate the cellular forces efficiently because the forces can be predicted by simply imaging the cells under the microscope, which is much simpler method compared to the TFM experiment.

Results

Full picture of WFM

Our goal is to construct a machine learning system that can predict the cellular force distributions from the microscope image or the extracted substrate wrinkles. The full process can be divided into three steps as shown in Fig. 1. First, we culture the cells (A7r5; embryonic rat vascular smooth muscle cells) on a silicone membrane substrate and measure both the cellular traction force and the substrate wrinkles simultaneously. As shown in Fig. 1(a), the cellular traction forces are obtained using TFM24,25, and cells generate wrinkles because the surface of PDMS (polydimethyl siloxane) layer is hardened by the plasma irradiation16,19,20,26,27. Second, the wrinkle positions are extracted from the microscope images as shown in Fig. 1(b) by using our SW-UNet (small world U-Net)19, which is a convolutional neural network (CNN) that reflects the concept of the small world networks28,29. Third, the machine learning system utilizing GAN30 is trained to understand a way to convert the microscope image, or the extracted wrinkle image, to the cellular force distributions as shown in Fig. 1(c). After the training, the network can be utilized to predict the cellular forces just from the microscope images as shown in Fig. 1(d).

Fig. 1: Overview of the methods and procedures that are utilized in the wrinkle force microscopy (WFM).
figure 1

a Schematic of our experimental setup. A silicone membrane, which can evaluate the cellular force distribution (obtained by TFM) and the surface wrinkles simultaneously, is utilized in this work. b The surface wrinkles are extracted from the microscope images by using our machine learning system (SW-UNet). c The machine learning system (GAN) is trained to understand the relation between the input images (raw microscope image, or extracted wrinkle images) and corresponding output images (cellular force distribution). d After sufficient training, WFM can predict the force distributions only from the microscope images.

Full size image

Simultaneous measurement of wrinkles and traction forces

Before applying the machine learning system, we begin by considering the results of the simultaneous force and wrinkle characterization. Figure 2 summarizes representative results obtained by the experiment and analysis. Due to the pairwise inward pulling generated by cellular traction force (fourth column), the substrates exhibit displacements toward the cell center (third column). As the result of the contraction, the wrinkles emerge mostly underneath the cells (second column). When the cell size is small, the majority of wrinkles are aligned in a same direction as in Fig. 2(a), while they tend to point in different directions when the cell size is large and the traction is strong as in Fig. 2(c).

Fig. 2: Three examples of the simultaneous measurement of wrinkles and traction forces.
figure 2

Each column a–c describes (from left to right) raw image, wrinkles (red lines), displacement field and traction force field. The white scale bar in the first column images is 20 μm. The blue and green lines inside the second column images describe the principal direction of the wrinkle and the traction, respectively.

Full size image

Figure 3(a) shows the probability distribution function (PDF) of the traction magnitude of N × M samples, where N = 103 is the number of the images and M = 26 × 26 is the number of the force observation points. The average traction is 50.3 ± 57.1 [Pa] (mean ± standard deviation). Figure 3(b) shows that the wrinkle length has a positive correlation with the mean traction of the images, which is in agreement with our previous experimental measurements26, where the relationship between the wrinkle length and applied force was experimentally investigated using microneedles. The mean traction is simply obtained by averaging the norm of the traction of the image as

$$\bar{f}=\frac{1}{M}\mathop{\sum }\limits_{m}^{M}| {{{{{{{{\boldsymbol{f}}}}}}}}}_{m}|$$
(1)

where m is the index of the observation points. The wrinkle length is measured by counting the number of pixels after skeletonizing the wrinkle images26. The wrinkle extincts when the mean traction in an image is less than 10 Pa, which is comparable to the noise level or the resolution of the current TFM.

Fig. 3: Quantitative analysis of the traction forces and wrinkles.
figure 3

a Probability distribution function (PDF) of the traction magnitude. b The wrinkle length as the function of the mean traction \(\bar{f}\). Note that the wrinkle length is evaluated by counting the number of pixels after skeletonizing the wrinkle images. c The wrinkle length as the function of the principal traction fp. d Probability distribution function of the angle differences between the wrinkle direction ϕw and the traction ϕs. The figure suggests that the direction of the wrinkles is predominantly perpendicular to the principal direction of the force.

Full size image

In order to analyze the principal direction of the traction, we construct a symmetric stress tensor for each image as

$${S}_{ij}=\frac{1}{2M}\mathop{\sum }\limits_{m}^{M}\big\{{n}_{j}({{{{{{{{\boldsymbol{x}}}}}}}}}_{m},{{{{{{{{\boldsymbol{x}}}}}}}}}_{0}){f}_{i}({{{{{{{{\boldsymbol{x}}}}}}}}}_{m})+{n}_{i}({{{{{{{{\boldsymbol{x}}}}}}}}}_{m},{{{{{{{{\boldsymbol{x}}}}}}}}}_{0}){f}_{j}({{{{{{{{\boldsymbol{x}}}}}}}}}_{m})\big\}$$
(2)

where f = (fx, fy) is the traction force, r = xm − x0 is the relative vector from the image center x0 and n = r/∣r∣ is the normal vector. By diagonalizing the tensor, we obtain the principal direction of the traction ϕs (shown in Fig. 2, second column with green lines on the bottom right corner) together with the corresponding principal traction magnitude fp, from the eigenvalue that has the largest norm. At the same time, we obtain the principal direction of the wrinkles ϕw (also shown in Fig. 2, second column with blue lines on the bottom right corner) from the 2D-FFT (fast Fourier transform) image of the wrinkles: ϕw is an angle that is perpendicular to the direction that has a largest power spectrum. Figure 3(c) shows that the traction force is contractile (fp < 0) and is almost linearly related to the wrinkle length (correlation R = −0.82 in a range fp < −5 Pa). While earlier work18,26 predicted a linear relationship between the traction force and the wrinkle length, analytical treatment21 showed that the relationship should be quadratic. The linear relation that is measured here validates a more recent theoretical prediction based on far-from threshold theory of wrinkling31 and is consistent with the experiments on droplets on polystyrene films32. As such, the wrinkles can be used for one qualitative marker or indicator for rough estimation of the cellular traction magnitude. Figure 3(d) shows that the two angles ϕs and ϕw are perpendicular most of the time. Since the wrinkle direction is perpendicular to that of the force dipoles, the wrinkle would be also practical to qualitatively predict the force directions, as previously done elsewhere16,18. Therefore, the length and direction of wrinkles provide a qualitative measure of the magnitude and direction of forces exerted by cells on the substrate, respectively.

Topological features of the wrinkle also give us insights into the force distributions. As shown in Fig. 4(a), some cells exhibit wrinkles in a single region (top) while the others show several separated regions of wrinkles (bottom). By categorizing the cell images (total 34 images) into two types as clustered patterns (9 images) and dispersed patterns (25 images), we summarize the difference in the force distributions in Fig. 4(b, c): the average force \(\bar{f}\) and force isotropy I are larger (significant difference only for \(\bar{f}\): p = 1.15 × 10−2 < 0.05, I: p = 1.42 × 10−1) for clustered pattern, where isotropy is evaluated as \(I=| {f}_{p}/{f}_{p}^{\min }|\) and \({f}_{p}^{\min }\) is the smaller eigenvalue of Sij. The wrinkles are generated by the pairwise forces that are transmitted via focal adhesions. When the wrinkles are dispersed, cells tend to be elongated, as seen in the pictures, and we conjecture that the cells might not be strong enough to generate continuous wrinkles as the clustered ones. This result suggests that the topological features of the wrinkles contain rich information about the force distributions.

Fig. 4: Topological features of wrinkles relate to force distributions.
figure 4

a Three examples of cells that exhibit clustered wrinkles or dispersed wrinkles.  b The average force \(\bar{f}\) and c force isotropy I  for two groups of cells; cells that exhibits clustered or dispersed wrinkles. Note that asterisk symbols denote the significance p < 0.05 (*); two-sided unpaired t-test. The data numbers are n = 9 (cluster) and 25 (dispersion), respectively. The box plots describe minimum, first quartile, meadian, third quartile and maximum. Black dot is an outlier.

Full size image

Finally, the correlations between cellular properties and characters of contractile forces are summarized in Table 1. Since each variable has moderate positive correlations (~0.5) with all other variables, it can be concluded that the cells are circular when the size is larger, and both force magnitude and isotropy increase with the cell size. Taken together, these results show that wrinkles can provide qualitative information about the magnitude, direction, and distribution of traction forces that cells generate. Nevertheless, a quantitative relation between the wrinkles and traction forces is missing.

Table 1 Correlations between cellular properties (area, roundness) and force characters (force isotropy, magnitude).
Full size table

Traction force prediction using GAN

Next, we employ the machine learning-based approach to provide a quantitative measure of the forces from the microscope images of wrinkles. We train the network and evaluate the performance of the force estimation using our GAN network. Figure 5 compares the predicted force distributions which were estimated by the three different methods. As also shown in Fig. S1, we trained the network with two different input images, the raw microscope images (second column in Fig. 5) and the extracted wrinkle images (third column), to compare the performance. We also evaluated the force distribution using a standard encoder-decoder type CNN and show the results in the fourth column. The figure shows that all the three methods reproduce approximately the same force direction as the ground truth (first column), and the forces are perpendicular to the wrinkles.

Fig. 5: Prediction of the traction forces from microscope images.
figure 5

Each column a–c shows the result of the traction force predictions by using different methods (from left to right): ground truth, GAN prediction (input: microscope image), GAN prediction (input: extracted wrinkle images), and CNN prediction (input: microscope image). See further examples in Movies S1–S4.

Full size image

Figure 6(a) compares the traction of ground truth \({f}_{x,y}^{{{{{{{{\rm{true}}}}}}}}}\) and GAN prediction \({f}_{x,y}^{{{{{{{{\rm{predict}}}}}}}}}\) (input image: microscope images), and it shows that the prediction is highly correlated with the experimental data. Note that we used N = 252 training image sets and 3 test images for the evaluation. The total error is calculated by averaging the error of 15 test images, which are obtained by repeating the evaluation 5 times with randomly selected different test images. The correlation coefficient R averaging 15 test images is 0.84–0.88 for GAN and 0.82–0.84 for CNN as shown in Fig. 6(b), and it suggests that there are striking agreements. In order to further quantify the error in the force estimation, we introduce two errors: the error in the force magnitude εf and the force direction εθ. The error εf is defined to evaluate the difference in the force magnitude between the ground truth ftrue and the prediction fpredict as:

$${\varepsilon }_{f}=\frac{1}{M}\mathop{\sum }\limits_{m}^{M}\frac{| {f}_{m}^{{{{{{{{\rm{predict}}}}}}}}}-{f}_{m}^{{{{{{{{\rm{true}}}}}}}}}| }{{f}_{m}^{{{{{{{{\rm{true}}}}}}}}}}\cdot {\omega }_{m}$$
(3)

where M = 26 × 26 is the number of observation points, \({\omega }_{m}={f}_{m}^{{{{{{{{\rm{true}}}}}}}}}/\bar{f}\) is the weight function and \(\bar{f}\) is the average force in the image which is defined in Eq. (1). Note that we introduce this weight function in order to put weight on the evaluation of large vectors rather than small vectors, which give huge errors even for small differences. Figure 6(c) shows that the error is 38–41% for GAN, and it has better performance compared to the encoder-decoder type simple CNN, which has an error of 50%. There is no obvious difference in the two input images (microscope images and wrinkle images), and this result indicates that the performance of the force estimation would not improve drastically by explicitly teaching the wrinkle position to the machine learning system. A comparison of the distributions between the prediction and ground truth is shown in Supplementary Fig. 1. Next, we evaluate the angle difference between the predicted force and the ground truth as

$${\varepsilon }_{\theta }=\frac{1}{M}\mathop{\sum }\limits_{m}^{M}| {\theta }_{m}^{{{{{{{{\rm{predict}}}}}}}}}-{\theta }_{m}^{{{{{{{{\rm{true}}}}}}}}}| \cdot {\omega }_{m}$$
(4)

where \(\theta =\arctan ({f}_{y}/{f}_{x})\) is the force direction. Figure 6(d) shows that the errors are 19–23° for GAN, and again shows better performance compared to conventional CNN (εθ = 24–28°). Note that the spatial distribution of errors εf and εθ is shown in Supplementary Figs. 2 and 3. As for a traditional CNN, the loss function is designed to measure the error between predicted results and ground truth, and these criteria for the error are fixed during the training. In the case of GAN, the loss function can adapt to the specific problem dynamically because of the discriminator network, and this difference brings GAN a better score as shown in the figure. It is important to note that we have so far acquired a minimal required amount of training (original data: ~100) to demonstrate the novel concept of cellular force detection from microscope images. These errors will be minimized by increasing the number of the training data. As demonstrated above, WFM succeeded in estimating the force distribution just from the input images with limited levels of errors in real time. Supplementary movies S1–S4 further show the application of the proposed system in providing high throughout, real time measure of the traction force distributions during dynamic cell locomotion.

Fig. 6: Quantification of the errors in prediction.
figure 6

a Comparison of the ground truth \({f}_{x,y}^{{{{{{{{\rm{true}}}}}}}}}\) and the predicted traction \({f}_{x,y}^{{{{{{{{\rm{predict}}}}}}}}}\). Dashed line shows a condition fpredict = ftrue. Note that we randomly reduced the number of data point 1/5 for the visibility. b Correlation coefficient R between \({f}_{x,y}^{{{{{{{{\rm{true}}}}}}}}}\) and \({f}_{x,y}^{{{{{{{{\rm{predict}}}}}}}}}\). c, d Errors of the predicted traction compared to the ground truth data: c error in the traction magnitude εf and d the traction direction εθ. Note that bars show average errors ± standard deviations among the sample number n = 15, and gray dots indicate the original data. The description m.s. denotes the microscope images.

Full size image

Discussion

We proposed a machine learning-based system, WFM, that can predict cellular force distributions from microscope images. The full process can be divided into three steps. First, we culture the cells on a plasma-irradiated silicone substrate and measure both the cellular traction force and the substrate wrinkles simultaneously. The cellular traction forces are obtained using the TFM, while cells generate wrinkles on the underlying substrates. Second, the wrinkle positions are extracted from the microscope images by using SW-UNet. Third, we train the GAN system by using sets of corresponding two images, the force distributions and the input images (raw microscope images or extracted wrinkle images), as the training data. The network understands the way to convert the input images to the force distributions from the training. After sufficient training, the network can be utilized to predict the cellular forces just from the input images.

The relationship between morphology and biological functions of living creatures has long been an intense subject of research33. This topic has been investigated at the individual cellular level as well27,34,35, in which cellular contractile forces were implicated in diverse functions including proliferation, differentiation, apoptosis, and tumorigenesis. Given its complicated nature, however, the general relationship associated with cellular forces remains poorly understood. In this regard, the technology described here has a potential to drastically advance research in this field as it allows for easier acquisition of the cellular traction force data. Indeed, compared to elongated cells, we found a stronger tendency for circular cells to produce contractile forces that are more isotropic and are higher in magnitude. Thus, WFM is expected to be applicable, in addition to drug screening as we discuss below, for extensively probing how the forces generated by cells is related to their functions including maintenance of morphological phenotypes.

It is instructive to discuss limitations and advantages of the proposed approach. In this paper we have presented data for a stiffness range 5.4−16.3 kPa. In principal, it is recommended to do the training again for different stiffness in most of the cases, except limited cases described below. As we have described in the Methods section “Wrinkle mechanics”, conditions for the wrinkle generation would be identical if the stiffness ratio is the same for new experiment: \({E}_{p}^{0}/{E}_{m}^{0}={E}_{p}^{1}/{E}_{m}^{1}\), where Ep and Em denote the Young's modulus for the elastomer and the oxidized surface layer, respectively. E Therefore, the only thing one needs to modify is to multiply E1/E0 to the force that the WFM predicted for this case. When the stiffness ratio is not fixed \({E}_{p}^{0}/{E}_{m}^{0}\;\ne\; {E}_{p}^{1}/{E}_{m}^{1}\), WFM cannot directly predict the force distribution since the wavelength λ (Eq. [9]) and the critical strain for the wrinkle generation εc (Eq. [11]) are different for this condition. Although it might be still possible to estimate the strain considering the condition differences, it is still difficult to do the direct prediction as before. Additionally, since the wrinkle patterns are determined only from the traction distributions, we expect the same principle can be applied to other cell lines unless the cells have comparable sizes with those used in the training data. When the cell size is not comparable, the model should be trained once again since wrinkle patterns with similar length scale might not be included in the training data. Supplementary Fig. 4 also shows the distribution of traction force for MEF (Mouse embryonic fibroblast), which has a comparable size with A7r5 cells (training data).

Comparing with the TFM experiment (test data), the prediction using our system is highly correlated with the experimental data, with the averaged correlation coefficient of 0.84–0.88 and with 38–41% errors in the force magnitude prediction and angle errors 19–23° in the force direction. We expect that this error would decrease further by increasing the number of training images. The WFM would be a powerful tool to evaluate the cellular forces efficiently because the forces can be predicted just by observing the cells, which is much simpler method compared to performing the TFM experiment every time needed.

TFM is one of the most used methods to evaluate the cellular forces in mechanobiology study, but as the accuracy of the measurement depends on the successful acquisition of the reference positions of the micro-beads that are obtained by removing the cells after each of the experiments in conventional TFM, this method is limited in throughput. The novel GAN-based system proposed here overcomes this limitation as it provides the nearly same information, with the high levels of the correlations with the experimental data and the limited levels of the errors on the cell mechanics. Importantly, this is achieved only from the still images that are acquired by plating the cells on the silicone substrate without taking care of the reference as the substrate surface is known to become planar again upon the absence of the cellular forces in a reversible manner. Given that early stages of drug screening require testing of a massive number of candidate compounds20, our system, with the potentially high-throughput data analysis capability, will be useful particularly in such screening studies. It is important to note that our WFM is not the one that essentially competes with TFM, but the huge advantage of the proposed system is focused on its capability to provide data equivalent to the TFM (with a level of the errors) and thereby circumvent performing the TFM that needs considerable technical care. Rather, because the machine learning system depends on the training data, further innovations in TFM such as super-resolution imaging36,37 are potentially introduced to our system to synergetically output more sophisticated data. Thus, our approach presents a versatile framework that integrates the sophisticated experimental techniques and the efficient measurements.

Materials and Methods

Step 1: simultaneous measurement of traction forces and wrinkles

Based on our previous studies16,19,20,26,27, we prepared the substrate that can reversibly generate wrinkles upon application of cellular forces. First, a circular cover glass is treated with oxygen plasma (SEDE-GE, Meiwafosis) to hydrophilize the surface and is desiccated after fluorescent micro-beads (0.2 μm in diameter, carboxylate yellow-green fluorescent beads; Invitrogen) in water solution are distributed on the surface. Second, parts A and B of CY 52-276 (Dow Corning Toray) are mixed at a weight ratio of 1.2:1 and poured onto the cover glass to create a PDMS layer with a height of 30–40 μm. Third, the cover glass is placed in a 60 °C oven for 20 h to cure the PDMS. Fourth, oxygen plasma is applied uniformly along the surface of the PDMS layer to create an oxide layer that works as the substrate for cell culture. Finally, the substrate is coated with 10 μg/mL collagen type I solution for 3 h.

For the TFM measurement, fluorescent micro-beads are attached to the substrate surface as position markers to measure the substrate deformations. The beads need to be firmly adhered to the surface so that cells would not move the beads due to endocytosis. In this work, the covalent bonding between the surface and the beads of 0.001% v/v are performed by following two steps: (i) silane coupling of the substrate surface using 3-Aminopropyltrimethoxysilane and (ii) the covalent bonding formation due to carbodiimide. The beads adhered on the glass surface are monitored to keep the reference position even after removing the cell using 0.25% Trypsin (Trypsin + 1mm mmol/I EDTA-4Na solution; Fuji Wako Pure Chemical Corporation).

Cell culture and microscope setup

A7r5 cells (purchased from ATCC) were maintained at 37 °C in a stage incubator (INUF-IX3W; Tokai Hit) under a humidified 5% CO2 incubator. An inverted microscope (1X73; Olympus) with a confocal unit (CSU10; Yokokawa Electric) and oil immersion lens (phase contrast, UPlanFLN 60x/1.25 Oil Iris Ph3, Olympus Corporation) are used to capture the cells and fluorescent beads. During the experiment, DMEM(L)+10% FBS+Penicillin-Streptomycin (Fuji Wako Pure Chemical Corporation) is used as the culture medium.

Note that the proliferation (Supplementary Fig. 5) and migration (Supplementary Fig. 6) with three different substrates (CY with mixed ratio 1.2:1, CY with 1.0:1 and glass) are evaluated, and we confirmed that the wrinkles have no fatal or harmful effect on the cell nature.

Traction force microscopy (TFM)

The software ImageJ/Fiji38 and its plugin FTTC (Fourier transform traction cytometry)39,40 are used to evaluate the force field from the displacement field. The substrate is considered as a soft elastic isotropic material that follows the linear elastic theory. First, the displacement of the substrate surface u is measured by tracking the movement of the fluorescent beads using PIV (particle image velocimetry). The spatial resolution of the force distribution is 3.44 μm × 3.44 μm. Second, the traction field is obtained from the displacement field by solving the governing equation for the elastic halfspace41,42 given by

$${{{{{{{\boldsymbol{u}}}}}}}}({{{{{{{\boldsymbol{x}}}}}}}})={\int}_{S}{{{{{{{\boldsymbol{G}}}}}}}}({{{{{{{\boldsymbol{x}}}}}}}},{{{{{{{\boldsymbol{y}}}}}}}}){{{{{{{\boldsymbol{t}}}}}}}}({{{{{{{\boldsymbol{y}}}}}}}})dS({{{{{{{\boldsymbol{y}}}}}}}})$$
(5)

where t is the traction force, x and y are the positions of the displacement and the traction force, respectively. G is the Green’s function that is given by

$${{{{{{{\boldsymbol{G}}}}}}}}({{{{{{{\boldsymbol{x}}}}}}}})=\frac{1+\nu }{\pi E{r}^{3}}\left(\begin{array}{ll}(1-\nu ){r}^{2}+\nu {r}_{x}^{2}&\nu {r}_{x}{r}_{y}\\ \nu {r}_{x}{r}_{y}&(1-\nu ){r}^{2}+\nu {r}_{y}^{2}\end{array}\right)$$
(6)

where E is the Young’s modulus, ν is the Poisson’s ratio, r = (rx, ry) = x − y is the relative position vector and r = ∣r∣. The software FFTC solves Eq. (5) in the Fourier space, which is given by

$$\tilde{{{{{{{{\boldsymbol{t}}}}}}}}}={({{{{{{{{\boldsymbol{G}}}}}}}}}^{T}{{{{{{{\boldsymbol{G}}}}}}}}+{\lambda }^{2}{{{{{{{\boldsymbol{I}}}}}}}})}^{-1}{{{{{{{{\boldsymbol{G}}}}}}}}}^{T}\tilde{{{{{{{{\boldsymbol{u}}}}}}}}}$$
(7)

where tilde symbols denote the variables in Fourier space, λ is the regularization parameter42 and I is the unit tensor. In order to evaluate the optimal parameter λ for the Tikhonov regularization, the L-curve criterion42,43 is applied. Note that E is experimentally determined16 to be 5400 Pa and ν is assumed 0.5 (incompressible) that is a typical value for PDMS material.

Step 2: Wrinkle extraction

We use a method SW-UNet19, which is a CNN based on U-Net44 to extract wrinkle patterns from the microscope image as shown in Fig. 1(b). As the training data, we prepare 236 sets of corresponding two images (microscope image and manually labeled wrinkle image). The number of data is increased to 2596 by using the image augmentation techniques. We used NVIDIA Titan RTX to accelerate the training process, and the Adam optimizer is utilized. Note that the procedure until Step 2 was already developed and utilized in our previous papers19,20, and GAN-based force estimation is newly presented in this work.

Step 3: prediction of traction force based on GAN-based system

Assume that we have No sets of corresponding images and data; the input images (microscope images, or extracted wrinkle images) and the force distributions as shown in Supplementary Fig. 7. We effectively have the number of training data set 2No because the wrinkle image has only 1D information at each pixel (intensity I(x, y); see also images in Fig. S1) while the force distributions have 2D information (2D force, f(x, y) = {fx(x, y), fy(x, y)}). We designed the network to evaluate the cellular force only for a single axis at one time and focus only on the x-directional force at each evaluation. An input image Ii is used as the training data set (Ii, fx) and (\({I}_{i}^{\prime}\), fy) where \({I}_{i}^{\prime}\) is an image that rotates Ii by 90 degrees.

The force distributions are converted to gray scale images that have intensities

$$I({f}_{d})=a\arctan \left(\frac{{f}_{d}}{b}\right)+{I}_{{{{{{{{\rm{mid}}}}}}}}}$$
(8)

where a = 81.2, b = 50.0, Imid = 255/2 are the coefficients for the conversions, and fd is the components of the force d = x, y. The force distributions in grayscale, which are generated from the test images, can be converted back to the force using this equation. As the training data, we prepared N = 252 sets (63 original images) of corresponding two images. Note that 63 images are acquired from experiments on five different days and seven different dishes. We increased the number of training data by rotating the images: 63 × 4 = 252. Python programs using TensorFlow and training data are provided in the following Github link https://github.com/Minatsukiyoshino/Wrinkle_force_microscopy.

Wrinkle mechanics

The wrinkle mechanics22 can be discussed by simplifying the current substrate as an elastomer (Young’s modulus: Em) with a stiff oxidized surface layer (Young’s modulus Ep, thickness h). Note that the Poisson’s ratio ν of both materials are considered to be the same for simplicity. Substrate compression results in a sinusoidal buckling with a wavelength λ and amplitude A as

$$\frac{\lambda }{h}=2\pi {\left(\frac{{E}_{p}}{3{E}_{m}}\right)}^{1/3},$$
(9)
$$\frac{A}{h}={\left(\frac{\varepsilon }{{\varepsilon }_{c}}-1\right)}^{1/2}$$
(10)

where εc is the critical strain that the wrinkle emerges, which is defined as

$${\varepsilon }_{c}=-\frac{1}{4}{\left(\frac{3{E}_{m}}{{E}_{p}}\right)}^{2/3}.$$
(11)

The wavelength in our present system typically in a range λ = 2−4μm while the thickness of the oxidized layer h is assumed to be in an order of 100 nm45. By assuming the ratio as λ/h ~ 30, we can give estimations on the stiffness ratio as Ep/Em ~ 300 and the critical strain as εc ~ − 0.01. Note if one needs to observe 20 wrinkles for each cell with a size ~100 μm, the stiffness ratio should be Ep/Em ≤ 1500 from Eq. [(9)]. Since the maximum strain is nearly ε = −0.05 in the experiment, the maximum amplitude of wrinkles is assumed to be A ~ 200 nm.

If the substrate stiffness is different from the trained condition, the system needs to be trained once again with a new set of data. However, the trained system is still applicable when the stiffness ratio Ep/Em of a new substrate is the same as the trained condition since the criteria for wrinkle generation would be identical, as shown in Eqs. [(9)] and [(11)]. By multiplying the ratio of Young’s modulus, we can estimate the force distribution for this case.

GAN structure

Our goal is to convert a physical quantity (wrinkle geometry) to another physical quantity (force distribution). Even though the mechanical formulation between the two quantities is given, it is not necessarily straightforward to solve this inverse problem because of its complex nonlinear dynamics21,46. Instead, we achieve this purpose by training our machine learning system to understand the underlying mechanical rules. Considering this conversion of the physical quantity as an image “translation”, we utilize GAN (generative adversarial network)47 in this work.

GAN mainly consists of two networks, generator G and discriminator D as shown in Fig. S1, and the network is trained by a competition of two networks. Goal of the generator G is to generate fake images (fake force distributions G(x)) from the input images x (microscope images or extracted wrinkle images) and tries to mimic the real images y (real force distributions), while the discriminator D tries to distinguish true and fake images from the group of images. As the training proceeds, the generator learns how to produce fake images that are difficult to be distinguished by the discriminator from the real images, and the discriminator learns the rules to distinguish true/fake images. Once the training is completed, the trained generator G can now be used as the translator to predict the force distribution from the input images x even for test images, which were not included in the training process.

In the present work, we design the generator G with a form of encoder-decoder which is based on U-Net44, and Markovian discriminator (PatchGAN)48 is utilized as the discriminator D. The generator G converts the input images x to the fake force distribution images G(x). The images of force distribution, y or G(x), and the input images x are concatenated into a single image as the input of the discriminator. The two networks G and D are trained based on the labels of real/fake, and we utilize the loss function \({{{{{{{\mathcal{L}}}}}}}}\) that is used in pix2pix30:

$${{{{{{{\mathcal{L}}}}}}}}(G,D)={{\mathbb{E}}}_{x,y}[\log D(x,y)]+{{\mathbb{E}}}_{x}[\log (1-D(x,G(x)))]+\lambda {L}_{1}(y,G(x))$$
(12)

where \({\mathbb{E}}\) is the expected value, L1(G) is the L1 distance between generated images G(x) and ground truths y and λ = 100 is the weight for the L1 term. The first term \({\mathbb{E}}[\log D]\) denotes the expected probability that the discriminator categorizes y as the real data, while the second term \({\mathbb{E}}[\log (1-D(x,G(x)))]\) denotes the probability that the discriminator categorizes the generated image G(x) as the fake data. The goal of the generator G is to minimize \({{{{{{{\mathcal{L}}}}}}}}\) while the discriminator D tries to maximize it.

We use the training parameters as follows: 100 training epochs (batch size = 1), ε = 0.0002 learning rate for generator and discriminator, the parameters β1 = 0.5 and β2 = 0.9 are used for the Adam optimizer. The whole learning process is again accelerated by Nvidia Titan RTX.

Statistics and reproducibility

66 cell images and corresponding force distributions are collected from the experiments of 5 different days and 7 different independently prepared dishes. The test data (3 cells) are randomly selected from the dataset, and all others (63 cells) are utilized as the training data. By changing the test data, the training and error evaluation were repeated 5 times. The sample numbers n for each analysis are listed in the figure caption. The p value of the unpaired t-test is evaluated with Microsoft Excel.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.