Wrinkle force microscopy: a new machine learning based approach to predict cell mechanics from images
Honghan Li, Daiki Matsunaga, Tsubasa S. Matsui, Hiroki Aosaki, Koki Inoue, Amin Doostmohammadi, Shinji Deguchi
DD R A F T Wrinkle force microscopy: a new machine learningbased approach to predict cell mechanics fromimages
Honghan Li a , Daiki Matsunaga a , Tsubasa S. Matsui a , Hiroki Aosaki a , Koki Inoue a , Amin Doostmohammadi b,a , and ShinjiDeguchi a a Division of Bioengineering, Graduate School of Engineering Science, Osaka University, 1-3 Machikaneyama Toyonaka, Osaka, 5608531, Japan; b Niels Bohr Institute,University of Copenhagen, Blegdamsvej 17, 2100 Copenhagen, DenmarkThis manuscript was compiled on February 25, 2021
Combining experiments with artificial intelligence algorithms, wepropose a new machine learning based approach to extract the cellu-lar force distributions from the microscope images. The full processcan be divided into three steps. First, we culture the cells on a spe-cial substrate allowing to measure both the cellular traction forceon the substrate and the corresponding substrate wrinkles simulta-neously. The cellular forces are obtained using the traction forcemicroscopy (TFM), at the same time that cell-generated contractileforces wrinkle their underlying substrate. Second, the wrinkle posi-tions are extracted from the microscope images. Third, we train themachine learning system with GAN (generative adversarial network)by using sets of corresponding two images, the traction field and theinput images (raw microscope images or extracted wrinkle images),as the training data. The network understands the way to convertthe input images of the substrate wrinkles to the traction distributionfrom the training. After sufficient training, the network is utilized topredict the cellular forces just from the input images. Our systemprovides a powerful tool to evaluate the cellular forces efficiently be-cause the forces can be predicted just by observing the cells underthe microscope, which is a way simpler method compared to the TFMexperiment. Additionally, the machine learning based approach pre-sented here has the profound potential for being applied to diversecellular assays for studying mechanobiology of cells. cell mechanics | traction force microscopy | GAN T here is now growing evidence showing that cells sensemechanical cues in the surrounding microenvironment toregulate their functions such as proliferation, differentiation,apoptosis, and pro-inflammation (1–6). In response to themechanical cues, cells often adjust their cytoskeletonal tensionand as such many of the mechanical information are translatedinto a level of inherent cellular traction forces, and in turninto intracellular signals regulating the related functions (3, 7–9). Traction forces, thus related to various cell functions,are generated by the activity of nonmuscle myosin II andactin filaments that determine cellular contractility (2, 10–12).Because these proteins work downstream of diverse signalingpathways, it is often difficult to predict how the force maychange upon perturbations to particular molecules such asgene mutations and drug administration. Thus, technologiesallowing for efficiently evaluating the cellular traction force areexpected to appear to enhance comprehensive understandingof the force-related pathways.We previously developed a wrinkle assay, a modified versionof the method originally reported by Harris and colleagues(13, 14), in which the silicone substrates are spatially treated with uniform oxygen plasma to allow them to buckle uponthe forces exerted by cells (15–17). As the individual wrin-kles are lengthened with the increase in the forces (18), thewrinkle length, detected for example by a machine learningapproach (19), can be used as a measure of the relative changein the force caused by perturbations such as specific genemutations. While this technology is promising in that theseexperiments are performed easily to potentially enable a high-throughput analysis on the force-related pathways, the inter-pretation of the wrinkle length was not necessarily straightfor-ward in terms of quantitatively measuring the magnitude anddirection of traction forces.To overcome this limitation in quantification, here we de-scribe a new machine learning system that converts the wrinkleinformation taken by a microscope into the actual cellular forcedistributions. For the initial training data, the cellular tractionforces are obtained using the traction force microscopy (TFM),and we train the machine learning system with GAN (gener-ative adversarial network) so that the network understandsthe way to convert the input microscope images to the forcedistributions from the training data. After sufficient training,the network can be utilized to predict the cellular forces justfrom the input images. The system would be a powerful tool Significance Statement
Cell-generated forces are indispensable determinants of fun-damental cell functions such as motility and cell division. Assuch, quantifying how the forces change upon perturbationsto the cells such as gene mutations and drug administrationis of profound importance. Here we present a novel machinelearning based system that allows for efficient estimations ofthe forces that are determined only by “observing” microscopeimages. Given that the cellular traction forces are regulateddownstream of diverse signaling pathways, our system – thathelps significantly improve the throughput of the measurements– presents a new, high throughput platform for real time analy-sis of the effects of a massive number of genetic and molecularperturbations on the forces and resulting cell mechanics.
H.L. and D.M developed the machine learning system, and K.I. supported the implementation.H.A. and T.S.M. designed and worked on the cell experiments. H.L., H.A. and D.M. analyzedthe experimental data. D.M., A.D. and S.D. conceived the idea of simultaneous TFM with wrinkleextraction. D.M. and A.D. analyzed the physics. H.L., D.M., A.D. and S.D. wrote the article anddesigned the research.The authors declare no competing interests. To whom correspondence may be addressed. E-mail: [email protected],[email protected]
XXX |
February 25, 2021 | vol. XXX | no. XX | a r X i v : . [ phy s i c s . b i o - ph ] F e b R A F T glass cell sustainedcontraction (a)(c) (b) oxide layer fluorescent beads substratePDMS stress field wrinkles (d) microscop images CNN (SW-UNet) GAN
GAN training extracted wrinkles
GAN prediction (generator) or or e Fig. 1.
Overview of the methods and procedures that are utilized in the wrinkle-force microscopy (WFM). (a) Schematic of our experimental setup. A silicone membrane, whichcan evaluate the cellular force distribution (obtained by TFM) and the surface wrinkles simultaneously, is utilized in this work. (b) The surface wrinkles are extracted from themicroscope images by using our machine learning system (SW-UNet). (c) The machine learning system (GAN) is trained to understand the relation between the input images(raw microscope image, or extracted wrinkle images) and corresponding output images (cellular force distribution). (d) After sufficient training, the system can predict the forcedistributions only from the microscope images. to evaluate the cellular forces efficiently because the forcescan be predicted just by observing the cells, which is a waysimpler method compared to the TFM experiment.
Full picture of the system
Our goal is to construct a machine learning system that canpredict the cellular force distributions from the microscopeimage or the extracted substrate wrinkles. The full process canbe divided into three steps as shown in Fig. 1. First, we culturethe cells (A7r5; embryonic rat vascular smooth muscle cells) ona silicone membrane substrate and measure both the cellulartraction force and the substrate wrinkles simultaneously. Asshown in Fig. 1(a), the cellular traction forces are obtainedusing TFM (20, 21), and cells generate wrinkles because thesurface of PDMS (polydimethyl siloxane) layer is hardened bythe plasma irradiation (16, 19, 22–24). Second, the wrinklepositions are extracted from the microscope images as shownin Fig. 1(b) by using our SW-UNet (small world U-Net) (19),which is a convolutional neural network (CNN) that reflectsthe concept of the small world networks (25, 26). Third, themachine learning system utilizing GAN (27) is trained tounderstand a way to convert the microscope image, or theextracted wrinkle image, to the cellular force distributions asshown in Fig. 1(c). After the training, the network can beutilized to predict the cellular forces just from the microscopeimages.
Simultaneous measurement of wrinkles and tractionforces
Before applying the machine learning system, we begin byconsidering the results of the simultaneous force and wrinkle characterization. Figure 2 summarizes representative resultsobtained by the experiment and analysis. Due to the pairwiseinward pulling generated by cellular traction force (fourthcolumn), the substrates exhibit displacements toward the cellcenter (third column). As the result of the contraction, thewrinkles emerge mostly underneath the cells (second column).When the cell size is small, the majority of wrinkles are alignedin a same direction as in Fig. 2(a), while they tend to point indifferent directions when the cell size is large and the tractionis strong as in Fig. 2(c).Figure 3(a) shows the probability distribution function(PDF) of the traction magnitude of N × M samples, where N = 103 is the number of the images and M = 26 ×
26 is thenumber of the force observation points. The average tractionis 50.3 ± ± standard deviation). Figure 3(b)shows that the wrinkle length has a positive correlation withthe mean traction of the images, which is in agreement withour previous experimental measurements (22), where the re-lationship between the wrinkle length and applied force wasexperimentally investigated using microneedles. The meantraction is simply obtained by averaging the norm of thetraction of the image as¯ f = 1 M M X m | f m | [1]where m is the index of the observation points. The wrinklelength is measured by counting the number of pixels afterskeletonizing the wrinkle images (22). The wrinkle extinctswhen the mean traction in a image is less than 10 Pa, which iscomparable to the noise level or the resolution of the currentTFM. | Li et al. R A F T Fig. 2.
Three examples of the simultaneous measurement of wrinkles and traction forces. Each column describes (from left to right) raw image, wrinkles (red lines), displacementfield and traction force field. The white scale bar in the first column images is µ m . The blue and green lines inside the second column images describe the principal directionof the wrinkle and the traction, respectively. In order to analyze the principal direction of the traction,we construct a symmetric stress tensor for each image as S ij = 12 M M X m { n j ( x m , x ) f i ( x m ) + n i ( x m , x ) f j ( x m ) } [2]where r = x m − x is the relative vector from the image center x and n = r / | r | is the normal vector. By diagonalizing thetensor, we obtain the principal direction of the traction φ s (shown in Fig. 2, second column with green lines) togetherwith the corresponding principal traction magnitude f p , fromthe eigenvalue that has the largest norm. At the same time, weobtain the principal direction of the wrinkles φ w (also shownin Fig. 2, second column with blue lines) from the 2D-FFT(fast Fourier transform) image of the wrinkles: φ w is an an-gle that is perpendicular to the direction that has a largestpower spectrum. Figure 3(c) shows that the traction forceis contractile ( f p <
0) and is almost linearly related to thewrinkle length (correlation R = − .
82 in a range f p < − φ s and φ w areperpendicular most of the time. Since the wrinkle direction isperpendicular to that of the force dipoles, the wrinkle wouldbe also practical to qualitatively predict the force directions,as previously done elsewhere (16, 18). Therefore, the lengthand direction of wrinkles provide a qualitative measure ofthe magnitude and direction of forces exerted by cells on thesubstrate, respectively. Next, we employ the machine learningbased approach to provide a quantitative measure of the forcesfrom the microscope images of wrinkles. Traction force prediction using GAN
Finally, we train the network and evaluate the performance ofthe force estimation using our GAN network. Figure 4 com-pares the predicted force distributions which were estimatedby the three different methods. As also shown in Fig. S1,we trained the network with two different input images, theraw microscope images (second column in Fig. 4) and theextracted wrinkle images (third column), to compare the per-formances. We also evaluated the force distribution using astandard encoder-decoder type CNN and show the results in Li et al. XXX |
February 25, 2021 | vol. XXX | no. XX || vol. XXX | no. XX |
February 25, 2021 | vol. XXX | no. XX || vol. XXX | no. XX | R A F T (a) P D F f [Pa] (b) w r i n k l e l e n g t h [ µ m ] ¯ f [Pa] (c) w r i n k l e l e n g t h [ µ m ] f p [Pa] (d) P D F | φ s − φ w | /π Fig. 3.
Quantitative analysis of the traction forces and wrinkles. (a) Probability distribution function (PDF) of the traction magnitude. (b) The wrinkle length as the function of themean traction ¯ f . Note that the wrinkle length is evaluated by counting the number of pixels after skeletonizing the wrinkle images. (c) The wrinkle length as the function of theprincipal traction f p . (d) Probability distribution function of the angle differences between the wrinkle direction φ w and the traction φ s . The figure suggests that the direction ofthe wrinkles is predominantly perpendicular to the principal direction of the force. the fourth column. The figure shows that all the three meth-ods reproduce approximately the same force direction as theground truth (first column), and the forces are perpendicularto the wrinkles.Figure 5(a) compares the traction of ground truth f true x,y andGAN prediction f predict x,y (input image: microscope images),and it shows that the prediction is highly correlated with theexperimental data. The correlation coefficient R is evaluatedfor all 15 test images and averaged correlations, 0.86-0.88 forGAN and 0.83-0.84 for CNN as shown in Fig. 5(b), suggestthat there are striking agreements. In order to further quantifythe error in the force estimation, we introduce two errors: theerror in the force magnitude ε f and the force direction ε θ .The error ε f is defined to evaluate the difference in the forcemagnitude between the ground truth f true and the prediction f predict as: ε f = 1 M M X m | f predict m − f true m | f true m · ω m [3]where M = 26 ×
26 is the number of observation points, ω m = f true m / ¯ f is the weight function and ¯ f is the average forcein a image which is defined in Eq. (1). Note that we introducethis weight function in order to put weight on the evaluationof large vectors rather than small vectors, which give hugeerrors even for small differences. We used N = 332 trainingimage sets and 3 test images for the evaluation. The total error is calculated by averaging the error of 15 test images,which are obtained by repeating the evaluation 5 times withrandomly selected different test images. Figure 5(c) shows thatthe error is 33-35% for GAN, and it has better performancecompared to the encoder-decoder type simple CNN, which hasan error 46%. There is no significant difference by the twoinput images (microscope images and wrinkle images), andthis result indicates that performance of the force estimationwould not improve drastically by explicitly teaching the wrinkleposition to the machine learning system. Next, we evaluate theangle difference between the predicted force and the groundtruth as ε θ = 1 M M X m | θ predict m − θ true m | · ω m [4]where θ = arctan( f y /f x ) is the force direction. Figure 5(d)shows that the errors are 19-20 ◦ for GAN, and again shows bet-ter performance compared to conventional CNN ( ε θ = 23-24 ◦ ).As for a traditional CNN, the loss function is designed to mea-sure the error between predicted results and ground truth, andthis criteria of the error are fixed during the training. Whilein the case of GAN, the loss function can adapt to the specificproblem dynamically because of the discriminator network,and this difference brings GAN a better score as shown in thefigure. It is important to note that we have so far acquireda minimal required amount of training (original data: ∼ | Li et al. R A F T Fig. 4.
Prediction of the traction forces from microscope images. Each column shows the result of the traction force predictions by using different methods (from left to right):ground truth, GAN prediction (input: microscope image), GAN prediction (input: extracted wrinkle images), and CNN prediction (input: microscope image). See furtherexamples in Movies S1-S4. to demonstrate the novel concept of cellular force detectionfrom microscope images. These errors will be minimized byincreasing the number of the training data. As demonstratedabove, our system succeeded in estimating the force distribu-tion just from the input images with limited levels of errors,in real time. Movies S1-S4 further show the application ofthe proposed system in providing high throughout, real timemeasure of the traction force distributions during dynamic celllocomotion.
Discussion
We proposed a new machine learning based system that canpredict the cellular force distributions from the microscopeimages. The full process can be divided into three steps.First, we culture the cells on a plasma-irradiated siloconesubstrate and measure both the cellular traction force and thesubstrate wrinkles simultaneously. The cellular traction forcesare obtained using the TFM, while cells generate wrinkleson the underlying substrates. Second, the wrinkle positionsare extracted from the microscope images by using SW-UNet.Third, we train the GAN system by using sets of correspondingtwo images, the force distributions and the input images (raw microscope images or extracted wrinkle images), as the trainingdata. The network understands the way to convert the inputimages to the force distributions from the training. Aftersufficient training, the network can be utilized to predict thecellular forces just from the input images. Comparing withthe TFM experiment (test data), the prediction using oursystem is highly correlated with the experimental data, withthe averaged correlation coefficient of 0.86-0.88 and with 33-35% errors in the force magnitude prediction and angle errors19-20 ◦ in the force direction. We expect that this error woulddecrease further by increasing the number of training images.The system would be a powerful tool to evaluate the cellularforces efficiently because the forces can be predicted just byobserving the cells, which is a way simpler method comparedto performing the TFM experiment every time needed.TFM is one of the most used methods to evaluate thecellular forces in mechanobiology study, but as the accuracyof the measurement depends on successful acquisition of thereference positions of the micro-beads that are obtained byremoving the cells after each of the experiments in conventionalTFM, this method is limited in throughput. The novel GAN-based system proposed here overcomes this limitation as it Li et al. XXX |
February 25, 2021 | vol. XXX | no. XX || vol. XXX | no. XX |
February 25, 2021 | vol. XXX | no. XX || vol. XXX | no. XX | R A F T -150-100-50050100150-150 -100 -50 0 50 100 150 (a) f p r e d i c t x , y [ P a ] f true x,y [Pa] GAN (m.s.) GAN (wrinkle) CNN (m.s.) CNN (wrinkle) (b) R [ − ] GAN (m.s.) GAN (wrinkle) CNN (m.s.) CNN (wrinkle) (c) ε f [ % ] GAN (m.s.) GAN (wrinkle) CNN (m.s.) CNN (wrinkle) (d) ε θ [ ◦ ] Fig. 5. (a) Comparison of the ground truth f true x,y and the predicted traction f predict x,y . Dashed line shows a condition f predict = f true . (b) Correlation coefficient R between f true x,y and f predict x,y . (c)-(d) Errors of the predicted traction compared to the ground truth data: (c) error in the traction magnitude ε f and (d) the traction direction ε θ . Bluesquares are the average value and black circles denote the outliers. Note that m.s. denotes the microscope images. provides the nearly same information, with the high levels ofthe correlations with the experimental data and the limitedlevels of the errors, on the cell mechanics only from the stillimages that are acquired just by plating the cells on thesilicone substrate without taking care of the reference as thesubstrate surface is known to become planar again upon theabsence of the cellular forces in a reversible manner. Giventhat early stages of drug screening require testing a massivenumber of candidate compounds (23), our system with thepotentially high-throughput data analysis capability will beuseful particularly in such screening studies. It is importantto note that our new system is not the one that essentiallycompetes with TFM, but the huge advantage of the proposedsystem is focused on its capability to provide data equivalentto the TFM (with a level of the errors) and thereby circumventperforming the TFM that needs considerable technical care.Rather, because the machine learning system depends onthe training data, further innovations in TFM such as super-resolution imaging (28, 29) are potentially introduced to oursystem to synergetically output more sophisticated data. Thus,our approach presents a versatile framework that integratesthe sophisticated experimental techniques and the efficientmeasurements. Materials and Methods
Step 1: Simultaneous measurement of traction forces and wrinkles.
Based on our previous studies (16, 19, 22–24), we prepared the substrate that can reversibly generate wrinkles upon application ofcellular forces. Firstly, a circular cover glass is treated with oxygenplasma (SEDE-GE, Meiwafosis) to hydrophilize the surface andis desiccated after fluorescent micro-beads (0.2 µ m in diameter,carboxylate yellow-green fluorescent beads; Invitrogen) in watersolution are distributed on the surface. Secondly, parts A and Bof CY 52-276 (Dow Corning Toray) are mixed at a weight ratio of1.2:1 and poured onto the cover glass to create a PDMS layer witha height of 30-40 µ m. Thirdly, the cover glass is placed in a 60°Coven for 20 hours to cure the PDMS. Fourthly, oxygen plasma isapplied uniformly along the surface of the PDMS layer to createan oxide layer that works as the substrate for cell culture. Finally,the substrate is coated with 10 µ g/mL collagen type I solution for3 hours.For the TFM measurement, fluorescent micro-beads are attachedto the substrate surface as position markers to measure the substratedeformations. The beads need to be firmly adhered to the surfaceso that cells would not move the beads due to endocytosis. In thiswork, the covalent bonding between the surface and the beads of0.001% v/v are performed by following two steps: (i) silane couplingof the substrate surface using 3-Aminopropyltrimethoxysilane and(ii) the covalent bonding formation due to carbodiimide. The beadsadhered on the glass surface are monitored to keep the referenceposition even after removing the cell using 0.25% Trypsin (Trypsin+ 1mm mmol/I EDTA-4Na solution; Fuji Wako Pure ChemicalCorporation). Cell culture and microscope setup.
A7r5 cells were maintained at37°C in a stage incubator (INUF-IX3W; Tokai Hit) under a hu-midified 5% CO incubator. An inverted microscope (1X73; Olym-pus) with a conforcal unit (CSU10; Yokokawa Electric) and oilimmersion lens (phase contrast, UPlanFLN 60x/1.25 Oil Iris Ph3,Olympus Corporation) are used to capture the cells and fluorescentbeads. During the experiment, DMEM(L)+10% FBS+Penicillin-Streptomycin (Fuji Wako Pure Chemical Corporation) is used as | Li et al. R A F T the culture medium. Traction force microscopy (TFM).
The software ImageJ/Fiji and itsplugin FTTC (Fourier transform traction cytometry) (30, 31) areused to evaluate the force field from the displacement field. Thesubstrate is considered as a soft elastic isotropic material that followsthe linear elastic theory. First, the displacement of the substratesurface u is measured by tracking the movement of the fluorescentbeads using PIV (particle image velocimetry). Second, the tractionfield is obtained from the displacement field by solving the governingequation for the elastic halfspace (32, 33) given by u ( x ) = Z S G ( x , y ) t ( y ) dS ( y ) [5]where t is the traction force, x and y are the positions of thedisplacement and the traction force, respectively. G is the Green’sfunction that is given by G ( x ) = 1 + νπEr (cid:16) (1 − ν ) r + νr x νr x r y νr x r y (1 − ν ) r + νr y (cid:17) [6]where E is the Young’s modulus, ν is the Poisson’s ratio, r =( r x , r y ) = x − y is the relative position vector and r = | r | . Thesoftware FFTC solves Eq. (5) in the Fourier space, which is givenby ˜ t = ( G T G + λ I ) − G T ˜ u [7]where tilde symbols denote the variables in Fourier space, λ is theregularization parameter (33) and I is the unit tensor. In order toevaluate the optimal parameter λ for the Tikhonov regularization,the L-curve criterion (33, 34) is applied. Note that E is exper-imentally determined (16) to be 5400 Pa and ν is assumed 0.5(incompressible) that is a typical value for PDMS material. Step2: Wrinkle extraction.
We use a new method SW-UNet (19),which is a CNN based on U-Net (35) to extract wrinkle patternsfrom the microscope image as shown in Fig. 1(b). As the trainingdata, we prepare 236 sets of corresponding two images (microscopeimage and manually labeled wrinkle image). The number of data isincreased to 2596 by using the image augmentation techniques. Weused NVIDIA Titan RTX to accelerate the training process, andthe Adam optimizer is utilized.
Step 3: Prediction of traction force based on GAN-based system.
As-sume that we have N o sets of corresponding images and data; theinput images x (microscope images, or extracted wrinkle images)and the force distributions y as shown in Fig. S1. We effectivelyhave the number of training data set 2 N o because the wrinkle imagehas only 1D information at each pixel (intensity I ( x, y ); see alsoimages in Fig. S1) while the force distributions have 2D information(2D force, f ( x, y ) = { f x ( x, y ) , f y ( x, y ) } ). We designed the networkto evaluate the cellular force only for a single axis at one time andfocus only on the x -directional force at each evaluation. An inputimage I i is used as the training data set ( I i , f x ) and ( I i , f y ) where I i is an image that rotates I i by 90 degrees.The force distributions are converted to gray scale images thathave intensities I ( f d ) = a arctan (cid:16) f d b (cid:17) + I mid [8]where a = 81 . b = 50 . I mid = 255 / f d is the components of the force d = x, y . Theforce distributions in gray scale, which are generated from the testimages, can be converted back to the force using this equation. Asthe training data, we prepared N = 332 sets (83 original images) ofcorresponding two images. Note that we increased the number oftraining data by rotating the images, 83 × GAN structure.
Our goal is to convert a physical quantity (wrinklegeometry) to another physical quantity (force distribution). Eventhough the mechanical formulation between the two quantitiesis given, it is not necessarily straightforward to solve this inverseproblem because of its complex nonlinear dynamics (36, 37). Instead,we achieve this purpose by training our machine learning systemto understand the underlying mechanical rules. Considering this conversion of the physical quantity as an image "translation", weutilize GAN (generative adversarial network) (38) in this work.GAN mainly consists of two networks, generator G and discrim-inator D as shown in Fig. S1, and the network is trained by acompetition of two networks. Goal of the generator G is to generatefake images (fake force distributions G ( x )) from the input images x (microscope images or extracted wrinkle images) and tries to mimicthe real images y (real force distributions), while the discriminator D tries to distinguish true and fake images from the group of images.As the training proceeds, the generator learns how to produce fakeimages that are difficult to be distinguished by the discriminatorfrom the real images, and the discriminator learns the rules todistinguish true/fake images. Once the training is completed, thetrained generator G can now be used as the translator to predictthe force distribution from the input images x even for test images,which were not included in the training process.In present work, we design the generator G with a form ofencoder-decoder which is based on U-Net (35) but without thecopy-and-crop path, and Markovian discriminator (PatchGAN) (39)is utilized as the discriminator D . The generator G converts theinput images x to the fake force distribution images G ( x ). Theimages of force distribution, y or G ( x ), and the input images x areconcatenated into a single image as the input of discriminator. Thetwo networks G and D are trained based on the labels of real/fake,and we utilize the loss function L that is used in pix2pix (27): L ( G, D ) = E x,y [log D ( x, y )]+ E x [log(1 − D ( x, G ( x )))]+ λL ( y, G ( x ))[9]where E is the expected value, L ( G ) is the L1 distance betweengenerated images G ( x ) and ground truths y and λ = 100 is theweight for the L1 term. The first term E [log D ] denotes the expectedprobability that the discriminator categorizes y as the real data,while the second term E [log(1 − D ( x, G ( x )))] denotes the probabilitythat the discriminator categorizes the generated image G ( x ) as thefake data. The goal of the generator G is to minimize L while thediscriminator D tries to maximize it.We use the training parameters as follows: 100 training epochs(batch size = 1), ε = 0 . β = 0 . β = 0 . ACKNOWLEDGMENTS.
This work was supported by JSPSKAKENHI Grant Number 18H03518 and 20K14649, ACT-X JST(Grant No. JPMJAX190S), and Multidisciplinary Research Lab-oratory System for Future Developments (MIRAI LAB). AD ac-knowledges support from the Novo Nordisk Foundation (grant no.NNF18SA0035142), Villum Fonden (grant no. 29476), Danish Coun-cil for Independent Research, Natural Sciences (DFF-117155-1001),and funding from the European Union’s Horizon 2020 researchand innovation program under the Marie Sklodowska-Curie grantagreement no. 847523 (INTERACTIONS).
1. AJ Engler, S Sen, HL Sweeney, DE Discher, Matrix elasticity directs stem cell lineage speci-fication. Cell , 677–689 (2006).2. B Ladoux, RM Mège, Mechanobiology of collective cell behaviours.Nat. reviews Mol. cell biology , 743–757 (2017).3. KH Vining, DJ Mooney, Mechanical forces direct stem cell behaviour in development andregeneration. Nat. reviews Mol. cell biology , 728–742 (2017).4. S Van Helvert, C Storm, P Friedl, Mechanoreciprocity in cell migration. Nat. cell biology ,8–20 (2018).5. NI Petridou, Z Spiró, CP Heisenberg, Multiscale force sensing in development.Nat. cell biology , 581–588 (2017).6. SE Murthy, AE Dubin, A Patapoutian, Piezos thrive under pressure: mechanically activatedion channels in health and disease. Nat. Rev. Mol. Cell Biol. , 771–783 (2017).7. N Wang, et al., Cell prestress. i. stiffness and prestress are closely associated in adherentcontractile cells. Am. J. Physiol. Physiol. , C606–C616 (2002).8. AW Holle, AJ Engler, More than a feeling: discovering, understanding, and influencingmechanosensing pathways. Curr. opinion biotechnology , 648–654 (2011).9. T Panciera, L Azzolin, M Cordenonsi, S Piccolo, Mechanobiology of yap and taz in physiologyand disease. Nat. reviews Mol. cell biology , 758–770 (2017).10. HB Wang, M Dembo, YL Wang, Substrate flexibility regulates growth and apoptosis of normalbut not transformed cells. Am. J. Physiol. Physiol. , C1345–C1350 (2000).11. NQ Balaban, et al., Force and focal adhesion assembly: a close relationship studied usingelastic micropatterned substrates. Nat. cell biology , 466–472 (2001).12. P Roca-Cusachs, V Conte, X Trepat, Quantifying forces in cell biology. Nat. cell biology ,742–751 (2017). Li et al. XXX |
February 25, 2021 | vol. XXX | no. XX || vol. XXX | no. XX |
February 25, 2021 | vol. XXX | no. XX || vol. XXX | no. XX | R A F T
13. AK Harris, P Wild, D Stopak, Silicone rubber substrata: a new wrinkle in the study of celllocomotion. Science , 177–179 (1980).14. AK Harris, D Stopak, P Wild, Fibroblast traction as a mechanism for collagen morphogenesis.Nature , 249–251 (1981).15. A Sakane, et al., Conformational plasticity of jrab/mical-l2 provides “law and order” in collec-tive cell migration. Mol. biology cell , 3095–3108 (2016).16. T Ichikawa, et al., Vinexin family (sorbs) proteins play different roles in stiffness-sensing andcontractile force generation. J Cell Sci , 3517–3531 (2017).17. S Fujiwara, S Deguchi, TM Magin, Disease-associated keratin mutations reduce tractionforces and compromise adhesion and collective migration. J. Cell Sci. (2020).18. K Burton, DL Taylor, Traction forces of cytokinesis measured with optically modified elasticsubstrata. Nature , 450–454 (1997).19. H Li, D Matsunaga, TS Matsui, H Aosaki, S Deguchi, Image based cellu-lar contractile force evaluation with small-world network inspired cnn: Sw-unet.Biochem. Biophys. Res. Commun. (2020).20. S Munevar, Yl Wang, M Dembo, Traction force microscopy of migrating normal and h-rastransformed 3t3 fibroblasts. Biophys. journal , 1744–1757 (2001).21. B Sabass, ML Gardel, CM Waterman, US Schwarz, High resolution traction force microscopybased on experimental and computational advances. Biophys. journal , 207–220 (2008).22. SP Fukuda, et al., Cellular force assay detects altered contractility caused by a nephritis-associated mutation in nonmuscle myosin iia. Dev. growth & differentiation , 423–433(2017).23. FJ Nehwa, et al., Multi-well plate cell contraction assay detects negatively corre-lated cellular responses to pharmacological inhibitors in contractility and migration.Biochem. Biophys. Res. Commun. , 527–532 (2020).24. N Kang, TS Matsui, S Liu, S Fujiwara, S Deguchi, Comprehensive analysis on the whole rho-gap family reveals that arhgap4 suppresses emt in epithelial cells under negative regulationby septin9. The FASEB J. , 8326–8340 (2020).25. DJ Watts, SH Strogatz, Collective dynamics of ‘small-world’networks. nature , 440 (1998).26. ZP Neal, How small is it? comparing indices of small worldliness. Netw. Sci. , 30–44 (2017).27. P Isola, JY Zhu, T Zhou, AA Efros, Image-to-image translation with conditional adversarialnetworks. CVPR (2017).28. H Colin-York, et al., Super-resolved traction force microscopy (stfm). Nano letters , 2633–2638 (2016).29. A Stubb, et al., Fluctuation-based super-resolution traction force microscopy. Nano letters ,2230–2245 (2020).30. Q Tseng, et al., Spatial organization of the extracellular matrix regulates cell–cell junctionpositioning. Proc. Natl. Acad. Sci. , 1506–1511 (2012).31. JL Martiel, et al., Measurement of cell traction forces with imagej in Methods in cell biology.(Elsevier) Vol. 125, pp. 269–287 (2015).32. SV Plotnikov, B Sabass, US Schwarz, CM Waterman, High-resolution traction force mi-croscopy in Methods in cell biology. (Elsevier) Vol. 123, pp. 367–394 (2014).33. US Schwarz, JR Soiné, Traction force microscopy on soft elastic substrates: A guide to re-cent computational advances. Biochimica et Biophys. Acta (BBA)-Molecular Cell Res. ,3095–3104 (2015).34. PC Hansen, Rank-deficient and discrete ill-posed problems: numerical aspects of linearinversion. (SIAM), (1998).35. O Ronneberger, P Fischer, T Brox, U-net: Convolutional networks for biomedical image seg-mentation in International Conference on Medical image computing and computer-assistedintervention. (Springer), pp. 234–241 (2015).36. E Cerda, L Mahadevan, Geometry and physics of wrinkling. Phys. review letters , 074302(2003).37. N Stoop, R Lagrange, D Terwagne, PM Reis, J Dunkel, Curvature-induced symmetry break-ing determines elastic surface patterns. Nat. materials , 337–342 (2015).38. I Goodfellow, et al., Generative adversarial nets in Advances in neural information processingsystems. pp. 2672–2680 (2014).39. C Li, M Wand, Precomputed real-time texture synthesis with markovian generative adversar-ial networks in European conference on computer vision. (Springer), pp. 702–716 (2016). | Li et al. R A F T (ground truth) x real or generator or discriminator or discriminatorfake (input images) G (prediction) x ( ) G x ( ) x y x
Fig. S1.
Schematic showing the structure of GAN (generative adversarial network) utilized in the present work.
Supplementary Information • Figure S1• Movies S1-4
Supplemental video captions • Movie S1 : Sample of the force estimation using our system.The cell is MEF (mouse embryonic fibroblast). The substrateis prepared by mixing parts A and B of CY 52-276 with aweight ratio of 1.1:1.•
Movie S2 : Sample of the force estimation using our system.The conditions are same as Movie 1.•
Movie S3 : Sample of the force estimation using our system.The conditions are same as Movie 1.•
Movie S4 : Sample of the force estimation using our system.The conditions are same as Movie 1. Li et al. XXX |
February 25, 2021 | vol. XXX | no. XX || vol. XXX | no. XX |