Precise multi-emitter localization method for fast super-resolution imaging
aa r X i v : . [ phy s i c s . op ti c s ] D ec Precise multi-emitter localization method for fast super-resolutionimaging
Yuto Ashida, Masahito Ueda , Department of Physics, University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan. Center for Emergent Matter Science (CEMS), RIKEN, Wako, Saitama 351-0198, Japan.
Compiled August 23, 2018We present a method that can simultaneously locate positions of overlapped multi-emitters at the theoretical-limit precision. We derive a set of simple equations whose solution gives the maximum likelihood estimator ofmulti-emitter positions. We compare the performance of our simultaneous localization analysis with the conven-tional single-molecule analysis for simulated images and show that our method can improve the time-resolutionof superresolution microscopy an order of magnitude. In particular, we derive the information-theoreticalbound on time resolution of localization-based superresolution microscopy and demonstrate that the boundcan be closely attained by our analysis. c (cid:13)
OCIS codes: (100.6640) Superresolution;(180.2520) Fluorescence microscopy; (110.4190) Multiple imaging.
Precisely and accurately locating point objects is along-standing common thread in science. Recent realiza-tions of super-resolved imaging of single molecules [1–3]have revolutionized our view of quasi-static nanostruc-tures in - vivo . In particular, a wide-field approach basedon localizing individual fluorophores has emerged as aversatile method [4–7]. The single-molecule localizationmicroscopy (SMLM) works under conditions in whichfluorescent molecules are activated at very low den-sity so that no more than a single molecule within anydiffraction-limited region emits photons simultaneously[8]. A set of single-molecule positions can then be pre-cisely determined beyond the diffraction limit by fittingeach image of molecules using a single point spread func-tion. Nevertheless, the slow temporal resolution of super-resolved imaging severely restricts the utility to thestudy of live-cell phenomena. This is because the anal-ysis discards the information from crowded moleculeswith overlapping images through filtering and, typically,SMLM requires accumulating thousands of frames togenerate a super-resolution image [9, 10]. A substantialreduction of the imaging time will significantly expandthe horizon of super-resolution techniques and enable toobserve fast, nanoscale dynamics in - vivo .Recently, there have been remarkable progresses in im-proving the temporal resolution of super-resolution mi-croscopy by developing multi-emitter localization algo-rithms [11–17]. In particular, Ref. [16] has utilized sC-MOS camera to achieve an impressively high imagingspeed (32 frame/s). Also, there exists now commerciallyavailable super-resolution microscopy for live-cell imag-ing from Nikon. Yet, the theoretical limit on the time res-olution has remained elsusive and hence, to maximize thepotential of the technique, a multi-emitter localizationanalysis that allows us to attain the theoretical boundneeds to be developed.Here we develop a multi-emitter localization analy-sis that can attain at nearly the theoretical-limit speedof localization-based superresolution microscopy. To be- gin with, let us discuss the theoretical limit on the tem-poral resolution of super-resolved imaging. The imageacquisition time T img of localization-based superresolu-tion microscopy is determined by a number of framesand an exposure time of each frame. We define the fi-delity F of super-resolved imaging as the fraction ofimaged molecules i.e., activated molecules at least onceduring the entire process of imaging. Then the requirednumber of frames to ensure the fidelity is given by − ln(1 − F ) ρ obj /ρ img , where ρ obj ( ρ img ) is the object (im-age) molecule density [18]. An exposure time of a singleframe is determined by the required number of photons N photon to achieve the desired resolution δ divided bythe collection rate ρ img S Γ, where S is the area of theregion of interest and Γ is the collection rate of photonsper molecule. While the switching rate of molecules isassumed to be optimized with the exposure time, thisshould not be considered to be realizable in a single ex-perimental setup; here we are interested in the funda-mental limit among all experimental situations. Infor-mation theory dictates that N photon be bounded by theFisher information matrix [19]. To obtain the net infor-mation gain, we assume the point spread function (PSF)of a single molecule as a 2D Gaussian with a standarddeviation σ and calculate the precision limit by assumingan arrayed configuration of molecules with density ρ img .Then, we evaluate the Fisher information matrix [19] andits inverse numerically. We define the diagonal elementof the inverse matrix in multi-molecule-position basis as[ I − ] DD . Consequntly, N photon is bounded as N photon ( ρ img , δ, σ ) ≥ δ [ I − ] DD ≡ δ ρ img S ∆ ( ρ img , σ ) , (1)where we define the normalized precision limit ∆ by thelast equality. While ∆ has a simple relation ∆ = σ when molecules are sufficiently sparse, its value can beobtained only numerically in a high-density regime. Theinformation-theoretic limit on the image acquisition time1ig. 1. (Color online) (a) Schematic figure about tempo-ral resolution of localization-based superresolution mi-croscopy. There is an optimal image molecule densitythat allows the fastest image acquisition. (b) Theoreti-cal limit on the image acquisition time plotted againstthe image molecule density, calculated for δ = 10nm,fidelity F = 0 .
9, Γ = 2 . × σ = 82 . T img ≥ ln (cid:16) − F (cid:17) ρ obj ρ ∗ img ∆ ( ρ ∗ img , σ ) δ Γ , (2)where ρ ∗ img is the optimized image density so that thelower-bound of the image acquisition time is minimized.Figure 1a summarizes the theoretical limit on the timeresolution of super-resolution microscopy for varying im-age density. In a low-density region, where the activatedmolecules are sparsely distributed so that the interfer-ence patterns rarely overlap (see also the inset figure (i)in Fig. 1b), the amount of information carried by a sin-gle photon is nearly constant. Thus, increasing an imagedensity directly reduces the acquisition time for a super-resolution image which scales as ∝ ρ − (indicated bythe black-dashed line in Fig. 1b). However, once the in-terference patterns significantly overlap and the peaksof molecules cannot be resolved anymore (see the insetfigure (iii)), the information gain per photon dwindlesrapidly. Consequently, the required number of photonsrapidly increases, resulting in a sharp increase in the im-age acquisition time. Hence, the fastest time resolution isachieved at a high image density between the above twosituations, where the information acquisition rate is max-imal (see the inset figure (ii)). Such density regime is in-dicated by the blue-shaded region in Fig. 1b. To achievethe theoretical-limit speed, the crucial step is to developa multi-emitter localization analysis that can faithfullywork under such high density region. As detailed later,our method can closely attain such limit. While the the-oretical curve in Fig. 1b makes sense irrespective of theperformance of a specific algorithm, we indicate, for con-venience, the region where our multi-emitter localizationmethod fails to work by the gray-shaded region.We now describe our simultaneous localization anal- ysis. In a previous work, we show that tracking pro-gressive collapse of many-body wavefunction enablesa diffraction-unlimited position measurement of ultra-cold atoms [20] in an optical lattice. We here general-ize this approach to classical objects, such as fluores-cent molecules, by treating the estimated position dis-tribution as a counterpart of the quantum-mechanicalwavefunction. We model the effective point spread func-tion (PSF) of a single molecule by a 2D Gaussian P [ r | R ] = exp( −| r − R | / (2 σ )) / (2 πσ ), where σ =0 . λ/ NA [21] is the standard deviation with NA be-ing the numerical aperture, and we denote the posi-tion of the molecule as R and that of photodetectionsas r . The interference pattern of multi-emitter is con-structed from an incoherent sum of these point spreadfunctions: P [ r |{ R } ] = (1 − ǫ ) /N P Nm =1 P [ r | R m ] + ǫ/S ,where we denote { R } ≡ { R , R , . . . , R N } as a set of N molecule positions, ǫ is the fraction of the backgroundnoise, which can be related to the signal-to-noise ratio(SNR) as ǫ = 1 / (1 + SNR), and S is the area of the re-gion of interest. We formulate the imaging as a stochasticprocess in which spatial locations of photodetections arerandomly generated according to the interference pat-tern.Let us assume that M photons are detectedat r , r , . . . , r M . The conditional probability dis-tribution of a set of N est molecule estimators { R est } ≡ { R , R , . . . , R N est } is given by theBayesian inference: P [ { R } est | r , r , . . . , r N photon ] ∝ Q Mi =1 P [ r i |{ R } est ] P [ { R } est ], where P [ { R } est ] repre-sents a prior distribution of the molecule distribution.Since we assume no prior knowledge about the configura-tion of molecules, the initial distribution P is chosen tobe a uniform distribution. In our formulation, the prob-lem of identifying the most probable set of molecule po-sitions is equivalent to maximizing the conditional prob-ability distribution with respect to possible multiple-molecule configurations { R est } .Remarkably, for Gaussian point spread functions, wecan show by analytical calculations that the problemof finding the most probable set of estimators is sub-stantially simplified to solving the following N est self-consistent equations: R m = P Mi =1 r i g m (cid:0) r i ; { R } est (cid:1)P Mi =1 g m (cid:0) r i ; { R } est (cid:1) , (3)where m = 1 , , . . . , N est is the label of each estimator.We then introduce the weight-function g m by g m (cid:0) r ; { R } est (cid:1) ≡ exp (cid:16) − | r − R m | σ (cid:17) γ + P N est k =1 exp (cid:16) − | r − R k | σ (cid:17) . (4)Here we define the term γ ≡ πσ N est ǫ/ (cid:0) S (1 − ǫ ) (cid:1) whichdescribes the contribution from the background noise.This type of equations can be efficiently solved by us-ing standard numerical methods [22]. We note that the2stimators coincide with the maximum likelihood esti-mator and hence, the theoretical-limit precision, i.e., theCram´er-Rao bound, can be asymptotically attained. Wecan easily generalize the above discussions to pixelatedmeasurements in which the number of photodetections N i,j at each pixel h i, j i constitutes the sufficient statis-tic. The result is the following set of N est self-consistentequations for estimators: R m = P h i,j i r i,j g ij ; m (cid:0) { R } est (cid:1)P h i,j i g ij ; m (cid:0) { R } est (cid:1) , (5)where r i,j is the pixel position and g ij ; m ≡ N ij g m (cid:0) r i,j ; { R } est (cid:1) . Note that such simple equationshave not been derived in other approaches of multi-emitter localization [11–14].The above formulation can be also extended to thecase with a nonuniform background noise. Let ǫ ( r ) be thefraction of the background noise at position r . By replac-ing the term γ in Eq. (4) with γ ( r ) ≡ (2 πσ N ǫ ( r )) / (1 − R S ǫ ( r ′ )d r ′ ) , we can use the self-consistent equationsin Eq. (3) to obtain the most probable estimators formolecule positions with nonuniform background. If weutilize the astigmatism [23], a generalization to 3Dimaging is also possible. The point spread function isdescribed by an asymmetric Gaussian function whosewidths are given by σ X,Y ( Z ) = σ p Z ∓ η ) /d ,where η is an axial astigmatism, σ is the focus width, d is the focus depth. The axial position Z m of the molecule m is obtained as the solution of the self-consistent equa-tion, Z m = η · P Mi =1 ( g Xi ; m − g Yi ; m ) P Mi =1 ( g Xi ; m + g Yi ; m ) . (6)We introduce the weight functions g αi ; m = f αi ; m e − h i ; m / ( γ + P N est k =1 e − h k ; m σ / ( σ X ( Z k ) σ Y ( Z k ))),where f αi ; m ≡ (( a i − α m ) − σ α ( Z m )) /σ α ( Z m ), h i ; m ≡ ( x i − X m ) / (2 σ X ( Z m )) + ( y i − Y m ) / (2 σ Y ( Z m )), α = X, Y , and a = x, y . These generalizations can beapplied jointly to deal with realistic situations.To solve the self-consistent equations (5), we applya high-order iterative method known as Steffensen’smethod [22], which allows quadratic convergence. In con-trast to an ordinary iterative method, this method doesnot need to calculate derivatives and, in our problem, canefficiently perform calculations. To avoid an unwanteddivergence and make a robust convergence, when a tem-poral value of estimators becomes unreasonably high,the simple successive substitution is concomitantly used.The iteration is terminated when the calculation con-verges or after 100 iterations.Finding the global maximum of the conditional prob-ability distribution is an essential step for the simul-taneous localization. To do so, we combine the pre-estimation and the optimization based on the informa-tion measure as follows. First, we pre-estimate and local-ize molecules based on the single-molecule analysis withthe well-established rejection algorithm in which the lo- cal maximum is identified by setting a suitable thresh-old [11]. Then, the image is fitted with a single pointspread function and the position of a molecule is esti-mated. If the result of the fitting significantly deviatesfrom the position of the local maximum, the image isjudged as constructed from multiple molecules and theestimated position is discarded. The resulting set of N ini localized positions constitutes the first set of initial po-sitions for iterations.Second, for each assumed number of molecules N est ,the residual N est − N ini initial positions are randomlygenerated according to the observed probability distribu-tion of photodetections. This enables the well-estimatedinitialization of estimators in successive iterative calcu-lations. A large number of different sets of initial posi-tions are generated. Then, iterative calculations to solvethe self-consistent equations (5) are performed by start-ing from each set (typically, preparing about one thou-sand different sets is sufficient to find the global solutionat the high-molecule density). The iteration result thatminimizes the Kullback-Leibler divergence between theobserved distribution and the distribution reconstructedfrom the estimated positions is chosen as the most prob-able set of molecule positions within the sector of N est -molecule configuration space. As a typical processingspeed, the analysis for 60 ×
60 pixels with 100 iterationscan be finished by the CPU calculation with C code pro-gram within about 3.5 s. The performance of the cal-culation can be enhanced by increasing both the trialnumber of preparing the initial configurations and thenumber of iterations.We perform the above procedures of estimating themost probable set of molecule positions for variousmolecule number N est ≥ N ini . For each iteration resultof the N est -molecule configurational space, we calculatethe expected probability distribution P N est of photode-tections and compare it with the observed probabilitydistribution P data by employing the Kullback-Leibler di-vergence D [ P data | P N est ]. The final set of the most prob-able molecule positions is determined by minimizing theKullback-Leibler divergence with respect to N est .To demonstrate how superior our simultaneous local-ization approach is to the conventional SMLM, we showa typical result in Fig. 2a, b. While SMLM can only iden-tify the well-isolated molecules as indicated by the cir-cles in Fig. 2a, our method identifies all molecules at thetheoretical-limit precision as shown in Fig. 2b. Hence,our analysis allows an accurate and precise localizationof multi-emitters despite a substantial overlap of images.We note that our value of the pixel size does not com-promise the superiority of our method since an expectedimprovement of the localization precision with respectto a larger pixel size (100nm) is only 5% [24].To achieve the fundamental limit of time resolution inEq. (2), the crucial fact is that our simultaneous localiza-tion significantly outperforms the single-molecule anal-ysis at high density region, where super-resolved imag-ing can be performed with ultimate time resolution. We3ig. 2. (Color online) (a) A simulated image of fluores-cent molecules with uniform background noise ( ρ img =16 /µ m , SNR=50, pixel size=28 nm). The crosses indi-cate the true positions of particles. SMLM only identi-fies emitters indicated by circles. (b) The result of ourmethod. (c,d) The fraction of identified molecules andthe localization precision, calculated for 10 photons permolecule, SNR=50, and 10 simulated images. In (d),the theoretical lower bound is shown by the red-dashedcurve. (e,f) The performance of our method in the lowSNR and in the presence of nonuniform noise, calculatedfor ρ img = 10 /µ m , 10 photons per molecule, and 10 images.demonstrate this by applying the simultaneous localiza-tion and SMLM analyses to different simulated imagesrepeatedly. The simultaneous localization analysis at-tains more than a tenfold improvement of the fractionof identified molecules (the so-called “recall” [25]) withrespect to the conventional single-molecule analysis (Fig.2c). Also, this is a fourfold improvement compared withthe reported performance of DAOSTORM at the samesignal-to-noise ratio (SNR) [11]. Note that our methodachieves some ninety percent accuracy in the blue-shaded region enabling the fastest super-resolved imag-ing. The simultaneous localization also achieves, in theregion of the ultimate time resolution, the theoretical- limit precision (Fig. 2d). In particular, our method canclosely attain the precision limit up to ρ img ≃ /µ m (with σ = 82 . ǫ ( r ) = ǫ (1 + te − r / (2 σ ′ ) ) / N , where ǫ =1 / (1 + SNR), N ≡ R S d r ′ (1 + te − r ′ / (2 σ ′ ) ), σ ′ = 4 σ is thewidth of the nonuniformity. The origin of the coordinatesrepresents the center of the region of interest. Figures2e-f show the fraction of identified molecules and the lo-calization precision normalized by the theoretical limit∆ in the high-molecule density ρ img = 10 /µ m (with σ = 82 . ≃ t in Fig. 2e-f clearly indicatethat the nonuniformity of the background noise does notcompromise the performance of our method.Finally, let us briefly mention the strengths and theweaknesses of our method. A major strength of ourmethod is the ability to localize multiple molecules withthe theoretical limit precision in a high-molecule density.Also, we utilize the separable property of the Gaussianfunction to derive a set of simple self-consistent equa-tions, which can enhance the fidelity of the multi-emitterlocalization. On the other hand, the drawback of our ap-proach is that the point spread function requires to bewell approximated by the Gaussian function. Also, asa common problem of maximum-likelihood-estimation-based localization methods [17], one needs a prior knowl-edge about the noise property.In summary, we have demonstrated a precise multi-emitter localization analysis that enables the super-resolved imaging at the theoretical-limit speed. A fastsuper-resolution microscopy should have an applicationin, for example, a non-invasive observation of intracellu-lar dynamics at molecular scale. Our method should alsoprovide a powerful means to precisely locate light emit-ters below the diffraction limit in wide areas of opticalscience. Funding.
This work was supported by KAKENHIGrant No. 26287088 from the Japan Society for thePromotion of Science, a Grant-in-Aid for Scientific Re-search on Innovation Areas “Topological Materials Sci-ence” (KAKENHI Grant No. 15H05855), the Photon4rontier Network Program from MEXT of Japan, andthe Mitsubishi Foundation. Y. A. was supported by theJapan Society for the Promotion of Science through Pro-gram for Leading Graduate Schools (ALPS).
Acknowledgment.
We are grateful for discussions withY. Okada, K. Goda, H. Mikami and T. Shitara.
References
1. T. A. Klar, S. Jakobs, M. Dyba, A. Egner, and S. W.Hell, Proc. Natl. Acad. Sci. USA , 8206-8210 (2000).2. M. G. L. Gustafsson, Proc. Natl Acad. Sci. USA ,13081-13086 (2005).3. E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lind-wasser, S. Olenych, J. S. Bonifacino, M. W. Davidson,J. Lippincott-Schwartz, and H. F. Hess, Science ,1642-1645 (2006).4. E. Betzig, Opt. Lett. , 237-239 (1995).5. T. S. Hess, T. P. K. Girirajan, and M. D. Mason, Bio-phys. J. , 4258-4272 (2006).6. M. J. Rust, M. Bates, and X. Zhuang, Nat. Met. ,793-795 (2006).7. A. Pertsinidis, Y. Zhang, and S. Chu, Nature , 647-651 (2010).8. G. Patterson, M. Davidson, S. Manley, and J.Lippincott-Schwartz, Annu. Rev. Phys. Chem. , 345-367 (2010).9. B. Huang, H. Babcock, and X. Zhuang, Cell , 1047-1058 (2010).10. L. Schermelleh, R. Heintzmann, and H. J. Leonhardt,Cell Bio. , 165-175 (2010).11. S. J. Holden, S. Uphoff, and A. N. Kapanidis, Nat. Meth-ods , 279-280 (2011).12. T. Quan, H. Zhu, X. Liu, Y. Liu, J. Ding, S. Zeng, andZ.-L. Huang, Opt. Exp. , 16963-16974 (2011).13. S. Cox, E. Rosten, J. Monypenny, T. Jovanovic-Talisman, D. T. Burnette, J. Lippincott-Schwartz, G.E. Jones, and R. Heintzmann, Nat. Methods , 195-200(2012).14. F. Huang, S. L. Schwartz, J. M. Byars, and K. A. Lidke,Biomed. Opt. Exp. , 1377-1393 (2011).15. Y. Wang, T. Quan, S. Zeng, and Z.-L. Huang, Opt. Exp. , 16039-16049 (2012).16. F. Huang, T. M. P. Hartwich, F. E. Rivera-Molina, Y.Lin, W. C. Duim, J. J. Long, P. D. Uchil, J. R. Myers,M. A. Baird, W. Mothes, M. W. Davidson, D. Toomre,and J. Bewersdorf, Nat. Methods , 653-658 (2013).17. A. R. Small, and S. Shane, Nat. Methods , 267-279(2014).18. A. R. Small, Biophys. J. , L16-L18 (2009).19. T. M. Cover, and J. A. Thomas, Elements of Infor-mation Theory, 2nd Edition. (Wiley-Interscience, NewYork, 2006).20. Y. Ashida, and M. Ueda, Phys. Rev. Lett. , 095301(2015).21. B. Zhang, J. Zerubia, and J.-C. Olivo-Martin, Appl.Opt. , 1819-1829 (2007).22. L. R. Burden, and J. D. Faires, Numerical Analysis,eighth-ed. (Brooks/Cole, 2004).23. L. Holtzer, T. Meckel, and T. Schmidt, Appl. Phys. Lett. , 053902 (2007). 24. R. E. Thompson, D. R. Larson, and W. W. Webb, Bio-phys. J. , 2775-2783 (2002).25. S. Wolter, U. Endesfelder, S. van de Linde, M. Heile-mann, and M. Sauer, Opt. Exp. , 7020-7033 (2011)., 7020-7033 (2011).