Go with the FLOW: Visualizing spatiotemporal dynamics in optical widefield calcium imaging
Nathaniel J Linden, Dennis R Tabuena, Nicholas A Steinmetz, William J Moody, Steven L Brunton, Bingni W Brunton
GGo with the FLOW: Visualizing spatiotemporal dynamics inoptical widefield calcium imaging
Nathaniel J. Linden , , Dennis R. Tabuena , , Nicholas A. Steinmetz ,William J. Moody , Steven L. Brunton , Bingni W. Brunton , ∗ Department of Bioengineering, University of Washington, Seattle Department of Biology, University of Washington, Seattle Graduate Program in Neuroscience, University of Washington, Seattle Department of Biological Structure, University of Washington, Seattle Department of Mechanical Engineering, University of Washington, Seattle
Abstract
Widefield calcium imaging has recently emerged as a powerful experimental technique to record coor-dinated large-scale brain activity. Because of the spatial and temporal resolution of these measurements,they present a unique opportunity to characterize spatiotemporally coherent structures that underlie neu-ral activity across many regions of the brain. In this work, we leverage analytic techniques from fluiddynamics to develop a visualization framework that highlights features of flow across the cortex, mappingtime-varying sources and wave fronts that may be correlated with behavioral events. First, we trans-form the time series of widefield calcium images into time-varying vector fields using optic flow. Next,we extract concise diagrams summarizing the dynamics, which we refer to as
FLOW (flow lines in opticalwidefield imaging) portraits . These FLOW portraits provide an intuitive map of dynamic calcium activity,including sources, sinks, and traveling fronts in the data, using the finite-time Lyapunov exponent (FTLE)technique developed to analyze time-varying manifolds in unsteady fluids. Importantly, our approachcaptures coherent structures that are poorly represented by traditional modal decomposition techniques.We demonstrate the application of FLOW portraits on two widefield calcium imaging datasets: activityof the developing mouse cortex during the first 8 postnatal days, and spontaneous cortical activity in anadult mouse.
Keywords– Widefield calcium imaging, computational neuroscience, dynamical systems, coherent structures, finite-time Lyapunov exponents.
Coordinated organization of neural activity among brain regions is believed to serve many crucial roles,including performing specific computations in the cortex [1–3] and supporting brain development [4–6];further, its disruption may lead to neurological disease [7–9]. One prominent characteristic of neural activityat the scale of brain regions is the rapid and coherent propagation of activity across the cortex, which hasbeen widely observed in a variety of contexts, including spontaneous activity, task engagement, sleep, anddevelopment [10–15]. Although such spatiotemporal dynamic features are often visually salient, it remainschallenging to quantify and succinctly summarize their behavior directly from neural recordings.Widefield optical imaging of calcium activity provides a unique opportunity to study coordinated spa-tiotemporal neural activity among brain areas, because this experimental approach achieves large fields-of-view with high temporal and spatial resolution. In general, widefield imaging experiments involve fluo-rescence imaging of the entire brain surface of transgenic animals that express optical indicator proteins inknown populations of neurons [16–19]. Many experiments choose to use genetically encoded calcium indi-cators from the GCaMP family to image neural calcium dynamics, which is a proxy for electrical neuronalactivity [20–23]; more generally, the visualization methods we discuss here can be applied to any widefieldoptical imaging experiment, such as imaging with voltage-sensitive indicators [24, 25]. Cortical activity hasbeen measured using widefield calcium imaging in a variety of experiments, notably to study perceptual ∗ Corresponding author: [email protected] . a r X i v : . [ q - b i o . N C ] S e p ecision making [1, 26–30], to extract cortical functional connectivity [8, 31, 32], to characterize cortical ac-tivity that organize brain development [33], and to study the effects of disease in the cortex [7–9]. In allof these data, it is typical to observe multiple regions activating transiently or in regular succession, withdistinct source regions and wave-like flows of activity across the fields of view. These features can often bedescribed as flow of activity with coherent traveling fronts, distinct sources, and sinks; interestingly, all ofthese patterns are well studied as nonlinear features of spatiotemporal dynamical systems [34].The most widely applied approaches to analyze time-varying recordings of high-dimensional neuralactivity are dimensionality reduction techniques, which extract modes that correspond to dominant, low-dimensional features of the high-dimensional data [35–37]. These low-dimensional features are useful asrepresentations of the neural activity that facilitate further analysis and modeling. Furthermore, the obser-vation that the dynamics of neuronal populations can be reduced to a relatively small number of featuresmay be a clue about the mechanisms that underlie coordinated neural activity [38–41]. Common modal de-composition algorithms used in neuroscience [35, 42] include singular value decomposition (SVD), whichis closely related to principle component analysis (PCA), independent component analysis (ICA), and non-negative matrix factorization (NNMF). These techniques all solve for combinations of relatively few modesin space and time that reconstruct an estimate of the original high-dimensional data; their solutions differby making different assumptions about the statistical structure of the modes.There are many exciting recent innovations in modal decomposition for analyzing large-scale neuraldata, some of which are extensions and derivatives of SVD, ICA, and NNMF. Interestingly, while some ofthese methods have explicit representations of the temporal dynamics (for instance, jPCA [39], dynamicmode decomposition (DMD) [43–45], and NNMF with temporal constraints [46–49]), they largely set out toachieve space/time separation. Applying PCA and NNMF to segments of synthetic and experimental data(Figure 1A) yields a set of spatial modes (Figure 1B; temporal modes not shown) that provide a represen-tation of the activity. However, these representations are static modes and may not adequately summarizespatiotemporal data.The visualization approach introduced in this work is inspired by the similarity of spatial flows ob-served in widefield optical imaging to flows of physical fluids. Humans have a deep intuition about fluidflows from our everyday experiences (e.g., from the patterns of milk mixing in coffee, or a river flowing).Representing brain data as a flow will allow researchers to leverage this intuition and decades of methodsfrom flow analysis and visualization. Propagation of neural activity has many commonalities and differ-ences with physical fluid flows. In both, there exist coherent structures whose boundaries may be invarianteven as the activity changes with time. In fluid physics, these invariant manifolds are known as Lagrangiancoherent structures (LCS) [50–52], which act as transport barriers in the flow, either repelling or attract-ing material. LCS are often visualized by computing ridges in the finite-time Lyapunov exponent (FTLE)field [53–56], although there are other computational approaches based on variational theory [57]. Somenoteworthy biological applications include the use of LCS to study the physics of jellyfish feeding [58] andunderstanding cardiovascular hemodynamics [59, 60]. Unlike physical flows, neural activity is not gov-erned by fundamental conservation laws; nevertheless, these dynamics are well described by time-varyingvector fields [61–64].In this work, we develop a visualization framework to capture the spatiotemporal dynamics of neu-ral activity by extracting field lines in optical widefield imaging, which we call FLOW portraits. FLOWportraits are generated by considering frame-by-frame dynamics as time-varying optical flow vector fields,from which we compute and integrate the ridges in its FTLE. To validate our approach, we show that FLOWportraits give accurate and intuitive summaries of a synthetic dataset where a spatial Gaussian grows andtranslates in time. Next, we apply our methods to analyze bouts of activity from two widefield calciumimaging datasets; both exhibit spontaneous, widespread activity across cortex. The first data are recordingsof spontaneous cortical activity of GCaMP6s-expressing mouse pups during their first 8 postnatal days [33].We next analyze recordings of spontaneous cortical activity in a GCaMP6s-expressing adult mouse [13]. Inboth examples, we demonstrate that FLOW portraits extract meaningful and interpretable outlines of thedominant patterns in the cortical activity that contribute to our understanding of the animals’ developmen-tal and behavioral states. 2igure 1: FLOW portraits capture coherent propagation of structures that are poorly represented by com-mon modal decompositions that aim to achieve space-time factorization. ( A ) Two examples of widefieldimaging data for which we compare principal component analysis (PCA), non-negative matrix factoriza-tion (NNMF), and our FLOW portraits. Dashed white lines at 0 sec indicate the midline of the brain. Themouse pup data shows a pan-cortical wave imaged as widefield optical GCaMP6s activity ( ∆ F/F ) froma P7 (postnatal 7 days) transgenic animal. Scale bar is 1 mm. The adult mouse data shows spontaneouswidefield optical activity from an adult transgenic animal. Scale bar is 2 mm. ( B ) FLOW portraits show asuccinct a summary of the spatiotemporal flow in each example dataset, while PCA and NNMF do not. ThePCA modes are the first four spatial principal components; the NNMF modes are from a 4-mode solution tothe factorization and are not ordered. The temporal components corresponding to these PCA and NNMFmodes are not shown. Both sets of modes decompose the growth and translating wavefronts of activityinto static spatial images, from which the flow of the activity cannot be easily appreciated. This work introduces FLOW (flow lines in optical widefield imaging) portraits, which are visualizationsthat provide a concise and intuitive summary of the spatiotemporal dynamics, highlighting coherent struc-tures in widefield recordings. Importantly, FLOW portraits differ from modal decomposition techniques inthat they do not provide a basis in which to approximate the data and cannot quantitatively explain vari-ance in the recordings. Instead, FLOW portraits explicitly convert the image stack into time-varying vectorfields to extract sources, sinks, and traveling fronts in the data (Figure 1). As our approach leverages andadapts analytic techniques from fluid dynamics [52] that are unfamiliar to most neuroscientists, this sec-tion describes how to compute finite time Lyapunov exponent (FTLE) from time-varying vector fields andbuilds intuition for how the ridges of the FTLE field can be interpreted in the context of widefield calciumimaging. 3igure 2: Finite Time Lyapunov Exponent (FTLE) fields are computed from spatiotemporal data. ( A ) Anillustration of how optic flow is computed from successive frames of images by correlating the relativemovement of pixel intensities. This procedure converts widefield imaging data into a vector field of ve-locities. ( B ) The flow map at every pixel location is a virtual particle at x integrated through the vectorfield for a duration of T , from t to t + T ; in reverse time, particles are integrated from t to t − T . Thisintegration stretches neighboring particles in some directions ( r ) and compresses them in others ( r ). ( C )The flow map computation is repeated starting at each frame of the movie, at base time t + k ∆ t , where ∆ t is the separation between frames; forward maps are orange and backward maps are purple. ( D ) TheFTLE fields are computed from the Jacobians of these flow maps; example FTLE fields are illustrated forsuccessive frames of widefield imaging data.The steps of our approach to compute FLOW portraits are illustrated in Figures 2 and 3. The input datais a video (i.e. image stack) of the relative change of fluorescence of the imaged optical protein indicator, ∆ F /F , as it changes in time over many frames. The raw fluorescence may drift over the course of an ex-periment, so ∆ F /F is considered to be a robust proxy for the magnitude of neural activation, normalizingthe change in fluorescence over a moving-window baseline [65]. FLOW portraits are well suited to sum-marize data where optical activity is seen to diffuse or flow across the field of view, with varied patternsthroughout the recording. To characterize the propagation of recorded neural activity across brain areasthrough space, we first compute the flow vector field using optic flow. Next, the FTLE is computed fromthe time-varying vector field using the standard integration method as outlined by Onu et al. [66]. Last,the FTLE field is post-processed to visualize ridge-like features that highlight the coherent features of aspatial flow [52, 53]. It is important to note that we refer to the processed FTLE ridges as
FLOW portraits to avoid misinterpretation with traditional LCS analysis in fluid dynamics [52]. Details of data collection,preprocessing, and computation are described in the Methods (Section 5).4 .1 Optical flow of widefield imaging data
We describe the frame-by-frame spread of neural activity as time-varying vector fields that describe itsflow by computing its optical flow. In specific, as regions of high pixel intensity in ∆ F/F move and diffuseacross the field of view, these coherent motions can be converted into a vector field of velocities, dx/dt and dy/dt , at every pixel in the recording. We denote this vector field as v ( x , t ) , defined at every point in space x at time t . Motion velocities are commonly estimated from video data in computer vision using optical flowalgorithms [67], and biological visual systems of vertebrates and invertebrates also perceive moving sceneswith computations akin to optical flow [68, 69]. In addition to well-known computer vision techniques,some prior work has explored optical flow computations in widefield calcium imaging data [62, 63]. Herewe use the Horn-Schunck (HS) [70] method because of its simplicity and its observed strong performanceon our sample data.Figure 3A and B show an example of snapshots of ∆ F /F data and the optical flow vector fields ex-tracted. The magnitude and direction of the vector at each vector is computed by solving for the optimalvector field that describes the change from each frame to the subsequent frame (see schematic in Figure 2A).In order to minimize the effects of noise and numerical differentiation on the optical flow field, we applytemporal scaling and smoothing to the computed vector fields. Briefly, the magnitude of each optical flowvector is scaled proportionally to the relative change in the raw pixel intensity for the corresponding pixelover a prescribed time delay. This scaling attenuates the magnitudes of vectors that do not represent cor-responding changes in the widefield imaging data. To mitigate the effects of pixel noise, we also applytemporal Gaussian smoothing to the scaled vector fields. The scaled and smoothed Horn-Schunck opticalflow vector fields are used throughout the rest of the FLOW portrait algorithm where velocity data is re-quired. This process of computing the optical flow vector field from widefield imaging data is analogousto the process of extracting the motion vector field from particle image velocimtery (PIV) data [71, 72] inexperimental fluid dynamics. Both approaches approximate the velocity field from experimental data of material transported through the studied flow.
Once a flow velocity field, v ( x , t ) , is computed, there are numerous computational approaches that can beperformed to study and characterize the flow. These methods include instantaneous metrics from vectorcalculus, such as the divergence and the curl of the vector field, modal decomposition techniques [73, 74],such as POD and DMD, and Lagrangian metrics such as the FTLE [50, 52, 53]. Although instantaneous met-rics have the potential to extract relevant features from widefield imaging optical flow fields (SupplementalFigure 1; [62]), the unsteady nature of this data suggests that Lagrangian metrics may provide a more usefulsummary of the activity. Here we compute the FTLE fields [53] to extract time invariant features of flow-likewidefield activity.The FTLE field is a scalar field σ ( x , t , T ) defined at every point in space x and time t , with respectto some relevant time-scale of integration, T . The FTLE field is a measure of how much neighboring initialconditions separate when integrated through the velocity field v for a duration T . Thus, regions of highstretching for positive T (forward time) or negative T (backward time) provide time-varying analogs of sta-ble and unstable manifolds, respectively [34, 52, 53]. The FTLE field is typically approximated numericallyfrom flow field snapshots at discrete instants in time [53, 56]. First, the flow map Φ Tt is approximated on adiscretized set of spatial points, typically the same discretized domain where the velocity field is defined.The flow map Φ Tt describes the position of an initial condition x ( t ) after it is integrated along the vectorfield v for a duration T , and is defined as x ( t + T ) = Φ Tt ( x ( t )) = x ( t ) + (cid:90) t + Tt v ( x ( τ ) , τ ) dτ. (1)Next, the flow map Jacobian DΦ Tt is approximated via finite-difference derivatives with neighboring points5n the flow. In two-dimensions, the flow map Jacobian at a point x is: DΦ Tt ( x ) ≈ Φ Tx,t ( x +∆ x ) − Φ Tx,t ( x − ∆ x )2∆ x Φ Tx,t ( x +∆ y ) − Φ Tx,t ( x − ∆ y )2∆ y Φ Ty,t ( x +∆ x ) − Φ Ty,t ( x − ∆ x )2∆ x Φ Ty,t ( x +∆ y ) − Φ Ty,t ( x − ∆ y )2∆ y , (2)where Φ Tx,t denotes the x component of Φ Tt and Φ Ty,t denotes the y component. The finite-time Lyapunovexponent σ is finally computed from the largest eigenvalue λ max of the Cauchy-Green deformation tensor ∆ = (cid:16) DΦ Tt (cid:17) (cid:124) DΦ Tt , which is the maximum singular value of the flow map Jacobian: σ ( x , t , T ) = 1 T ln (cid:16)(cid:112) λ max [ ∆ ( x , t , T )] (cid:17) . (3)Figures 2 and 3 illustrate the intuition behind this FTLE computation, and additional implementationdetails are provided in the Methods. The key insight in the FTLE computation is that virtual particles atevery pixel location flow according to the vector field from t to t + T , and these integrated optical flowfields form a flow map Φ Tt (Figure 2B). This flow stretches neighboring virtual particles, so that equidistantparticles have stretched in some directions and compressed in others. Relative deformations are describedby the Cauchy-Green strain tensor at every pixel, and the FTLE corresponds to the log-normalized leadingeigenvalue of this tensor. The same procedure is repeated by reversing the ordering of frames to computeflow maps in backwards time. The forward and backward FTLE fields computed for each example timesnapshot are shown in Figure 3C and D.It is important that we build some intuition for how to interpret these FTLE fields. Drawing again onour analogy to physical fluid flows, ridges in the FTLE field correspond to time-varying analogs of invariantmanifolds, and they approximate LCS [51, 52]. In forward time, these features repel fluid material, similarto a stable manifold in a dynamical system. The opposite is true for backward time features, where materialis attracted, as with the unstable manifold. A similar interpretation can be extended to the FTLE of opticalactivity flows, where forward time structures are sources that repel activity, while backward time structuresare sinks to which activity is attracted. By aggregating the forward and backward FTLE ridges within a window in time, we summarize the coher-ent structures of propagating activity within that window with a single FLOW portrait. Ridges of an FTLEfield have been shown to approximate LCS, and several mathematical definitions are suggested to extractthem from data [50, 53]. We found that implementing existing strategies for ridge extraction on FTLE ofwidefield calcium imaging data did not adequately extract ridge-like features (Figure 3E). We developed apost-processing approach to take a threshold and skeletonize these ridges.Ridges lie along local extremes in a field, thus we can approximate their locations by extracting maximalregions and computing the skeleton structure. To compute the dominant features over the entire recording,we first threshold the mean of all non-negative FTLE values, to isolate local maxima in the field (Figure 4A).Next, we approximate ridges from the local FTLE maxima by performing a morphological skeletonizationoperation (Figure 4B). Lastly, these ridges are smoothed by applying morphological image processing (Fig-ure 4C). Thus, the resultant FLOW portrait depicts the average approximate FTLE ridges in a recordingwindow to summarize the time invariant patterns of activity.
We demonstrate the application of our approach on several optical widefield datasets, all recordings ofspontaneous calcium activation imaged from the cortical surface of transgenic mice. In each example, wehave chosen to focus on windows in time when bouts of activity are observed across large portions ofcortex. We show that FLOW portraits extracted from these windows summarize the extent and direction6igure 3: Steps to compute a FLOW portrait. Starting with widefield data preprocessed as ∆ F /F ( A ), op-tical flow is used to convert the frame-by-frame changes in pixel intensity to a vector field, shown zoomedin for the smaller area outlined with the yellow box ( B ). Next, the FTLE fields are computed in forwards( C ) and backwards ( D ) time. Ridges of these fields highlight coherent structures of the flow ( E ), and theseridges are used to compute the final FLOW portrait (see Figure 4). The forward time FTLE ridges (orange)highlight regions that repel flow, while the backward time ridges (purple) show regions that attract activity.Note that ridges in neighboring frames are similar but do vary in time.of calcium flow, highlighting cortical areas whose neural activations can be interpreted in the context of thebehavioral and developmental context of the animals. Pan-cortical waves are bouts of activity that propagate across large areas of the cortex [5, 75–78] and aresuggested to play a critical role in cortical development [33]. These events are defined heuristically bysimultaneous activity of a large fraction of the imaged cortical surface. In Figure 5A, the gray bars highlightindividual cortical wave events, defined as when the fraction of active cortex rises to above / and fallsback to the baseline ( ∼ / ). To contribute to our understanding of pan-cortical waves in development, weuse FLOW portraits to summarize the activity during each wave event, thus facilitating direct comparisonsacross individual waves and developmental time points.We construct FLOW portrait to summarize the flow of activity during each pan-cortical wave. Spatialintegration of the FTLE fields yields the FTLE intensity (Figure 5A, orange and purple traces), which indi-cates the relative amount of time-averaged flow throughout the recording. In particular, when the forwardFTLE intensity is high there is a relatively large amount number of sources of activity; the same appliesto high backward FTLE intensity. The resulting FLOW portraits for two pan-cortical waves can be seen inFigure 5B, alongside 12 frames of the ∆ F /F data from each wave (see also Supplemental Movies 1 and 2).The portraits of every pan-cortical wave are shown in Supplemental Figure 2.7igure 4: Ridges in the FTLE field within an analysis window are summarized to form the FLOW portrait.The FTLE field is first thresholded at every frame and then averaged over all frames in the window. Ridgesin the FTLE are approximated by performing a skeletonization of the local maxima. FLOW portraits areproduced by morphological image processing to smooth the approximate FTLE ridges. Note that forwardtime (orange) FTLE and backward time (purple) portraits are computed separately and superimposed forvisualization.Each FLOW portrait provides a summary of the prominent activity observed during each wave event,highlighting the source (forward FTLE, orange) and sink (backward FTLE, purple) features. Indeed, bothwaves shown in Figure 5B exhibit two stages of propagation, where activity spreads and pauses briefly atsensorimotor cortex (outlined by the purple rings) before spreading towards frontal cortex. This concisevisualization allows us to easily compare such qualitative features of wave propagation without having toparse through the raw data manually.
To further investigate the role of spontaneous cortical activity during development, we analyzed opticalrecordings of spontaneous calcium activity in mouse pups during the first 8 postnatal days of development.We computed FLOW portraits on bouts of spontaneous cortical activity during sleep in 12 animals of agesP1, P2, P3, P5, P7 and P8 (Figure 6). Briefly, the sleep state was determined by binning time points intothree categories (sleep, wake, and moving-wake) using the power of nuchal EMG spectrum [33, 79, 80]. Wechose to focus on sleep state cortical activity for its proposed developmental roles and observed changesduring development [33]. For each animal, we computed FLOW portraits for up to the 10 longest bouts ofsleep (fewer portraits were computed for short recordings where there were less than 10 sleep bouts).Five (5) example FLOW portraits for each animal are in Figure 6A, with a complete set in SupplementalFigure 3. This organization allows us to leverage FLOW portraits to examine developmental changes incortical activity across long recordings from different animals. We observe a qualitative change betweenthe portraits from the early postnatal days (P1–3) to the later days (P5–8). The FLOW portraits from theearly days show more diffuse activity, with less consolidated FTLE ridges. After P5, the FLOW portraitsshow cortical activity during sleep becoming more consolidated and following more defined flow patterns.Thus, we may easily visualize how cortical activity during sleep undergoes a substantial change duringdevelopment.
Lastly, we analyze the FLOW portraits of spontaneous cortical activity in a head-fixed, behaving adultmouse [13]. To investigate how FLOW portraits align with an animal’s behavior, we analyze IR videosof spontaneous facial and limb movements alongside cortical calcium activity. A movement score was8igure 5: Pan-cortical wave events in a P7 mouse pup summarized as FLOW portraits. ( A ) Pan corticalwaves are defined as events where the fraction of active cortex (black-trace) exceeds 50-percent. ( B ) FLOWportraits are shown for two example waves, indicated as i and ii in A . Orange shows forward time FTLEridges where calcium activity propagates from. Purple ring backward time FTLE ridges where calciumactivity propagates towards.assigned to each recording time point by using the total pixel-wise difference between the current and nextframes (the forward difference) and normalizing this to the maximum observed difference. During bouts oflimb movement or whisking the movement score was greater, approaching the maximum score of 1, thanduring periods of rest, when the score approached the minimum score of 0.We chose two bouts of spontaneous movement (gray shading in Figure 7A highlights the two bouts,i and ii) to compute the corresponding FLOW portraits (see also Supplemental Movies 3 and 4). Largevariations in the movement score (Figure 7A, blue trace) can be observed throughout these bouts, indicatingthat the animal is continuously switching from a resting to a moving state.We see signatures of these movement behaviors in the calcium activity, when we expect sensorimotorcortical regions to be more active than during periods of rest. Indeed, the FLOW portrait for each activitybout provides a clear summary of calcium activity surrounding the sensorimotor cortex (Figure 7B). Dur-ing both bouts, a ring-like repelling (forward, orange) field line outlines the sensory-motor region, whileattracting (backward, purple) field lines fill in the centers of the rings. We note that these patterns are dif-9igure 6: FLOW portraits highlight developmental changes of sleep-state cortical activity. FLOW portraitsfor 5 sleep bouts from 12 P 1-8 mouse pups are shown. During the first 2-3 postnatal days, activity is diffuse,as indicated by many short-length structures in the FLOW portrait. As animals grow older (postnatal days5-8), sleep-state cortical activity becomes more structured, as indicated by a consolidation of features in theFLOW portraits. Orange indicates repelling or source structures, and purple indicates attracting or sinkstructures. All images are of the left-hemisphere, such that the mid-line and anterior direction are orientedtowards and left of the images, respectively.ferent from our analysis of Example 1 of pan-cortical waves. Specifically, these features suggest a dominantpattern of cortical calcium activity as diffusion of activity from the entirety (or outer edges) of sensorimotorregions towards the center. In other words, our FLOW portraits point to sensory-motor cortex as a sinkof cortical activity during spontaneous movement behaviors. Interestingly, compared to the overlaid AllenCommon Coordinate Framework (white lines in Figure 7B), the attracting (backward, purple) field lines areclose to the boundary between somatosensory and primary motor cortices.10igure 7: Examples of spontaneous cortical calcium activity associated with movements of an adult mousesummarized as FLOW portraits. ( A ) A movement score extracted from IR video of the mouse movingspontaneously in the dark shows bouts of large movements among more quiescent periods. These boutsof movements do not correspond necessarily to when a large fraction of the cortical surface is active (seeMethods for threshold criteria). ( B ) FLOW portraits for two bouts involving spontaneous movements la-beled i and ii show coherent structures that highlight activity appear in sensorimotor regions and are thenattracted to the centers of these regions bilaterally. Boundaries aligned to the Allen Common CoordinateFramework [81] are overlaid in white. This paper introduces FLOW portraits as a novel approach to visualize spatiotemporal flow of coherentfeatures in optical widefield calcium imaging data. Viewed at this meso-scale of temporal and spatial reso-lution, neural activity at the cortical surface is typified by multiple brain regions activating transiently andsometimes in spatial succession. Motivated by an analogy between this flow of neural activity over cortexand physical fluid flows, we leverage techniques well established to study physical fluid flows, in particularthe finite-time Lyapunov exponent (FTLE). Here we convert movies of ∆ F/F over the cortical surface intovector fields, and the FTLE ridges in these vector fields form an intuitive map of dynamic calcium activity.Importantly, our FLOW portraits do not decompose the data into modes and are not models of the data.Instead, they capture succinct portraits of diverse, variable, and non-stationary spatiotemporal patterns,such as those often observed in spontaneous or task-driven widefield calcium imaging experiments.The FLOW portrait analysis makes several assumptions that are usually true of physical fluid systemsbut often not met by neural data. Coherent propagation of neural activity on the cortex does not obey mass11f energy conversation, so the extraction of FTLE ridges are only approximate “material” accumulationlines. This assumption is particularly invalid for long bouts of data and over long integration windows, socaution must be exercised in choosing these parameters in the analysis (the same is true of FTLE analysisin fluid flows). Further, although widefield imaging offers much larger fields of view at a higher temporalresolution than many other imaging methods, there remains much unobservable neural activity. Brainareas outside the imaging window and underneath the cortical surface contribute to the imaged activity,yet the flow of neural activity among these regions cannot be captured by our analysis and may bias theextracted flow lines. This limitation is more severe in considering brains with sulci and gyri, as our analysisfundamentally assumes that neighboring pixels are also neighbors on the cortical sheet.The quality and interpretability of FLOW portraits requires the imaging data to have been acquiredwith sufficient temporal and spatial resolution to support the analysis. To be specific, we require that thesampling in time to be fast enough that successive frames of the movie are very similar. If the frame rateis too slow and neighboring frames differ substantially, then the optical flow computation infers inaccu-rate vector fields and can no longer disambiguate between gradual flow of activity and sudden jumps inactivation. Despite the relatively slow dynamics of GCaMP6s compared to single neuron activity [22], thetemporal dynamics of lasting neural synchrony at this meso-scale is adequately matched to the kineticsof the indicator protein in all the data we highlight here. The choice of calcium or voltage indicator alsointroduces filtering in time, so that our analysis relies on the dynamics of the indicator to be faster thanthe dynamics of the underlying flow across the brain. Similarly, the spatial resolution of the data need notsupport disambiguation of single neurons, but it is important that spatial averaging in the field of viewdoes not obscure coherent features of interest.We suggest our approach expands our toolbox of techniques to analyze and understand widefieldimaging data, especially facilitating direct comparison of multiple bouts of spatiotemporal activity thatare interpretability in the context of behavior and development. This visualization framework can be de-veloped to explicitly quantify features of the flow. Such quantification may be of value in further workthat connects features of FLOW portraits with states of relevance to behavior, development, or disease.The transformation of widefield calcium imaging data into a vector field representation suggests multipleavenues for development of analytic tools. Intriguingly, it may be possible to discover partial differentialequations that govern the flow of activity through these vector fields using data-driven techniques [82, 83].
These experimental procedures were conducted at University of Washington, and all protocols were re-viewed and approved by the University of Washington IACUC. Neonatal mice expressing GCaAMP6S incortical neurons were bred by crossing mice heterozygous expressing a Emx1 driven Cre (Emx1-Cre+/-,Jackson Labs ID 005628) with mice homozygously expressing GCaAMP6S under control of a cre promoter(Ai162+/+, Donated by Allen Institute, Jackson Labs ID 031562). This cross resulted in mice expressingGCaAMP6S primarily in glutamatergic cortical neurons early in development. On the day of recording,mice were placed on a heating pad and anesthetized using 1–2% isoflurane carried by 100% O2, while localanesthetic bupivacaine was delivered subcutaneously at the scalp. The skin over the cortex was removedover a window spanning between the ears to just above the eyes of the pup, to reveal the skull. The pe-riosteum was then removed with fine tip forceps and cotton swabs. At this developmental stage, the skullis uncalcified and largely transparent, so thinning or cutting a window was unnecessary. A stainless steelU-shaped bracket was then attached to the skull with cyanoacrylate glue. The bracket was clamped in placeto the heating pad and stage to stabilise the head. To prevent the skull from drying and to preserve clarity,the exposed skull was also sealed with a thin layer of cyanoacrylate. Silver wire hook leads were implantedin to the nuchal muscle through the same incision to monitor neck electromyography (EMG).Once glue had dried, isoflurane anesthesia was removed and the pup along with heating pad and stagewas positioned for imaging on a Nikon AZ100 with 2X objective and 0.6X reducer. Nuchal EMG activitywas amplified with and AM Systems Model 1700 amplifier (10Hz high pass, 60Hz notch, 10kHz low pass)12nd was sampled at 10kHz using a Powerlab 4/26 and Labchart v8 (AD Instruments). GCaMPP6s activitywas excited using an Intensilight mercury lamp (Nikon), captured using an CCD camera (ORCA Flash2.8), and recorded using the HCImage application (Hamamatsu). Frame capture rates varied from 10–50Hz with maximum exposure times (100–20ms, respectively). To further increase signal to noise ratio,the camera was set to perform online hardware based pixel binning, reducing a 1920 × × ∆ F /F image stacks for FLOWportrait analysis. Briefly, imaging runs were further downsampled by pixel binning the 960 × × F for each frame; each pixel in F was set to the minimum value for that pixel across the 40-sec window. ∆ F was calculated as the difference between the raw pixel intensity and this calculated moving minimum.The difference was then normalized to relative change by dividing ( ∆ F /F ). A small Gaussian spatial blurwas used to attenuate “speckled” noise. Region of interests (ROI) masks of the visible cortical surface weregenerated by excluding any pixel whose mean-to-variance ratio was greater than 400:1. This value wasdetermined heuristically to optimize exclusion of any pixels that displayed minimal change in fluorescenceover time, such as those that lie outside the cortical window.
These experimental procedures were conducted at UCL according to the UK Animals Scientific ProceduresAct (1986) and under personal and project licenses granted by the Home Office following appropriate ethicsreview. The dataset and associated procedures was described previously [13]. In brief, the data were froman adult (30 weeks) male mouse expressed GCaMP6s in excitatory neurons (tetO-GCaMP6s; CaMK2a-tTagenotype [14]). The mouse was implanted with a metal headplate, plastic light isolation chamber, andtransparent covering over the dorsal skull. On the day of recording, the mouse was head-fixed underthe microscope on a stable seat with a rubber wheel underneath the forelimbs. Video cameras capturedthe frontal aspect of the mouse as well as its eye. Imaging was conducted at 70 Hz with alternating blueand violet illumination, and imaging data was corrected for hemodynamic components. The data wereprocessed by singular value decomposition (SVD) compression.The images were aligned to the Allen Common Coordinate Framework [81] by manually identifyingBregma and the orientation of the midline in the images. Bregma was taken to be located at the coordinate5.7 mm AP in the CCF. Since the pixel size in the camera was known (21.7 m / pixel), the CCF regionboundaries could then be overlaid on the images.
Pan-cortical, as defined by [33], are cortical activity events where recorded activity spreads over a large areaof the imaged cortex. We defined large cortical area to be when 50 percent of the cortical pixels (pixels whichshow the cortex) are active. At any time point, a pixel is active if its intensity is more than one standarddeviation above the temporal mean for that pixel. To extract pan-cortical wave events, we computed thefraction of active cortical pixels throughout the recording, and noted the time points where the active areaexceeded the 50 percent threshold. Each wave event was then defined by the time points when the activearea crossed 10 percent active prior to the time of crossing the 50 percent threshold and the time whenthe active area crossed this 10 percent lower bound following the peak. Overlapping wave events weremerged into a signal pan-cortical wave to avoid redundancy. Furthermore, events that lasted less than theFTLE integration length ( T ) plus the optical flow scaling delay ( . sec or frames for the mouse pupdata; . sec or frames for the adult mouse data) were not analyzed because the FTLE and optical flowcomputations require longer bouts of data. 13 .2.2 Sleep bouts during developmental Sleep state cortical activity was segmented using the nuchal EMG as an indicator of state (sleep or awake).Time points were clustered into three groups based on the nuchal EMG power spectrum as in [33, 79, 80],where the lowest power group is known to represent the sleep state. We defined a sleep bout as a period ofcontinuous classification in the sleep state, and extracted the 10 longest bouts from each recording over thedevelopmental time span. Any bout that did not meet the FTLE and optical flow length requirement ( . sec or frames for the mouse pup data; . sec or frames for the adult mouse data) was not analyzedfurther. In cases when there were less than 10 bouts that met the length requirement, we chose to includefewer sleep bouts for that recording. We extracted movement events from video of the face and front arms of the adult mouse during the wide-field imaging experiment. We defined a movement score for each time point in the video based on thedifference between the current time point and the previous time point. Each video frame was assigned amovement score given by the sum (over all pixels in the frame) of the difference between the current andprevious frame. For time point t , the score is given by MovementScore t = (cid:80) pixels ( I t − I t − ) , where I is thepixel intensity for each of the pixels in the frame. The time series of movement scores was normalized to themaximum observed value for ease of interpretation and visualization. Timestamps of video frames weredetermined by recording TTL pulses emitted by the camera on each exposure, for both calcium imagingand behavioral videos. We then compared cortical activity across varying movement regimes. We computed optical flow vector fields using the Horn-Schunck optical flow algorithm [70] implementedin MATLAB [84]. Two parameters must be supplied to the optical flow algorithm: the maximum number ofiterations and the α smoothness parameter. Values for both parameters were selected such that the errorsin the Horn-Schucnk minimization problem (see [70] for details) were simultaneously minimized. We setthe maximum number of iterations to 100 and α to for all computations. To minimize the effects noise on optical flow fields, we applied an activity-based scaling to the magnitudesof the optical flow vectors. First, we created a time series of weights for each pixel by normalizing changein raw pixel intensity between the current time and the intensity of that pixel . seconds in the past tothe maximum observed change. We chose a time delay of . and . seconds, for the developmental andadult mouse datasets respectively, to empirically to match the time scale of large changes observed in theraw data. Next, we took the sliding windowed average, over a window of . seconds, of the weightsin order to further reduce the effects of recording noise. We then scaled the magnitude of the optical flowvectors by applying the weights to the corresponding vector. Lastly, we temporally smoothed the opticalflow fields using a 5-point Gaussian window created with MATLAB’s guasswin() function. The gausswinfunction takes an additional parameter, α , which is proportional to the inverse of the standard deviation ofthe Gaussian smoothing kernel. We set this parameter to . for all smoothing operations for it observedability to reduce noise in the processed vector fields. We computed the FTLE of all vector fields using the LCS Tool [66] ( https://github.com/jeixav/LCS-Tool ) MATLAB software package. We computed the FTLE using an integration length of . seconds( frames) for the developing mouse data and an integration length of ∼ . seconds ( frames) for theadult mouse data. To chose the integration length T , we followed the criteria outlined in [53] of choosing avalue such that the FTLE ridges are sufficiently resolved. Using a sample of each dataset, we computed the14TLE for a range of integration lengths ( to frames) and visualized the resulting FTLE fields. We thenchose the smallest integration length where the corresponding FTLE field had well resolved, sharp, ridges. FLOW portraits are constructed through several steps processing steps that extract ridges from the FTLEfield (Figure 4 shows the intermediate processing steps for an example segment of data). We perform thesame steps on both the forward and backward FTLE fields separately and overlay them to create the finalFLOW portrait. For each segment of data, the forward and backward FTLE fields are separately averagedin time to aggregate the flow features into two mean FTLE images. Next, we isolate possible ridge-likefeatures by binarizing the mean FTLEs using a single threshold value. The threshold value is set to the valuerepresenting the percentile between 85 and 95 percent. The exact percentile is chosen to produce the binaryimage that depicts the clearest ridge-like features. We then perform two sets of morphological operationson the each (forward and backward) binary mean FTLE, using MATLAB’s bwmorph() function, to extractapproximate ridges and spatially smooth the images. The bwmorph() (see ) function applies a specified morphological operation a givennumber of times (specified by the n parameter), or until the input image remains unchanged ( n = Inf bydefault). Unless otherwise specified, we performed morphological operations until the image no longerchanged. First, we skeletonize the binary mean FTLE by performing the (cid:48) close (cid:48) , the (cid:48) thin (cid:48) , and the (cid:48) skel (cid:48) (performed operation with n = 4 ) operations sequentially. We then smooth the skeletonized image toproduce the FLOW portrait by performing the (cid:48) diag (cid:48) , the (cid:48) spur (cid:48) , and the (cid:48) close (cid:48) operations sequentially.We found that these two series of operations provide strong approximations to the ridge features that weobserve in the FTLE fields. Lastly, we overlay the processed forward and backward images to create thefinal FLOW portrait. Our code is publicly available without restriction, other than citation, on Github at https://github.com/natejlinden/FLOWPortrait . The code and data in this repository can reproduce all main analy-ses, findings, and figures from our paper.
Acknowledgements
We are grateful for helpful discussion with Aditya Nair, Kameron Decker Harris, and Seth Hirsh. NJL ac-knowledges support through a Neuroengineering Undergraduate Research Fellowship from the Universityof Washington Institute of Neuroengineering (UWIN) and the Washington Research Foundation Funds forInnovation in Neuroengineering. DRT acknowledges funding support from UW Neuroscience GraduateProgram (T32NS099578) and the UW Computational Neuroscience Center (5T90DA032436). NAS was sup-ported by the Human Frontiers Science Program (Fellowship LT001071), and the European Unions Horizon2020 research and innovation programme (Marie Sklodowska-Curie fellowship 656528). SLB acknowledgesfunding support from the Army Research Office (ARO W911NF-19-1-0045). BWB acknowledges fundingfrom the Washington Research Foundation, the Alfred P. Sloan Foundation, and Weill Neurohub.
Author contributions
NJL, SLB, and BWB conceived of the study and designed the analyses. NJL carried out the analyses. DRT,NAS, and WJM collected the imaging data and helped interpret the results. NJL and BWB wrote the paper,and all authors contributed to editing the manuscript.
References [1] J. B. Wekselblatt, E. D. Flister, D. M. Piscopo, and C. M. Niell, “Large-scale imaging of cortical dynamics duringsensory perception and behavior,”
Journal of neurophysiology , 2016.
2] L. Muller, F. Chavane, J. Reynolds, and T. J. Sejnowski, “Cortical travelling waves: mechanisms and computationalprinciples,”
Nature Reviews Neuroscience , vol. 19, no. 5, p. 255, 2018.[3] S. Musall, M. T. Kaufman, S. Gluf, and A. K. Churchland, “Movement-related activity dominates cortex duringsensory-guided decision making,”
BioRxiv , p. 308288, 2018.[4] R. Corlew, M. M. Bosma, and W. J. Moody, “Spontaneous, synchronous electrical activity in neonatal mouse corti-cal neurones,”
The Journal of physiology , vol. 560, no. 2, pp. 377–390, 2004.[5] J. Conhaim, E. R. Cedarbaum, M. Barahimi, J. G. Moore, M. I. Becker, H. Gleiss, C. Kohl, and W. J. Moody, “Bi-modal septal and cortical triggering and complex propagation patterns of spontaneous waves of activity in thedeveloping mouse cerebral cortex,”
Developmental neurobiology , vol. 70, no. 10, pp. 679–692, 2010.[6] H. J. Luhmann, A. Sinning, J.-W. Yang, V. Reyes-Puerta, M. C. St ¨uttgen, S. Kirischuk, and W. Kilb, “Spontaneousneuronal activity in developing neocortical networks: from single cells to large-scale interactions,”
Frontiers inneural circuits , vol. 10, p. 40, 2016.[7] L. F. Rossi, R. C. Wykes, D. M. Kullmann, and M. Carandini, “Focal cortical seizures start as standing waves andpropagate respecting homotopic connectivity,”
Nature communications , vol. 8, no. 1, pp. 1–11, 2017.[8] J. V. Cramer, B. Gesierich, S. Roth, M. Dichgans, M. D ¨uring, and A. Liesz, “In vivo widefield calcium imagingof the mouse cortex for analysis of network connectivity in health and brain disease,”
Neuroimage , vol. 199, pp.570–584, 2019.[9] A. McGirr, J. LeDue, A. W. Chan, Y. Xie, and T. H. Murphy, “Cortical functional hyperconnectivity in a mousemodel of depression and selective network effects of ketamine,”
Brain , vol. 140, no. 8, pp. 2210–2225, 2017.[10] A. G. Siapas and M. A. Wilson, “Coordinated interactions between hippocampal ripples and cortical spindlesduring slow-wave sleep,”
Neuron , vol. 21, no. 5, pp. 1123–1128, 1998.[11] D. A. McVea, T. H. Murphy, and M. H. Mohajerani, “Large scale cortical functional networks associated withslow-wave and spindle-burst-related spontaneous activity,”
Frontiers in neural circuits , vol. 10, p. 103, 2016.[12] D. Shimaoka, N. A. Steinmetz, K. D. Harris, and M. Carandini, “The impact of bilateral ongoing activity on evokedresponses in mouse cortex,”
ELife , vol. 8, p. e43533, 2019.[13] P. Zatka-Haas, N. A. Steinmetz, M. Carandini, and K. D. Harris, “A perceptual decision requires sensory but notaction coding in mouse cortex,” bioRxiv , p. 501627, 2020.[14] J. B. Wekselblatt and C. M. Niell, “Distinct functional classes of excitatory neurons in mouse v1 are differentiallymodulated by learning and task engagement,” bioRxiv , p. 533463, 2019.[15] M. Liu, C. Song, Y. Liang, T. Kn¨opfel, and C. Zhou, “Assessing spatiotemporal variability of brain spontaneousactivity by multiscale entropy and functional connectivity,”
NeuroImage , vol. 198, pp. 198–220, 2019.[16] H. Dana, T.-W. Chen, A. Hu, B. C. Shields, C. Guo, L. L. Looger, D. S. Kim, and K. Svoboda, “Thy1-gcamp6transgenic mice for neuronal population imaging in vivo,”
PloS one , vol. 9, no. 9, 2014.[17] J. N. Stirman, I. T. Smith, M. W. Kudenov, and S. L. Smith, “Wide field-of-view, multi-region, two-photon imagingof neuronal activity in the mammalian brain,”
Nature biotechnology , vol. 34, no. 8, pp. 857–862, 2016.[18] G. Silasi, D. Xiao, M. P. Vanni, A. C. Chen, and T. H. Murphy, “Intact skull chronic windows for mesoscopicwide-field imaging in awake mice,”
Journal of neuroscience methods , vol. 267, pp. 141–149, 2016.[19] N. A. Steinmetz, C. Buetfering, J. Lecoq, C. R. Lee, A. J. Peters, E. A. Jacobs, P. Coen, D. R. Ollerenshaw, M. T.Valley, S. E. De Vries et al. , “Aberrant cortical activity in multiple gcamp6-expressing transgenic mouse lines,” eneuro , 2017.[20] J. Nakai, M. Ohkura, and K. Imoto, “A high signal-to-noise ca 2+ probe composed of a single green fluorescentprotein,”
Nature biotechnology , vol. 19, no. 2, pp. 137–141, 2001.[21] L. Tian, S. A. Hires, T. Mao, D. Huber, M. E. Chiappe, S. H. Chalasani, L. Petreanu, J. Akerboom, S. A. McKinney,E. R. Schreiter, C. I. Bargmann, V. Jayaraman, K. Svoboda, and L. L. Looger, “Imaging neural activity in worms,flies and mice with improved gcamp calcium indicators,”
Nature methods , vol. 6, no. 12, p. 875, 2009.[22] T.-W. Chen, T. J. Wardill, Y. Sun, S. R. Pulver, S. L. Renninger, A. Baohan, E. R. Schreiter, R. A. Kerr, M. B. Orger,V. Jayaraman, L. L. Looger, K. Svoboda, and D. S. Kim, “Ultrasensitive fluorescent proteins for imaging neuronalactivity,”
Nature , vol. 499, no. 7458, pp. 295–300, 2013.[23] M. P. Vanni and T. H. Murphy, “Mesoscale transcranial spontaneous activity mapping in gcamp3 transgenic micereveals extensive reciprocal connections between areas of somatomotor cortex,”
Journal of Neuroscience , vol. 34,no. 48, pp. 15 931–15 946, 2014.
24] D. A. McVea, M. H. Mohajerani, and T. H. Murphy, “Voltage-sensitive dye imaging reveals dynamic spatiotem-poral properties of cortical activity after spontaneous muscle twitches in the newborn rat,”
Journal of Neuroscience ,vol. 32, no. 32, pp. 10 982–10 994, 2012.[25] C. Song, D. M. Piscopo, C. M. Niell, and T. Kn¨opfel, “Cortical signatures of wakeful somatosensory processing,”
Scientific reports , vol. 8, no. 1, pp. 1–12, 2018.[26] B. B. Scott, S. Y. Thiberge, C. Guo, D. G. R. Tervo, C. D. Brody, A. Y. Karpova, and D. W. Tank, “Imaging corticaldynamics in gcamp transgenic rats with a head-mounted widefield macroscope,”
Neuron , vol. 100, no. 5, pp. 1045–1058, 2018.[27] S. Musall, M. T. Kaufman, A. L. Juavinett, S. Gluf, and A. K. Churchland, “Single-trial neural dynamics are domi-nated by richly varied movements,”
Nature neuroscience , vol. 22, no. 10, pp. 1677–1686, 2019.[28] W. E. Allen, I. V. Kauvar, M. Z. Chen, E. B. Richman, S. J. Yang, K. Chan, V. Gradinaru, B. E. Deverman, L. Luo,and K. Deisseroth, “Global representations of goal-directed behavior in distinct cell types of mouse neocortex,”
Neuron , vol. 94, no. 4, pp. 891–907, 2017.[29] L. Pinto, K. Rajan, B. DePasquale, S. Y. Thiberge, D. W. Tank, and C. D. Brody, “Task-dependent changes in thelarge-scale dynamics and necessity of cortical regions,”
Neuron , vol. 104, no. 4, pp. 810–824, 2019.[30] E. A. Jacobs, N. A. Steinmetz, M. Carandini, and K. D. Harris, “Cortical state fluctuations during sensory decisionmaking,”
Biorxiv , p. 348193, 2018.[31] P. W. Wright, L. M. Brier, A. Q. Bauer, G. A. Baxter, A. W. Kraft, M. D. Reisman, A. R. Bice, A. Z. Snyder, J.-M. Lee,and J. P. Culver, “Functional connectivity structure of cortical calcium dynamics in anesthetized and awake mice,”
PloS one , vol. 12, no. 10, 2017.[32] M. P. Vanni, A. W. Chan, M. Balbi, G. Silasi, and T. H. Murphy, “Mesoscale mapping of mouse cortex revealsfrequency-dependent cycling between distinct macroscale functional modules,”
Journal of Neuroscience , vol. 37,no. 31, pp. 7513–7533, 2017.[33] D. R. Tabuena, R. Huynh, J. Metcalf, T. Richner, A. Stroh, B. W. Brunton, W. J. Moody, and C. R. Easton, “Pan-cortical waves in the neonatal rodent brain in vivo : A precursor of adult sleep waves?” In review., 2019.[34] P. Holmes and J. Guckenheimer,
Nonlinear oscillations, dynamical systems, and bifurcations of vector fields , ser. AppliedMathematical Sciences. Berlin, Heidelberg: Springer-Verlag, 1983, vol. 42.[35] R. Pang, B. J. Lansdell, and A. L. Fairhall, “Dimensionality reduction in neuroscience,”
Current Biology , vol. 26,no. 14, pp. R656–R660, 2016.[36] J. P. Cunningham and M. Y. Byron, “Dimensionality reduction for large-scale neural recordings,”
Nature neuro-science , vol. 17, no. 11, pp. 1500–1509, 2014.[37] E. L. Dyer, M. G. Azar, M. G. Perich, H. L. Fernandes, S. Naufel, L. E. Miller, and K. P. K¨ording, “A cryptography-based approach for movement decoding,”
Nature Biomedical Engineering , vol. 1, no. 12, pp. 967–976, 2017.[38] S. Ganguli and H. Sompolinsky, “Compressed sensing, sparsity, and dimensionality in neuronal information pro-cessing and data analysis,”
Annual review of neuroscience , vol. 35, pp. 485–508, 2012.[39] M. M. Churchland, J. P. Cunningham, M. T. Kaufman, J. D. Foster, P. Nuyujukian, S. I. Ryu, and K. V. Shenoy,“Neural population dynamics during reaching,”
Nature , vol. 487, no. 7405, pp. 51–56, 2012.[40] J. A. Gallego, M. G. Perich, L. E. Miller, and S. A. Solla, “Neural manifolds for the control of movement,”
Neuron ,vol. 94, no. 5, pp. 978–984, 2017.[41] J. A. Gallego, M. G. Perich, R. H. Chowdhury, S. A. Solla, and L. E. Miller, “Long-term stability of cortical popula-tion dynamics underlying consistent behavior,”
Nature neuroscience , vol. 23, no. 2, pp. 260–270, 2020.[42] J. P. Cunningham and Z. Ghahramani, “Linear dimensionality reduction: Survey, insights, and generalizations,”
The Journal of Machine Learning Research , vol. 16, no. 1, pp. 2859–2900, 2015.[43] B. W. Brunton, L. A. Johnson, J. G. Ojemann, and J. N. Kutz, “Extracting spatial–temporal coherent patterns inlarge-scale neural recordings using dynamic mode decomposition,”
Journal of neuroscience methods , vol. 258, pp.1–15, 2016.[44] J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton, and J. N. Kutz, “On dynamic mode decomposition:theory and applications,”
Journal of Computational Dynamics , vol. 1, no. 2, pp. 391–421, 2014.[45] J. N. Kutz, S. L. Brunton, B. W. Brunton, and J. L. Proctor,
Dynamic mode decomposition: data-driven modeling ofcomplex systems . SIAM, 2016.[46] C. J. MacDowell and T. J. Buschman, “Low-dimensional spatiotemporal dynamics underlie cortex-wide neural ctivity,” Current Biology , 2020.[47] S. Saxena, I. Kinsella, S. Musall, S. H. Kim, J. Meszaros, D. N. Thibodeaux, C. Kim, J. Cunningham, E. M. Hillman,A. Churchland et al. , “Localized semi-nonnegative matrix factorization (locanmf) of widefield calcium imagingdata,”
PLOS Computational Biology , vol. 16, no. 4, p. e1007791, 2020.[48] E. L. Mackevicius, A. H. Bahle, A. H. Williams, S. Gu, N. I. Denisenko, M. S. Goldman, and M. S. Fee, “Unsu-pervised discovery of temporal sequences in high-dimensional datasets, with applications to neuroscience,”
Elife ,vol. 8, p. e38471, 2019.[49] P. Zhou, S. L. Resendez, J. Rodriguez-Romaguera, J. C. Jimenez, S. Q. Neufeld, A. Giovannucci, J. Friedrich, E. A.Pnevmatikakis, G. D. Stuber, R. Hen et al. , “Efficient and accurate extraction of in vivo calcium signals from mi-croendoscopic video data,”
Elife , vol. 7, p. e28728, 2018.[50] G. Haller, “Lagrangian coherent structures from approximate velocity data,”
Physics of Fluids , vol. 14, no. 6, pp.1851–1861, June 2002.[51] S. C. Shadden,
Lagrangian Coherent Structures . John Wiley & Sons, Ltd, 2011, ch. 3, pp. 59–89. [Online]. Available:https://onlinelibrary.wiley.com/doi/abs/10.1002/9783527639748.ch3[52] G. Haller, “Lagrangian coherent structures,”
Annual Review of Fluid Mechanics , vol. 47, pp. 137–162, 2015.[53] S. C. Shadden, F. Lekien, and J. E. Marsden, “Definition and properties of lagrangian coherent structures fromfinite-time lyapunov exponents in two-dimensional aperiodic flows,”
Physica D: Nonlinear Phenomena , vol. 212, no.3-4, pp. 271–304, 2005.[54] M. Mathur, G. Haller, T. Peacock, J. E. Ruppert-Felsot, and H. L. Swinney, “Uncovering the Lagrangian skeleton ofturbulence,”
Physical Review Letters , vol. 98, pp. 144 502–1–144 502–4, 2007.[55] M. A. Green, C. W. Rowley, and G. Haller, “Detection of Lagrangian coherent structures in 3D turbulence.”
Journalof Fluid Mechanics , vol. 572, pp. 111–120, 2007.[56] S. L. Brunton and C. W. Rowley, “Fast computation of FTLE fields for unsteady flows: a comparison of methods,”
Chaos , vol. 20, p. 017503, 2010.[57] M. Farazmand and G. Haller, “Computing Lagrangian coherent structures from their variational theory,”
Chaos ,vol. 22, no. 013128, pp. 013 128–1–013 128–12, 2012.[58] J. Peng and J. Dabiri, “Transport of inertial particles by lagrangian coherent structures: application to predator–prey interaction in jellyfish feeding,”
Journal of Fluid Mechanics , vol. 623, pp. 75–84, 2009.[59] V. Duvernois, A. L. Marsden, and S. C. Shadden, “Lagrangian analysis of hemodynamics data from fsi simulation,”
International journal for numerical methods in biomedical engineering , vol. 29, no. 4, pp. 445–461, 2013.[60] S. C. Shadden and A. Arzani, “Lagrangian postprocessing of computational hemodynamics,”
Annals of biomedicalengineering , vol. 43, no. 1, pp. 41–58, 2015.[61] M. H. Mohajerani, A. W. Chan, M. Mohsenvand, J. LeDue, R. Liu, D. A. McVea, J. D. Boyd, Y. T. Wang, M. Reimers,and T. H. Murphy, “Spontaneous cortical activity alternates between motifs defined by regional axonal projec-tions,”
Nature neuroscience , vol. 16, no. 10, p. 1426, 2013.[62] N. Afrashteh, S. Inayat, M. Mohsenvand, and M. H. Mohajerani, “Optical-flow analysis toolbox for characteriza-tion of spatiotemporal dynamics in mesoscale optical imaging of brain activity,”
Neuroimage , vol. 153, pp. 58–74,2017.[63] R. G. Townsend and P. Gong, “Detection and analysis of spatiotemporal patterns in brain activity,”
PLoS computa-tional biology , vol. 14, no. 12, p. e1006643, 2018.[64] D. M. Ashby, J. LeDue, T. H. Murphy, and A. McGirr, “peripheral nerve ligation elicits widespread alterations incortical sensory evoked and spontaneous activity,”
Scientific reports , vol. 9, no. 1, pp. 1–10, 2019.[65] H. Jia, N. L. Rochefort, X. Chen, and A. Konnerth, “In vivo two-photon imaging of sensory-evoked dendriticcalcium signals in cortical neurons,”
Nature protocols , vol. 6, no. 1, pp. 28–35, 2011.[66] K. Onu, F. Huhn, and G. Haller, “Lcs tool: A computational platform for lagrangian coherent structures,”
Journalof Computational Science , vol. 7, pp. 26–36, 2015.[67] N. Paragios, Y. Chen, and O. D. Faugeras,
Handbook of mathematical models in computer vision . Springer Science &Business Media, 2006.[68] C. J. Duffy and R. H. Wurtz, “Sensitivity of mst neurons to optic flow stimuli. ii. mechanisms of response selectivityrevealed by small-field stimuli,”
Journal of neurophysiology , vol. 65, no. 6, pp. 1346–1359, 1991.
69] H. G. Krapp and R. Hengstenberg, “Estimation of self-motion by optic flow processing in single visual interneu-rons,”
Nature , vol. 384, no. 6608, pp. 463–466, 1996.[70] B. K. Horn and B. G. Schunck, “Determining optical flow,” in
Techniques and Applications of Image Understanding ,vol. 281. International Society for Optics and Photonics, 1981, pp. 319–331.[71] C. E. Willert and M. Gharib, “Digital particle image velocimetry,”
Experiments in fluids , vol. 10, no. 4, pp. 181–193,1991.[72] J. Westerweel, “Fundamentals of digital particle image velocimetry,”
Measurement science and technology , vol. 8,no. 12, p. 1379, 1997.[73] K. Taira, S. L. Brunton, S. Dawson, C. W. Rowley, T. Colonius, B. J. McKeon, O. T. Schmidt, S. Gordeyev, V. Theofilis,and L. S. Ukeiley, “Modal analysis of fluid flows: An overview,”
AIAA Journal , vol. 55, no. 12, pp. 4013–4041, 2017.[74] K. Taira, M. S. Hemati, S. L. Brunton, Y. Sun, K. Duraisamy, S. Bagheri, S. Dawson, and C.-A. Yeh, “Modal analysisof fluid flows: Applications and outlook,”
AIAA Journal , vol. 58, no. 3, pp. 998–1022, 2020.[75] O. Garaschuk, J. Linn, J. Eilers, and A. Konnerth, “Large-scale oscillatory calcium waves in the immature cortex,”
Nature neuroscience , vol. 3, no. 5, pp. 452–459, 2000.[76] J. Conhaim, C. R. Easton, M. I. Becker, M. Barahimi, E. R. Cedarbaum, J. G. Moore, L. F. Mather, S. Dabagh, D. J.Minter, S. P. Moen et al. , “Developmental changes in propagation patterns and transmitter dependence of waves ofspontaneous activity in the mouse cerebral cortex,”
The Journal of physiology , vol. 589, no. 10, pp. 2529–2541, 2011.[77] C. R. Easton, K. Weir, A. Scott, S. P. Moen, Z. Barger, A. Folch, R. F. Hevner, and W. J. Moody, “Genetic elimi-nation of gabaergic neurotransmission reveals two distinct pacemakers for spontaneous waves of activity in thedeveloping mouse cortex,”
Journal of Neuroscience , vol. 34, no. 11, pp. 3854–3863, 2014.[78] Z. Barger, C. R. Easton, K. E. Neuzil, and W. J. Moody, “Early network activity propagates bidirectionally betweenhippocampus and cortex,”
Developmental neurobiology , vol. 76, no. 6, pp. 661–672, 2016.[79] A. M. Seelke and M. S. Blumberg, “Developmental appearance and disappearance of cortical events and oscilla-tions in infant rats,”
Brain research , vol. 1324, pp. 34–42, 2010.[80] M. S. Blumberg, A. J. Gall, and W. D. Todd, “The development of sleep–wake rhythms and the search for elementalcircuits in the infant brain.”
Behavioral neuroscience , vol. 128, no. 3, p. 250, 2014.[81] Q. Wang, S.-L. Ding, Y. Li, J. Royall, D. Feng, P. Lesnar, N. Graddis, M. Naeemi, B. Facer, A. Ho et al. , “The allenmouse brain common coordinate framework: A 3d reference atlas,”
Cell , 2020.[82] S. H. Rudy, S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Data-driven discovery of partial differential equations,”
Science Advances , vol. 3, no. 4, p. e1602614, 2017.[83] H. Schaeffer, “Learning partial differential equations via data discovery and sparse optimization,” in
Proc. R. Soc.A , vol. 473, no. 2197. The Royal Society, 2017, p. 20160446.[84] M. Kharbat, “Horn-schunck optical flow method,”
MATLAB Central File Exchange