Michael Broxton

Wave optics theory and 3-D deconvolution for the light field microscope

Light field microscopy is a new technique for high-speed volumetric imaging of weakly scattering or fluorescent specimens. It employs an array of microlenses to trade off spatial resolution against angular resolution, thereby allowing a 4-D light field to be captured using a single photographic exposure without the need for scanning. The recorded light field can then be used to computationally reconstruct a full volume. In this paper, we present an optical model for light field microscopy based on wave optics, instead of previously reported ray optics models. We also present a 3-D deconvolution method for light field microscopy that is able to reconstruct volumes at higher spatial resolution, and with better optical sectioning, than previously reported. To accomplish this, we take advantage of the dense spatio-angular sampling provided by a microlens array at axial positions away from the native object plane. This dense sampling permits us to decode aliasing present in the light field to reconstruct high-frequency information. We formulate our method as an inverse problem for reconstructing the 3-D volume, which we solve using a GPU-accelerated iterative algorithm. Theoretical limits on the depth-dependent lateral resolution of the reconstructed volumes are derived. We show that these limits are in good agreement with experimental results on a standard USAF 1951 resolution target. Finally, we present 3-D reconstructions of pollen grains that demonstrate the improvements in fidelity made possible by our method.

SPED Light Sheet Microscopy: Fast Mapping of Biological System Structure and Function.

The goal of understanding living nervous systems has driven interest in high-speed and large field-of-view volumetric imaging at cellular resolution. Light sheet microscopy approaches have emerged for cellular-resolution functional brain imaging in small organisms such as larval zebrafish, but remain fundamentally limited in speed. Here, we have developed SPED light sheet microscopy, which combines large volumetric field-of-view via an extended depth of field with the optical sectioning of light sheet microscopy, thereby eliminating the need to physically scan detection objectives for volumetric imaging. SPED enables scanning of thousands of volumes-per-second, limited only by camera acquisition rate, through the harnessing of optical mechanisms that normally result in unwanted spherical aberrations. We demonstrate capabilities of SPED microscopy by performing fast sub-cellular resolution imaging of CLARITY mouse brains and cellular-resolution volumetric Ca(2+) imaging of entire zebrafish nervous systems. Together, SPED light sheet methods enable high-speed cellular-resolution volumetric mapping of biological system structure and function.

Enhancing the performance of the light field microscope using wavefront coding

Light field microscopy has been proposed as a new high-speed volumetric computational imaging method that enables reconstruction of 3-D volumes from captured projections of the 4-D light field. Recently, a detailed physical optics model of the light field microscope has been derived, which led to the development of a deconvolution algorithm that reconstructs 3-D volumes with high spatial resolution. However, the spatial resolution of the reconstructions has been shown to be non-uniform across depth, with some z planes showing high resolution and others, particularly at the center of the imaged volume, showing very low resolution. In this paper, we enhance the performance of the light field microscope using wavefront coding techniques. By including phase masks in the optical path of the microscope we are able to address this non-uniform resolution limitation. We have also found that superior control over the performance of the light field microscope can be achieved by using two phase masks rather than one, placed at the objectives back focal plane and at the microscopes native image plane. We present an extended optical model for our wavefront coded light field microscope and develop a performance metric based on Fisher information, which we use to choose adequate phase masks parameters. We validate our approach using both simulated data and experimental resolution measurements of a USAF 1951 resolution target; and demonstrate the utility for biological applications with in vivo volumetric calcium imaging of larval zebrafish brain.

Localizing a sensor network via collaborative processing of global stimuli

In order for nodes in a sensor network to meaningfully correlate their sensor readings, they must first determine their position in a globally shared coordinate system. Though there are many approaches which are suitable for achieving localization in the general case, sensor nodes are uniquely suited to use their sensing capabilities to aid them in this task. Global events which are detected in the environment surrounding the sensor network can serve as points of correspondence which, through collaborative processing on the network, provide nodes with sufficient information to compute their position. We have implemented an algorithm based on this approach in the Pushpin Computing sensor network: a dense, 55 node network which is spread over an area of 0.5 square meters. By queuing off of the minimum number of ultrasound pulses and light flashes needed to determine 2D coordinates using a simple lateration approach, we show that nodes in the Pushpin network can compute their position with an average error of 5-cm and a error standard deviation of 3-cm. In this paper we present this localization system and characterize its accuracy in our hardware testbed.

Localization on the pushpin computing sensor network using spectral graph drawing and mesh relaxation

This work approaches the problem of localizing the nodes of a distributed sensor network by leveraging distance constraints such as inter-node separations or ranges between nodes and a globally observed event. Previous work has shown this problem to suffer from false minima, mesh folding, slow convergence, and sensitivity to initial position estimates. Here, we present a localization system that combines a technique known as spectral graph drawing (SGD) for initializing node position estimates and a standard mesh relaxation (MR) algorithm for converging to finer accuracy. We describe our combined localization system in detail and build on previous work by testing these techniques with real 40-kHz ultrasound time-of-flight range data collected from 58 nodes in the Pushpin Computing network, a dense hardware testbed spread over an area of one square meter. In this paper, we discuss convergence characteristics, accuracy, distributability, and the robustness of this localization system.

A bayesian formulation for sub-pixel refinement in stereo orbital imagery

Generating accurate three dimensional planetary models is becoming increasingly more important as NASA plans manned missions to return to the moon in the next decade. This paper describes a stereo correspondence system for orbital images and focuses on a novel approach for the sub-pixel refinement of the disparity maps. Our method uses a Bayesian formulation that generalizes the Lucas-Kanade method for optimal matching between stereo pair images. This approach reduces significantly the pixel locking effect of the earlier methods and reduces the influence of image noise. The method is demonstrated on a set of high resolution scanned images from the Apollo era missions.

Experiences and directions in pushpin computing

Over the last three years we have built and experimented with the Pushpin computing wireless sensor network platform. The Pushpin platform is a tabletop multihop wireless sensor network testbed comprised of 100 nodes arbitrarily placed within a one-square-meter area. The Pushpin platforms concise form factor and extreme node density allow for fine-grained control of its environment and immediate user interaction, thereby uniquely situating it between simulated and real world sensor networks. This paper details our salient successes and lessons learned along the way. We also discuss how these experiences have shaped our vision of the future of wireless sensor networks and some concrete research directions to follow.

3D Lunar Terrain Reconstruction from Apollo Images

Generating accurate three dimensional planetary models is becoming increasingly important as NASA plans manned missions to return to the Moon in the next decade. This paper describes a 3D surface reconstruction system called the Ames Stereo Pipeline that is designed to produce such models automatically by processing orbital stereo imagery. We discuss two important core aspects of this system: (1) refinement of satellite station positions and pose estimates through least squares bundle adjustment; and (2) a stochastic plane fitting algorithm that generalizes the Lucas-Kanade method for optimal matching between stereo pair images.. These techniques allow us to automatically produce seamless, highly accurate digital elevation models from multiple stereo image pairs while significantly reducing the influence of image noise. Our technique is demonstrated on a set of 71 high resolution scanned images from the Apollo 15 mission.

Rapid, precise quantification of bacterial cellular dimensions across a genomic-scale knockout library.

BackgroundThe determination and regulation of cell morphology are critical components of cell-cycle control, fitness, and development in both single-cell and multicellular organisms. Understanding how environmental factors, chemical perturbations, and genetic differences affect cell morphology requires precise, unbiased, and validated measurements of cell-shape features.ResultsHere we introduce two software packages, Morphometrics and BlurLab, that together enable automated, computationally efficient, unbiased identification of cells and morphological features. We applied these tools to bacterial cells because the small size of these cells and the subtlety of certain morphological changes have thus far obscured correlations between bacterial morphology and genotype. We used an online resource of images of the Keio knockout library of nonessential genes in the Gram-negative bacterium Escherichia coli to demonstrate that cell width, width variability, and length significantly correlate with each other and with drug treatments, nutrient changes, and environmental conditions. Further, we combined morphological classification of genetic variants with genetic meta-analysis to reveal novel connections among gene function, fitness, and cell morphology, thus suggesting potential functions for unknown genes and differences in modes of action of antibiotics.ConclusionsMorphometrics and BlurLab set the stage for future quantitative studies of bacterial cell shape and intracellular localization. The previously unappreciated connections between morphological parameters measured with these software packages and the cellular environment point toward novel mechanistic connections among physiological perturbations, cell fitness, and growth.

Autonomous Robotic Inspection for Lunar Surface Operations

In this paper, we describe NASA Ames Research Center’s K10 rover as used in the 2006 Coordinated Field Demonstration at Meteor Crater, Arizona. We briefly discuss the control software architecture and describe a high dynamic range imaging system and panoramic display system used for the remote inspection of an EVA crew vehicle.