Juan Castorena

Autocalibration of lidar and optical cameras via edge alignment

We present a new method for joint automatic extrinsic calibration and sensor fusion for a multimodal sensor system comprising a LIDAR and an optical camera. Our approach exploits the natural alignment of depth and intensity edges when the calibration parameters are correct. Thus, in contrast to a number of existing approaches, we do not require the presence or identification of known alignment targets. On the other hand, the characteristics of each sensor modality, such as sampling pattern and information measured, are significantly different, making direct edge alignment difficult. To overcome this difficulty, we jointly fuse the data and estimate the calibration parameters. In particular, the joint processing evaluates and optimizes both the quality of edge alignment and the performance of the fusion algorithm using a common cost function on the output. We demonstrate accurate calibration in practical configurations in which depth measurements are sparse and contain no reflectivity information. Experiments on synthetic and real data obtained with a three-dimensional LIDAR sensor demonstrate the effectiveness of our approach.

Sampling of Time-Resolved Full-Waveform LIDAR Signals at Sub-Nyquist Rates

Third-generation full-waveform (FW) light detection and ranging (LIDAR) systems collect time-resolved 1-D signals generated by laser pulses reflected off of intercepted objects. From these signals, scene depth profiles along each pulse path can be readily constructed. By emitting a series of pulses toward a scene using a predefined scanning pattern and with the appropriate sampling and processing, an image-like depth map can be generated. Unfortunately, massive amounts of data are typically acquired to achieve acceptable depth and spatial resolutions. The sampling systems acquiring this data, however, seldom take into account the underlying low-dimensional structure generally present in FW signals and, consequently, they sample very inefficiently. Our main goal and focus here is to develop efficient sampling models and processes to collect individual time-resolved FW LIDAR signals. Specifically, we study sub-Nyquist sampling of the continuous-time LIDAR FW reflected pulses, considering two different sampling mechanisms: 1) modeling FW signals as short-duration pulses with multiple band-limited echoes; and 2) modeling them as signals with finite rates of innovation.

Using finite moment rate of innovation for LIDAR waveform complexity estimation

LIDAR sensing collects data to obtain detailed topographical information about a region. A major challenge here is the large amount of data that needs to be collected for accurate surface reconstruction which in turn imposes significant storage, processing and transmission requirements. Current efforts to solve this problem have been focusing on applying compression algorithms to the range measurements. This process, however, requires the collection of large amounts of data, most of which is ultimately discarded. Instead, our approach is to apply a compressive sensing paradigm to sparsely sample the scene using the estimated pulse complexity to characterize the scene complexity, thus determining the number of samples needed for accurate reconstruction. As a first step towards this goal, we characterize here the individual laser return pulses for scenes of varying complexity, assuming that the pulses are composed by a finite number of innovations. Each scene is thus sampled at a rate that is much lower than the traditional Nyquist/Shannon limit. Our results show that accurate classifications of waveform and scene complexity with the simple nearest neighbor classifier are obtained using the proposed algorithm.

Modeling lidar scene sparsity using compressive sensing

One of the major problems associated with LIDAR sensing is that significant amounts of data must be collected to obtain detailed topographical information about a region. Current efforts to solve this problem have focused on designing compression algorithms which operate on the collected data. These, however, require the collection of large amounts of data only to discard most of it in some transformed domain. Instead, compressive sensing has demonstrated that highly accurate signal reconstructions are achievable even when sampling below the Nyquist rate. Such sensing is clearly desirable for LIDAR range data compression if it can be achieved. One notes, however, that compressive sensing requires a priori knowledge of the sparsifying basis of the signal which is a major problem for LIDAR since that basis depends not only on the underlying scene complexity but also on the laser spot size and target distance. For these reasons, the goal of this research is to take the first steps in establishing a relationship between typical LIDAR scenes of varying complexity and the sparsity of the scene compressively sampled.

The Restricted Isometry Property for Banded Random Matrices

In this paper, we investigate the problem of determining the conditions under which the restricted isometry property (RIP) is satisfied for a particular type of matrix referred to in here as a banded random matrix (BRM). Such matrices have been recognized as suitable models for a number of compressive-sensing based sampling architectures, including the interleaved random demodulator, the random demodulator, the parallel non-interleaved random demodulator, the random sampler, and the periodic nonuniform sampler. It is thus important to establish the conditions under which the BRM satisfies the RIP; to our knowledge, this question has not been theoretically addressed in the literature. If the resulting conditions are satisfied, full signal recovery using a convex optimization algorithm is guaranteed. The specific objective of this research is to determine the conditions under which the RIP is satisfied for two possible sampling matricies: a BRM and a BRM multiplied by the discrete Fourier matrix.

Random impulsive scan for lidar sampling

In this paper, we address the problem of sampling the LIDAR range map. The significance of this problem is based upon the fact that large datasets generated by sampling inefficiently impose storage, processing and transmission limitations. Current compression approaches addressing these issues rely on collecting large amounts of data to analyze and throw away the redundancies. Unfortunately, the sampling performed by these approaches is still inefficient. Our approach to compression consists instead, on using a random uniform impulsive scan sampling scheme with sampling densities depending on the surface complexity. Turns out that the number of scanned time-resolved waveforms required to achieve “good” approximations, needs to satisfy the bound equation, where equation is an estimator of the surface numerical rank. Such a bound allows one to sub-sample surfaces which are close to a low-rank space and update, on the fly, the sampling densities required to improve the quality of the approximation.

Quantifying the accuracy of FRI-based LIDAR waveform analysis

Third generation full-waveform (FW) LIDAR systems collect time-resolved 1D signals generated by laser pulses reflecting off of intercepted objects. From these signals, scene depth profiles along each pulse path can be readily constructed. Using the conventional sampling process, however, massive amounts of data are typically required in order to achieve acceptable depth and spatial resolutions, and this data must be stored, transmitted, and processed. We have previously shown that such signals can be sampled at sub-Nyquist rates by using a finite rate of innovations (FRI) model. That work, however, used LIDAR data for which ground-truth distances were not available and it was therefore not possible to fully evaluate the range precision of an FRI-based representation. Here, we apply the proposed methodology to carefully ground-truthed laboratory data in order to better characterize its capabilities and limitations.

Sub-spot localization for spatial super-resolved LIDAR

Third generation LIDAR full-waveform (FW) based systems collect 1D temporal profiles of laser pulses reflected by the intercepted objects to construct depth profiles along each pulse path. By emitting a series of pulses towards a scene using a predefined scanning pattern, a 3D image containing spatial-depth information can be constructed. Achieving super-resolution to resolve finer spatial details is of great interest because the spatial resolution of a LIDAR system is typically limited by the size of the spot on the target. In this study, we consider the problem of resolving range maps to resolutions smaller than the size of the spot using overlapping spots. This overlap provides the additional information needed to locate multiple objects within a spot using sparse source separation, thus achieving super-resolution.

A unified framework for 3rd generation lidar pulse processing based on finite rate of innovations

In this paper, we introduce a LIDAR return pulse analysis framework based on the concept of finite rate of innovations (FRI). Specifically, the proposed FRI-based model allows us to characterize the temporal return pulse envelopes captured by 3rd generation LIDAR systems in a low dimensional space. Furthermore, the extracted model parameters can often be mapped to specific physical features of the scene being captured, aiding in high-level interpretation. After describing the model formulation and extraction process, we illustrate its potential utility in two specific applications: sub-spot size ranging (super-resolution) and random impulsive scene scanning. In the course of this discussion, we also relate the FRI model to compressive sensing and sparse range-map reconstruction.