Howard Schultz

Retrieval of short ocean wave slope using polarimetric imaging

We present a passive optical remote sensing technique for recovering shape information about a water surface, in the form of a two-dimensional slope map. The method, known as polarimetric slope sensing (PSS), uses the relationship between surface orientation and the change in polarization of reflected light to infer the instantaneous two-dimensional slope across the field-of-view of an imaging polarimeter. For unpolarized skylight, the polarization orientation and degree of linear polarization of the reflected skylight provide sufficient information to determine the local surface slope vectors. A controlled laboratory experiment was carried out in a wave tank with mechanically generated gravity waves. A second study was performed from a pier on the Hudson River, near Lamont-Doherty Earth Observatory. We demonstrated that the two-dimensional slope field of short gravity waves could be recovered accurately without interfering with the fluid dynamics of the air or water, and water surface features appear remarkably realistic. The combined field and laboratory results demonstrate that the polarimetric camera gives a robust characterization of the fine-scale surface wave features that are intrinsic to wind-driven air‐sea interaction processes.

Retrieving shape information from multiple images of a specular surface

In many remote sensing and machine vision applications, the shape of a specular surface such as water, glass, or polished metal must be determined instantaneously and under natural lighting conditions. Most image analysis techniques, however, assume surface reflectance properties or lighting conditions that are incompatible with these situations. To retrieve the shape of smooth specular surfaces, a technique known as specular surface stereo was developed. The method analyzes multiple images of a surface and finds a surface shape that results in a set of synthetic images that match the observed ones. An image synthesis model is used to predict image irradiance values as a function of the shape and reflectance properties of the surface, camera geometry, and radiance distribution of the illumination. The specular surface stereo technique was tested by processing four numerical simulations-a water surface illuminated by a low- and high-contrast extended light source, and a mirrored surface illuminated by a low- and high-contrast extended light source. Under these controlled circumstances, the recovered surface shape showed good agreement with the known input. >

An overview of sea state conditions and air-sea fluxes during RaDyO

[1] Refining radiative-transfer modeling capabilities for light transmission through the sea surface requires a more detailed prescription of the sea surface roughness beyond the probability density function of the sea surface slope field. To meet this need, exciting new measurement methodologies now provide the opportunity to enhance present knowledge of sea surface roughness, especially at the microscale. In this context, two intensive field experiments using R/P Floating Instrument Platform were staged within the Office of Naval Research’s Radiance in a Dynamic Ocean (RaDyO) field program in the Santa Barbara Channel and in the central Pacific Ocean south of Hawaii. As part of this program, our team gathered and analyzed a comprehensive suite of sea surface roughness measurements designed to provide optimal coverage of fundamental optical distortion processes associated with the air-sea interface. This contribution describes the ensemble of instrumentation deployed. It provides a detailed documentation of the ambient environmental conditions that prevailed during the RaDyO field experiments. It also highlights exciting new sea surface roughness measurement capabilities that underpin a number of the scientific advances resulting from the RaDyO program. For instance, a new polarimetric imaging camera highlights the complex interplay of wind and surface currents in shaping the roughness of the sea surface that suggests the traditional Cox-Munk framework is not sufficient. In addition, the breaking crest length spectral density derived from visible and infrared imagery is shown to be modulated by the development of the wavefield (wave age) and alignment of wind and surface currents at the intermediate (dominant) scale of wave breaking.

An efficient method for geo-referenced video mosaicing for environmental monitoring

Environmental monitoring applications require seamless registration of optical data into large area mosaics that are geographically referenced to the world frame. Using frame-by-frame image registration alone, we can obtain seamless mosaics, but it will not exhibit geographical accuracy due to frame-to-frame error accumulation. On the other hand, the 3D geo-data from GPS, a laser profiler, an INS system provides a globally correct track of the motion without error propagation. However, the inherent (absolute) errors in the instrumentation are large for seamless mosaicing. The paper describes an effective two-track method for combining two different sources of data to achieve a seamless and geo-referenced mosaic, without 3D reconstruction or complex global registration. Experiments with real airborne video images show that the proposed algorithms are practical in important environmental applications.

Automatic In Situ Identification of Plankton

Earths oceans are a soup of living micro-organisms known as plankton. As the foundation of the food chain for marine life, plankton are also an integral component of the global carbon cycle which regulates the planets temperature. In this paper, we present a technique for automatic identification of plankton using a variety of features and classification methods including ensembles. The images were obtained in situ by an instrument known as the flow cytometer and microscope (FlowCAM), that detects particles from a stream of water siphoned directly from the ocean. The images are of necessity of limited resolution, making their identification a rather difficult challenge. We expect that upon completion, our system will become a useful tool for marine biologists to assess the health of the worlds oceans.

Mosaic generation for under vehicle inspection

As the threat of terrorism becomes more prevalent, the need for technological innovations for tighter security rises. Some of the areas where security needs to be increased are at border customs stations, airports, federal buildings, embassies, sporting events, and the like where checkpoints are being set up to inspect vehicles coming onto the premises. The proposed vehicle inspection system uses an at-ray of cameras and mosaicing techniques to generate different perspective views that provide a 3D representation of a vehicles undercarriage. These mosaics allow a complete inspection of the undercarriage using a graphical user interface without requiring the use of inspection mirrors or the vehicle to drive over a manned inspection bay. The system is designed for simplicity so that operators will not require long training periods.

Terrain reconstruction from widely separated images

When a terrain elevation map is computed from widely separated images the perspective distortion may result in a large number of false matches and poor reconstruction accuracy. This paper describes three image matching algorithms designed specifically to process images taken with large base-to-height ratios. They include a new match score, a subpixel interpolation scheme, and a multi-resolution unwarping technique. The algorithms are incorporated into a stereo analysis package and the system is tested by processing a single pair of high altitude images with a base-to-height ratio of 0.63 and a sequence of simulated images with base-to-height ratios that varied between 0.25 and 2.25. Analysis of the simulated data show that when these techniques are implemented the reconstruction accuracy remains independent of the base-to-height ratio.

Error detection and DEM fusion using self-consistency

The ability to efficiently and robustly recover accurate 3D terrain models from sets of stereoscopic images is important to many civilian and military applications. Our long-term goal is to develop an automatic, multi-image 3D reconstruction algorithm that can be applied to these domains. To develop an effective and practical terrain modeling system, methods must be found for detecting unreliable elevations in digital elevation maps (DEMs), and for fusing several DEMs from multiple sources into an accurate and reliable result. This paper focuses on two key factors for generating robust 3D terrain models, (1) the ability to detect unreliable elevations estimates, and (2) to fuse the reliable elevations into a single optimal terrain model. The techniques discussed in this paper are based on the concept of using self-consistency to identify potentially unreliable points. We apply the self-consistency methodology to both the two-image and multi-image scenarios. We demonstrate that the recently developed concept of self-consistency can be effectively employed to determine the reliability of values in a DEM. Estimates with a reliability below an error threshold can be excluded from further processing. We test the effectiveness of the methodology, as well as the relationship between error rate and scene geometry by processing both real and photo-realistic simulations.

Automatic Extraction of Buildings and Terrain from Aerial Images

A system has been developed to acquire, extend and refine 3D geometric site models from aerial imagery. The system hypothesizes potential building roofs in an image, automatically locates supporting geometric evidence in other images, and determines the precise shape and position of the new buildings via multi-image triangulation. Model-to-image registration techniques are applied to align new images with the site model, and model extension and refinement procedures are performed to acquire previously unseen buildings and improve the geometric accuracy of the existing 3D models. A correlation-based terrain recovery algorithm provides complementary information about the site, in the form of a digital elevation map.

Polarized light field under dynamic ocean surfaces: Numerical modeling compared with measurements

[1] As part of the Radiance in a Dynamic Ocean (RaDyO) program, we have developed a numerical model for efficiently simulating the polarized light field under highly dynamic ocean surfaces. Combining the advantages of the three‐dimensional Monte Carlo and matrix operator methods, this hybrid model has proven to be computationally effective for simulations involving a dynamic air‐sea interface. Given water optical properties and ocean surface wave slopes obtained from RaDyO field measurements, model‐simulated radiance and polarization fields under a dynamic surface are found to be qualitatively comparable to their counterparts from field measurements and should be quantitatively comparable if the light field measurement and the wave slope/water optical property measurements are appropriately collocated and synchronized. This model serves as a bridge to connect field measurements of water optical properties, wave slopes and polarized light fields. It can also be used as a powerful yet convenient tool to predict the temporal underwater polarized radiance in a real‐world situation. When appropriate surface measurements are available, model simulation is shown to reveal more dynamic features in the underwater light field than direct measurements.