Fazlul Shahriar

Increasing the Accuracy of MODIS/Aqua Snow Product Using Quantitative Image Restoration Technique

The National Aeronautics and Space Administrations Moderate Resolution Imaging Spectroradiometer (MODIS)-based snow mask product critically uses 1.6 μm band 6. The snow mask algorithm for MODIS on Aqua has been adapted to use the 2.1 μm band 7, since some of Aquas MODIS detectors are nonfunctional. We have previously introduced an algorithm for quantitative image restoration (QIR) that can restore missing pixels or scan lines, using multilinear regression with input from a spatial-spectral window in other bands. In this letter, we argue that the use of MODIS Aqua band 6 data restored with the QIR technique in the snow algorithm results in a higher accuracy snow product as compared to the current MODIS Aqua snow product based on band 7 data. We show this by comparing a QIR-restored band 6 based product to the band 7 based product, applied to MODIS Terra, where we have simulated the Aqua-like damage to band 6. We demonstrate improved performance on representative granules covering different surface land-type conditions.

Impact of the Aqua MODIS Band 6 Restoration on Cloud/Snow Discrimination

AbstractDistinguishing between clouds and snow is an intrinsically challenging problem because both have similar high albedo across many bands. The 1.6-μm channel (band 6) on the Moderate Resolution Imaging Spectroradiometer (MODIS) instrument provides an essential tool for distinguishing clouds from snow, since snow typically has a much lower albedo in this band. Unfortunately, this band is severely damaged on the MODIS/Aqua platform and is typically not used in either snow or cloud products. An algorithm was previously introduced for quantitative image restoration (QIR) that can restore missing pixels of band 6 using multilinear regression with input from a spatial-spectral window in other bands. Also previously demonstrated was the effectiveness of this restoration for snow products over cloud-free pixels only. The focus of the authors’ previous work was to evaluate the impact of this restoration on the snow product, and they had relied on the current cloud mask, which does not use any information from...

A multiband statistical restoration of the Aqua MODIS 1.6 micron band

Currently, the MODIS instrument on the Aqua satellite has a number of broken detectors resulting in unreliable data for 1.6 micron band (band 6) measurements. Damaged detectors, transmission errors, and electrical failure are all vexing but seemingly unavoidable problems leading to line drop and data loss. Standard interpolation can often provide an acceptable solution if the loss is sparse. Interpolation, however, introduces a-priori assumptions about the smoothness of the data. When the loss is significant, as it is on MODIS/Aqua, interpolation creates statistically or physically implausible image values and visible artifacts. We have previously developed an algorithm to recreate the missing band 6 data from reliable data in the other 500m bands using a quantitative restoration. Our algorithm uses values in a spectral/spatial neighborhood of the pixel to be estimated, and proposes a value based on training data from the uncorrupted pixels. In this paper, we will present extensions of that algorithm that both improve the performance and robustness of the algorithm. We compare with prior work that just restores band 6 from band 7, and present statistical evidence that data from bands 3, 4, and 5 are also pertinent. We will demonstrate that the increased accuracy from our multi-band statistical estimate has significant consequences at the product level. As an example we show that the restored band 6 has potential benefit to the NASA snow mask for MODIS/Aqua when compared with using band 7 as a replacement for the damaged band 6.

Quantitative image restoration

Even with the most extensive precautions and careful planning, space based imagers will inevitably experience problems resulting in partial data corruption and possible loss. Such a loss occurs, for example, when individual image detectors are damaged. For a scanning imager this results in missing lines in the image. Images with missing lines can wreak havoc since algorithms not typically designed to handle missing pixels. Currently the metadata stores the locations of missing data, and naive spatial interpolation is used to fill it in. Naive interpolation methods can create image artifacts and even statistically or physically implausible image values. We present a general method, which uses non-linear statistical regression to estimate the values of the missing data in a principled manner. A statistically based estimate is desirable because it will preserve the statistical structure of the uncorrupted data and avoid the artifacts of naive interpolation. It also means that the restored images are suitable as input for higher-level statistical products. Previous methods replaced the missing values with those of a single closely related band, by applying a function or lookup table. We propose to use the redundant information in multiple bands to restore the lost information. The estimator we present in this paper uses values in a neighborhood of the pixel to be estimated, and propose a value based on training data from the uncorrupted pixels. Since we use the spatial variations in other channels, we avoid the blurring inherent spatial interpolation, which have implicit smoothness priors.

Improved VIIRS and MODIS SST Imagery

Moderate Resolution Imaging Spectroradiometers (MODIS) and Visible Infrared Imaging Radiometer Suite (VIIRS) radiometers, flown onboard Terra/Aqua and Suomi National Polar-orbiting Partnership (S-NPP)/Joint Polar Satellite System (JPSS) satellites, are capable of providing superior sea surface temperature (SST) imagery. However, the swath data of these multi-detector sensors are subject to several artifacts including bow-tie distortions and striping, and require special pre-processing steps. VIIRS additionally does two irreversible data reduction steps onboard: pixel aggregation (to reduce resolution changes across the swath) and pixel deletion, which complicate both bow-tie correction and destriping. While destriping was addressed elsewhere, this paper describes an algorithm, adopted in the National Oceanic and Atmospheric Administration (NOAA) Advanced Clear-Sky Processor for Oceans (ACSPO) SST system, to minimize the bow-tie artifacts in the SST imagery and facilitate application of the pattern recognition algorithms for improved separation of ocean from cloud and mapping fine SST structure, especially in the dynamic, coastal and high-latitude regions of the ocean. The algorithm is based on a computationally fast re-sampling procedure that ensures a continuity of corresponding latitude and longitude arrays. Potentially, Level 1.5 products may be generated to benefit a wide range of MODIS and VIIRS users in land, ocean, cryosphere, and atmosphere remote sensing.

Virtual green band for GOES-R

The ABI on GOES-R will provide imagery in two narrow visible bands (red, blue), which is not sufficient to directly produce color (RGB) images. In this paper we present a method to estimate green band from a simulated ABI multi-spectral image. To address this problem we propose to use statistical learning to train and update functions that estimate the value for the 550 nm green channel using the values that will be present in other bands of the ABI as input parameters. One challenge is that in order to exploit as many bands as possible, we cannot use straightforward non-parametric methods such as a look-up tables because the number of entries in look-up tables grows exponentially with the number of input parameters. Other simple approaches such as simple linear regression on the multi-spectral input parameters will not produce satisfactory results due to the underlying non-linearity of the data. For instance, the relationship among different spectra for cloud footprints will be radically different from that of a desert surface. The approach we propose is to use piecewise multi-linear regression on the multi-spectral input to train the green channel predictor. Our predictor is built from the combination of a classifier followed by a multi-linear function. The classifier assigns each pixel to a class based on the array of values from the simulated (or proxy) ABI bands at that pixel. To each class is associated a set of coefficients for a multi-linear predictor for 550 nm green channel to be predicted. Thus, the parameters of the predictor consist of parameters of the classifier, as well as coefficients defining the approximating hyperplane for each class. To determine these classifiers we will use methods based on K-means clustering, as well as multi-variable piecewise linear approximation.

Exploring pattern recognition enhancements to ACSPO clear-sky mask for VIIRS: potential and limitations

Discriminating clear-ocean from cloud in the thermal IR imagery is challenging, especially at night. Thresholds in automated cloud detection algorithms are often set conservatively leading to underestimation of the Sea Surface Temperature (SST) domain. Yet an expert user can visually distinguish the cloud patterns from SST. In this study, available pattern recognition methodologies are discussed and an automated algorithm formulated. Analyses are performed with the SSTs retrieved from the VIIRS sensor onboard S-NPP using the NOAA ACSPO system. Based on the analyses of global data, we have identified low-level spectral and spatial features potentially useful for discriminating cloud from clear-ocean. The algorithm attempts to mimic the visual perception by a human operator such as gradient information, spatial connectivity, and high/low frequency discrimination. It first identifies contiguous areas with similar features, and then makes decision based on the statistics of the whole region, rather than on a per pixel basis. Our initial objective was to automatically identify clear sky regions misclassified by ACSPO as cloud, and improve coverage of dynamic areas of the ocean and coastal zones.

JPSS VIIRS level 3 uncollated sea surface temperature product at NOAA

Abstract. Following the launch of the Suomi National Polar-orbiting Partnership satellite in October 2011 with the Visible Infrared Imager Radiometer Suite (VIIRS) sensor onboard, National Oceanic and Atmospheric Administration (NOAA) started generating a global level 2 preprocessed (L2P) sea surface temperature (SST) product. The NOAA Advanced Clear-Sky Processor for Ocean (ACSPO) L2P data are organized into 144 10-min granules per day, with a total volume of ∼27  GB. The L2P product has been successfully assimilated in several level 4 (L4) analyses. At the same time, some other users requested a gridded level 3 (L3) product with a reduced data volume. An L3U “uncollated” product (in which multiple passes over the same grid are independently saved) was produced by mapping the L2P product into equal 0.02° grids. Similar to the L2P, the L3U data are also reported in 10-min granules, but with a daily volume <1  GB. Currently, the NOAA VIIRS L3U SST product is operationally used or tested in several major international numerical weather prediction centers. The L3U shows comparable performance with L2P, suggesting that both products can be used interchangeably as input into L4 analyses. The original L2P pixel-level swath data continue to be produced and available to interested users from NOAA (NCEI) and JPL (Physical Oceanography) data archives.

Error mitigation for CCSD compressed imager data

To efficiently use the limited bandwidth available on the downlink from satellite to ground station, imager data is usually compressed before transmission. Transmission introduces unavoidable errors, which are only partially removed by forward error correction and packetization. In the case of the commonly used CCSD Rice-based compression, it results in a contiguous sequence of dummy values along scan lines in a band of the imager data. We have developed a method capable of using the image statistics to provide a principled estimate of the missing data. Our method outperforms interpolation yet can be performed fast enough to provide uninterrupted data flow. The estimation of the lost data provides significant value to end users who may use only part of the data, may not have statistical tools, or lack the expertise to mitigate the impact of the lost data. Since the locations of the lost data will be clearly marked as meta-data in the HDF or NetCDF header, experts who prefer to handle error mitigation themselves will be free to use or ignore our estimates as they see fit.

Rich client data exploration and research prototyping for NOAA

Data from satellites and model simulations is increasing exponentially as observations and model computing power improve rapidly. Not only is technology producing more data, but it often comes from sources all over the world. Researchers and scientists who must collaborate are also located globally. This work presents a software design and technologies which will make it possible for groups of researchers to explore large data sets visually together without the need to download these data sets locally. The design will also make it possible to exploit high performance computing remotely and transparently to analyze and explore large data sets. Computer power, high quality sensing, and data storage capacity have improved at a rate that outstrips our ability to develop software applications that exploit these resources. It is impractical for NOAA scientists to download all of the satellite and model data that may be relevant to a given problem and the computing environments available to a given researcher range from supercomputers to only a web browser. The size and volume of satellite and model data are increasing exponentially. There are at least 50 multisensor satellite platforms collecting Earth science data. On the ground and in the sea there are sensor networks, as well as networks of ground based radar stations, producing a rich real-time stream of data. This new wealth of data would have limited use were it not for the arrival of large-scale high-performance computation provided by parallel computers, clusters, grids, and clouds. With these computational resources and vast archives available, it is now possible to analyze subtle relationships which are global, multi-modal and cut across many data sources. Researchers, educators, and even the general public, need tools to access, discover, and use vast data center archives and high performance computing through a simple yet flexible interface.