Medhavy Thankappan

Rapid, high-resolution detection of environmental change over continental scales from satellite data – the Earth Observation Data Cube

ABSTRACT The effort and cost required to convert satellite Earth Observation (EO) data into meaningful geophysical variables has prevented the systematic analysis of all available observations. To overcome these problems, we utilise an integrated High Performance Computing and Data environment to rapidly process, restructure and analyse the Australian Landsat data archive. In this approach, the EO data are assigned to a common grid framework that spans the full geospatial and temporal extent of the observations – the EO Data Cube. This approach is pixel-based and incorporates geometric and spectral calibration and quality assurance of each Earth surface reflectance measurement. We demonstrate the utility of the approach with rapid time-series mapping of surface water across the entire Australian continent using 27 years of continuous, 25 m resolution observations. Our preliminary analysis of the Landsat archive shows how the EO Data Cube can effectively liberate high-resolution EO data from their complex sensor-specific data structures and revolutionise our ability to measure environmental change.

Issues in the application of Digital Surface Model data to correct the terrain illumination effects in Landsat images

The accuracy of topographic correction of Landsat data based on a Digital Surface Model (DSM) depends on the quality, scale and spatial resolution of the DSM data used and the co-registration between the DSM and the satellite image. A physics-based bidirectional reflectance distribution function (BRDF) and atmospheric correction model in conjunction with a 1-second DSM was used to conduct the analysis in this paper. The results show that for the examples used from Australia, the 1-second DSM, can provide an effective product for this task. However, it was found that some remaining artefacts in the DSM data, originally due to radar shadow, can still cause significant local errors in the correction. Where they occur, false shadows and over-corrected surface reflectance factors can be observed. More generally, accurate co-registration between satellite images and DSM data was found to be critical for effective correction. Mis-registration by one or two pixels could lead to large errors of retrieved surface reflectance factors in gully and ridge areas. Using low-resolution DSM data in conjunction with high-resolution satellite images will also fail to correct significant terrain components where they occur at the finer scales of the satellite images. DSM resolution appropriate to the resolution of satellite image and the roughness of the terrain is needed for effective results, and the rougher the terrain, the more critical will be the accurate registration.

Applying machine learning methods and time series analysis to create a National Dynamic Land Cover Dataset for Australia

The National Dynamic Land Cover Dataset (DLCD) classifies Australian land cover into 34 categories, which conform to 2007 International Standards Organisation (ISO) Land Cover Standard (19144-2). The DLCD has been developed by Geoscience Australia and the Australian Bureau of Agricultural and Resource Economics and Sciences (ABARES), aiming to provide nationally consistent land cover information to federal and state governments and general public. This paper describes the modeling procedure to generate the DLCD, including machine learning methodologies and time series analysis techniques involved in the process.

The variability of satellite derived surface BRDF shape over Australia from 2001 to 2011

The intra- and inter-annual variability of surface bidirectional reflectance distribution function (BRDF) in Australia has been analyzed using 11 years (2001-2011) of MODIS BRDF data. A statistic called here Root Mean Square (RMS) was used as a BRDF shape indicator to represent the overall BRDF shape and an Australian vegetation structure map was used to separate the different BRDF shape patterns by structure. The results show that the intra-annual variation of BRDF shape is stronger than the inter-annual variation although it is not clear yet whether the variation is related more to climate patterns or to vegetation structure (height and cover) or landcover class. However, BRDF shape patterns have strong similarity with vegetation structure classes. There is strong correlation between RMS and the Normalized Difference Vegetation Index (NDVI) at annual scale within structural classes indicating good relationship between BRDF and annual changes in cover within the classes.

Corner reflectors for the Australian Geophysical Observing System and support for calibration of satellite-borne synthetic aperture radars

Geoscience Australia is implementing the geospatial component of the Australian Geophysical Observing System (AGOS). AGOS infrastructure will include a network of radar corner reflectors, in addition to a geodetic ground mark network for monitoring ground deformation in Australia. In this paper we describe the design of radar corner reflectors for deformation studies and calibration of Synthetic Aperture Radar (SAR) sensors. Through a prototyping campaign in 2013 we will seek a single reflector design that can be used for deformation and calibration studies using X- and C-band SAR sensors. Prototyping will involve precise determination of reflector radar cross section at an outdoor radar test range and a temporary field deployment in the Canberra region during which X- and C-band data acquisitions will be made from orbiting SAR satellites. We also outline plans to deploy corner reflectors in the Surat Basin of Queensland to monitor ground deformation induced by coal seam gas (CSG) extraction.

Dynamic Land Cover Dataset version 2: 2001-now…a land cover odyssey

Understanding how land cover responds to natural and anthropogenic drivers is critical as increasing population, climate fluctuations and competing land uses place increased pressure on both natural and food/fibre production systems. The Dynamic Land Cover Dataset (DLCD) Version 2 is a series of biennial land cover maps that uses the ISO 19144-2 land cover classification scheme. The Moderate Resolution Image Spectroradiometer (MODIS) Enhanced Vegetation Index (EVI) time series [1] are used to characterize greenness dynamics observed at 250-metre scale. These greenness dynamics are used to generate a series of 9 land cover maps. Version 2 of the DLCD provides a series of land cover maps updated on an annual basis to enable resource managers, decision makers and biophysical modelers to track the change in land cover on a systematic basis.

Creating multi-sensor time series using data from Landsat-5 TM and Landsat-7 ETM+ to characterise vegetation dynamics

The Landsat series of satellites provide the longest contiguous earth observation record of the Earths surface. This provides the unique capacity to track changes in vegetation over multiple decades. This paper illustrates how standardized Landsat data can be combined to create a time series of sensor independent observations. The impact of side-lap and cloud frequency on observation frequency are also examined with reference to two adjacent path/rows of data in southern Australia. The generation of Landsat scale time series provides the opportunity to track both subtle and dramatic changes in vegetation cover in much higher levels of detail than previously possible. However the approach presents new challenges associated with developing time series analysis techniques to characterize time series that have uncertain observation frequencies.

Platform for Environmental Modelling Support: a Grid Cell Data Infrastructure for Modellers

Australian Spatial Data Infrastructure (SDI) is in need of a grid cell-based data component that supports the needs of landscape or environmental process modellers and other GIS users. Spatial data infrastructure and innovation diffusion concepts based on the Organisational Innovation Process are used to establish the innovative nature of this proposed component of the SDI and the process for its acceptance by key stakeholders in Australia.

Evaluation of the TanDEM-X intermediate DEM for terrain illumination correction in Landsat data

An appropriate resolution of the Digital Elevation Model (DEM) data with sufficient quality of the gradient field is critical for effective correction of remotely sensed data over mountainous areas. Conversely, using performance of terrain illumination correction and scale-based analysis, such as filter bank analysis, the quality of DEM data can be evaluated. In this study, TanDEM-X Intermediate DEM (IDEM) data at 12 m resolution and the 1-arc second Shuttle Radar Topography Mission (SRTM) data were used independently to evaluate the relative effectiveness of the terrain illumination correction for Landsat 8 optical data over Tasmania. Results from the terrain illumination correction and filter bank analysis show that IDEM 12 m data can resolve finer details of terrain shading than the SRTM based DEM and deliver better results in areas with detail-rich terrain. However, since the data available for this study is an intermediate product, spikes and other noise artefacts were prevalent, especially over areas covered by water. Operational use of the IDEM would require the removal of such noise artefacts.

Iterating Petabyte-scale Earth observation processes in the Australian Geoscience Data Cube

This paper presents a Near-Real-Time multi-GPU accelerated solution of the ωk Algorithm for Synthetic Aperture Radar (SAR) data focusing, obtained in Stripmap SAR mode. Starting from an input raw data, the algorithm subdivides it in a grid of a configurable number of bursts along track. A multithreading CPU-side support is made available in order to handle each graphic device in parallel. Then each burst is assigned to a separate GPU and processed including Range Compression, Stolt Mapping via ChirpZ and Azimuth Compression steps. We prove the efficiency of our algorithm by using Sentinel-1 raw data (approx. 3.3 GB) on a commodity graphics card; the single-GPU solution is approximately 4x faster than the industrial multi-core CPU implementation (General ACS SAR Processor, GASP), without significant loss of quality. Using a multi-GPU system, the algorithm is approximately 6x faster with respect to the CPU processor.For decades, field help in case of disasters on the Earth’s surface - like floods, fires or earthquakes - is supported by the analysis of remotely sensed data. In recent years, the monitoring of vehicles, buildings or areas fraught with risk has become another major task for satellite-based crisis intervention. Since these scenarios are unforeseen and time-critical, they require a fast and well coordinated reaction. If useful information is extracted out of image data in realtime directly on board a spacecraft, the timespan between image acquisition and an appropriate reaction can be shortened significantly. Furthermore, on board image analysis allows data of minor interest, e.g. cloud-contaminated scenes, to be discarded and/or treated with lower priority, which leads to an optimized usage of storage and downlink capacity. This paper describes the modular application framework of VIMOS, an on board image processing experiment for remote sensing applications. Special focus will be on resource management, safety and modular commandability.Gaia is an ESA cornerstone mission, which was successfully launched December 2013 and commenced operations in July 2014. Within the Gaia Data Processing and Analysis consortium, Coordination Unit 7 (CU7) is responsible for the variability analysis of over a billion celestial sources and nearly 4 billion associated time series (photometric, spectrophotometric, and spectroscopic), encoding information in over 800 billion observations during the 5 years of the mission, resulting in a petabyte scale analytical problem. In this article, we briefly describe the solutions we developed to address the challenges of time series variability analysis: from the structure for a distributed data-oriented scientific collaboration to architectural choices and specific components used. Our approach is based on Open Source components with a distributed, partitioned database as the core to handle incrementally: ingestion, distributed processing, analysis, results and export in a constrained time window.The seamless mosaicing of massive very high resolution imagery addresses several aspects related to big data from space. Data volume is directly proportional to the size the input data, i.e., order of several TeraPixels for a continent. Data velocity derives from the fact that the input data is delivered over several years to meet maximum cloud contamination constraints with the considered satellites. Data variety results from the need to collect and integrate various ancillary data for cloud detection, land/sea mask delineation, and adaptive colour balancing. This paper details how these 3 aspects of big data are handled and illustrates them for the creation of a seamless pan-European mosaic from 2.5m imagery (Land Monitoring/Urban Atlas Copernicus CORE 03 data set).The current development of satellite imagery means that a great volume of images acquired globally has to be understood in a fast and precise manner. Processing this large quantity of information comes at the cost of finding unsupervised algorithms to fulfill these tasks. Change detection is one of the main issues when talking about the analysis of satellite image time series (SITS). In this paper, we propose a method to analyze changes in SITS based on binary descriptors and on the Hamming distance, regarded as a similarity metric. In order to render an automatic and completely unsupervised technique towards solving this problem, the obtained distances are quantized into change levels using the Lloyd-Max’s algorithm. The experiments are carried on 11 Landsat images at 30 meters spatial resolution, covering an area of approximately 59 × 51 km2 over the surroundings of Bucharest, Romania, and containing information from six subbands of frequency.The Euclid Archive System prototype is a functional information system which is used to address the numerous challenges in the development of fully functional data processing system for Euclid. The prototype must support the highly distributed nature of the Euclid Science Ground System, with Science Data Centres in at least eight countries. There are strict requirements both on data quality control and traceability of the data processing. Data volumes will be greater than 10 Pbyte, with the actual volume being dependent on the amount of reprocessing required.In the space domain, all scientific and technological developments are accompanied by a growth of the number of data sources. More specifically, the world of observation knows this very strong acceleration and the demand for information processing follows the same pace. To meet this demand, the problems associated with non-interoperability of data must be efficiently resolved upstream and without loss of information. We advocate the use of linked data technologies to integrate heterogeneous and schema-less data that we aim to publish in the 5 stars scale in order to foster their re-use. By proposing the 5 stars data model, Tim Berners-Lee drew the perfect roadmap for the production of high quality linked data. In this paper, we present a technological framework that allows to go from raw, scattered and heterogeneous data to structured data with a well-defined and agreed upon semantics, interlinked with other dataset for their common objects.Reference data sets, necessary to the advancement of the field of object recognition by providing a point of comparison for different algorithms, are prevalent in the field of multimedia. Although sharing the same basic object recognition problem, in the field of remote sensing there is a need for specialized reference data sets. This paper would like to open the topic for discussion, by taking a first attempt at creating a reference data set for a satellite image. In doing so, important differences between annotating photographic and satellite images are highlighted, along with their impact on the creation of a reference data set. The results are discussed with a view toward creating a future methodology for the manual annotation of satellite images.The future atmospheric composition Sentinel missions will generate two orders of magnitude more data than the current missions and the operational processing of these big data is a big challenge. The trace gas retrieval from remote sensing data usually requires high-performance radiative transfer model (RTM) simulations and the RTM are usually the bottleneck for the operational processing of the satellite data. To date, multi-core CPUs and also Graphical Processing Units (GPUs) have been used for highly intensive parallel computations. In this paper, we are comparing multi-core and GPU implementations of an RTM based on the discrete ordinate solution method. With GPUs, we have achieved a 20x-40x speed-up for the multi-stream RTM, and 50x speed-up for the two-stream RTM with respect to the original single-threaded CPU codes. Based on these performance tests, an optimal workload distribution scheme between GPU and CPU is proposed. Finally, we discuss the performance obtained with the multi-core-CPU and GPU implementations of the RTM.The effective use of Big Data in current and future scientific missions requires intelligent data handling systems which are able to interface the user to complicated distributed data collections. We review the WISE Concept of Scientific Information Systems and the WISE solutions for the storage and processing as applied to Big Data.Interactive visual data mining, where the user plays a key role in learning process, has gained high attention in data mining and human-machine communication. However, this approach needs Dimensionality Reduction (DR) techniques to visualize image collections. Although the main focus of DR techniques lays on preserving the structure of the data, the occlusion of images and inefficient usage of display space are their two main drawbacks. In this work, we propose to use Non-negative Matrix Factorization (NMF) to reduce the dimensionality of images for immersive visualization. The proposed method aims to preserve the structure of data and at the same time reduce the occlusion between images by defining regularization terms for NMF. Experimental validations performed on two sets of image collections show the efficiency of the proposed method in respect to controlling the trade-off between structure preserving and less occluded visualization.This article provides a short overview about the TanDEM-X mission, its objectives and the payload ground segment (PGS) based on data management, processing systems and long term archive. Due to the large data volume of the acquired and processed products a main challenge in the operation of the PGS is to handle the required data throughput, which is a new dimension for the DLR PGS. To achieve this requirement, several solutions were developed and coordinated. Some of them were more technical nature whereas others optimized the workflows.Clustering of Earth Observation (EO) images has gained a high amount of attention in remote sensing and data mining. Here, each image is represented by a high-dimensional feature vector which could be computed as the results of coding algorithms of extracted local descriptors or raw pixel values. In this work, we propose to learn the features using discriminative Nonnegative Matrix factorization (DNMF) to represent each image. Here, we use the label of some images to produce new representation of images with more discriminative property. To validate our algorithm, we apply the proposed algorithm on a dataset of Synthetic Aperture Radar (SAR) and compare the results with the results of state-of-the-art techniques for image representation. The results confirm the capability of the proposed method in learning discriminative features leading to higher accuracy in clustering.