Shiraj Khan

Nonlinear statistics reveals stronger ties between ENSO and the tropical hydrological cycle

Cross-spectrum analysis based on linear correlations in the time domain suggested a coupling between large river flows and the El Nino-Southern Oscillation (ENSO) cycle. A nonlinear measure based on mutual information (MI) reveals extrabasinal connections between ENSO and river flows in the tropics and subtropics, that are 20-70% higher than those suggested so far by linear correlations. The enhanced dependence observed for the Nile, Amazon, Congo, Paran{acute a}, and Ganges rivers, which affect large, densely populated regions of the world, has significant impacts on inter-annual river flow predictabilities and, hence, on water resources and agricultural planning.

Knowledge Discovery from Sensor Data For Scientific Applications

The current advances in sensors and sensor infrastructures offer new opportunities for monitoring the operations and conditions of man-made and natural environments. The ability to generate insights or new knowledge from sensor data is critical for many high-priority scientific applications especially weather, climate, and associated natural hazards. One example is sensor-based early warning systems for geophysical extremes such as tsunamis or extreme rainfall, which can help preempt disaster damage. Indeed, the loss of life during the 2004 Indian Ocean tsunami may have been significantly reduced, if not totally prevented, had sensor-based early warning systems been in place. One other example is high-resolution risk-mapping of insights obtained through a combination of historical and real-time sensor data, with physics-based computer simulations. Weather, climate and associated natural hazards have established history of using sensor data, such as data from DOPPLER radars. Recent advances in sensor technology and computational strengths have created a need for new approaches to analyzing data associated with weather, climate, and associated natural hazards. Knowledge discovery offers tools for extracting new, useful and hidden insights from data repositories. However, knowledge discovery techniques need to be geared towards scalable and efficient implementations of predictive insights, online or fast real-time analysis of incremental information, and solution processes for strategic and tactical decisions. Predictive insights regarding weather, climate and associated natural hazards may require models of rare, anomalous and extreme events, nonlinear phenomena, and change analysis, in particular from massive volumes of dynamic data streams. On the other hand, historical data may also be noisy and incomplete, thus robust tools need to be developed for these situations. This chapter describes some of the research challenges of knowledge discovery from sensor data for weather, climate and associated natural hazard applications and summarizes our approach towards addressing these challenges.

High performance statistical computing with parallel R: applications to biology and climate modelling

Ultrascale computing and high-throughput experimental technologies have enabled the production of scientific data about complex natural phenomena. With this opportunity, comes a new problem – the massive quantities of data so produced. Answers to fundamental questions about the nature of those phenomena remain largely hidden in the produced data. The goal of this work is to provide a scalable high performance statistical data analysis framework to help scientists perform interactive analyses of these raw data to extract knowledge. Towards this goal we have been developing an open source parallel statistical analysis package, called Parallel R, that lets scientists employ a wide range of statistical analysis routines on high performance shared and distributed memory architectures without having to deal with the intricacies of parallelizing these routines.

Data Mining and Data Fusion for Enhanced Decision Support

The process of data mining converts information to knowledge by using tools from the disciplines of computational statistics, database technologies, machine learning, signal processing, nonlinear dynamics, process modeling, simulation, and allied disciplines. Data mining allows business problems to be analyzed from diverse perspectives, including dimensionality reduction, correlation and co-occurrence, clustering and classification, regression and forecasting, anomaly detection, and change analysis. The predictive insights generated from data mining can be further utilized through real-time analysis and decision sciences, as well as through human-driven analysis based on management by exceptions or objectives, to generate actionable knowledge. The tools that enable the transformation of raw data to actionable predictive insights are collectively referred to as decision support tools. This chapter presents a new formalization of the decision process, leading to a new decision superiority model, partially motivated by the joint directors of laboratories (JDL) data fusion model. In addition, it examines the growing importance of data fusion concepts.