Robert J. Meijer

Flood early warning system: design, implementation and computational modules

We present a prototype of the flood early warning system (EWS) developed within the UrbanFlood FP7 project. The system monitors sensor networks installed in flood defenses (dikes, dams, embankments, etc.), detects sensor signal abnormalities, calculates dike failure probability, and simulates possible scenarios of dike breaching and flood propagation. All the relevant information and simulation results are fed into an interactive decision support system that helps dike managers and city authorities to make informed decisions in case of emergency and in routine dike quality assessment. In addition to that, a Virtual Dike computational module has been developed for advanced research into dike stability and failure mechanisms, and for training the artificial intelligence module on signal parameters induced by dike instabilities. This paper describes the UrbanFlood EWS generic design and functionality, the computational workflow, the individual modules, their integration via the Common Information Space middleware, and the first results of EWS monitoring and performance benchmarks.

Sensor Data Storage Performance: SQL or NoSQL, Physical or Virtual

Sensors are used to monitor certain aspects of the physical or virtual world and databases are typically used to store the data that these sensors provide. The use of sensors is increasing, which leads to an increasing demand on sensor data storage platforms. Some sensor monitoring applications need to automatically add new databases as the size of the sensor network increases. Cloud computing and virtualization are key technologies to enable these applications. A key issue therefore becomes the performance of virtualized databases and how this relates to physical ones. Traditional SQL databases have been used for a long time and have proven to be reliable tools for all kinds of applications. NoSQL databases have gained momentum in the last couple of years however, because of growing scalability and availability requirements. This paper compares three databases on their relative performance with regards to sensor data storage: one open source SQL database(PostgreSQL) and two open source NoSQL databases (Cassandra and MongoDB). A comparison is also made between running these databases on a physical server and running them on a virtual machine. A minimal sensor data structure is used and tested using four operations: a single write, a single read, multiple writes in one statement and multiple reads in one statement.

Time-frequency methods for structural health monitoring.

Detection of early warning signals for the imminent failure of large and complex engineered structures is a daunting challenge with many open research questions. In this paper we report on novel ways to perform Structural Health Monitoring (SHM) of flood protection systems (levees, earthen dikes and concrete dams) using sensor data. We present a robust data-driven anomaly detection method that combines time-frequency feature extraction, using wavelet analysis and phase shift, with one-sided classification techniques to identify the onset of failure anomalies in real-time sensor measurements. The methodology has been successfully tested at three operational levees. We detected a dam leakage in the retaining dam (Germany) and “strange” behaviour of sensors installed in a Boston levee (UK) and a Rhine levee (Germany).

User Programmable Virtualized Networks

This paper introduces the concept of a User Programmable Virtualized Network, which allows networks to deliver application specific services using network element components that developers can program as part of a users application. The use of special tokens in data or control packets is the basis of a practical, yet powerful security and AAA framework. This framework allows for implementations with a low footprint that can operate in a multi domain network operator environment. We demonstrate the ease with which one can build applications and address networking problems as they appear for example in sensor networks.

Dynamically Scaling Apache Storm for the Analysis of Streaming Data

Stream processing platforms allow applications to analyse incoming data continuously. Several use cases exist that make use of these capabilities, ranging from monitoring of physical infrastructures to pre selecting video surveillance feeds for human inspection. It is difficult to predict how much computing resources are needed for these stream processing platforms, because the volume and velocity of input data may vary over time. The open source Apache Storm software provides a framework for developers to build processing applications that use the computing resources of all machines within an established cluster. Because of the varying processing needs of such applications, the platform should be able to automatically grow and shrink as needed. Unfortunately, the current Storm platform does not provide this capability. In this paper we describe the design and implementation of a tool that monitors several aspects of the Storm platform, the applications running on top of it, and external systems such as queues and databases. Based on this information, the tool decides whether extra servers are needed or machines may be decommissioned from the cluster, and then acts upon it by actually creating new virtual machines or shutting them down.

Artificial intelligence and finite element modelling for monitoring flood defence structures

We present a hybrid approach to monitoring the stability of flood defence structures equipped with sensors. This approach combines the finite element modelling with the artificial intelligence for real-time signal processing and anomaly detection. This combined method has been developed for the UrbanFlood early warning system and successfully tested on a large-scale sea dike during a simulated strong storm with very high water level. The artificial intelligence module detects the onset of dike instability after being trained on the data from the Virtual Dike finite element simulation.

Toward executable scientific publications

Reproducibility of experiments is considered as one of the main principles of the scientific method. Recent developments in data and computation intensive science, i.e. e-Science, and state of the art in Cloud computing provide the necessary components to preserve data sets and re-run code and software that create research data. The Executable Paper (EP) concept uses state of the art technology to include data sets, code, and software in the electronic publication such that readers can validate the presented results. In this paper we present how to advance current state of the art to preserve, data sets, code, and software that create research data, the basic components of an execution platform to preserve long term compatibility of EP, and we identify a number of issues and challenges in the realization of EP.

Multi-domain lightpath authorization, using tokens

This paper highlights the concepts and results of our research, leading to demonstrations during the period 2005-2007 to develop a flexible and simple access control model, and corresponding support tools to provision multi-domain optical network resources on demand. We introduce the general network resources provisioning model that extends the Generic AAA Authorisation sequences for multi-domain scenarios, and explain how token based access control and policy enforcement can be used during the provisioned resource access. To build a solid conceptual foundation for the proposed token, based access control, the paper revisits existing token definition and proposes a new definition in the context of our research. We subsequently show the use of tokens during different stages of the lightpath provisioning process. The paper identifies and describes two major scenarios in multidomain lightpath provisioning: the chain and tree approaches. The proposed token concept allows a simple combination of access control enforcement at different networking layers: the packet layer, the path layer, and the service layer. We end with a brief description of a few demonstrations that proves the proposed concepts and illustrates its acceptance by a wider networking community.

Dynamic paths in multi-domain optical networks for grids

Many Grid applications require high bandwidth end-to-end connections between Grid resources in different domains. Fiber optic networks, owned by different providers, have to cooperate in a coordinated manner in order to provide an end-to-end connection. Currently, multi-domain optical network solutions require paper-based long-term contracts between administrative domains. This paper describes a solution for dynamically creating optical connections between different autonomous domains. This was implemented in the form of a Grid Service following the Open Grid Service Architecture. In our prototype, each switch belongs to a different network domain. Our Grid Service uses a toolkit based on the Generic Authorization, Authentication, and Accounting framework. This toolkit authorizes the use of optical infrastructure elements based on specific policies that are active within each domain. To complete our multi-domain authorization architecture, a Broker Service was also implemented. Our Broker Service interacts with the Grid Service instances to provide Grid application with a simplified way to set up end-to-end connections on demand.

Token based path authorization at interconnection points between hybrid networks and a lambda grid

In order to provide cost effective transport services for highly demanding data-intensive grid applications, National Research Networks (NRNs) are considering additional types of access to their network infrastructures. Next to traditional IP access, NRNs like to provide automated, grid application driven access to their underlying connection-oriented network infrastructure. This combination is called hybrid networking. Recently, both NRNs and grid communities started to acquire their own global optical network connections. A lambda grid was created when these organizations decided to interconnect these links via interconnection points such as Starlight and Netherlight. Apart from supporting scientific applications, the lambda grid allows network research. Within this context, this paper will present a novel token-based path selection mechanism that will enable authorized access to lambda grid links. The token-based approach allows temporal separation of the path authorization process from obtaining access to a lambda grid link. The path authorization process may involve many parties and complex, time consuming decisions whereas path access requires a fast real-time implementation. We will describe the application of the token-based approach for an interconnection point between a hybrid network and a lambda grid