Dimitri P. Solomatine

Model Induction with Support Vector Machines: Introduction and Applications

The rapid advance in information processing systems in recent decades had directed engineering research towards the development of intelligent systems that can evolve models of natural phenomena automatically, ‘by themselves’, so to speak. In this respect, a wide range of machine learning techniques like decision trees, artificial neural networks (ANNs), Bayesian methods, fuzzy-rule based systems and evolutionary algorithms have been successfully applied to model different civil engineering systems. In this study, the possibility of using yet another machine learning paradigm which is firmly based on the theory of statistical learning, namely that of the Support Vector Machine (SVM), is investigated. An interesting property of this approach is that it is an approximate implementation of a structural risk minimisation (SRM) induction principle that aims at minimising a bound on the generalisation error of a model, rather than only minimising the mean square error over the data set. In this paper, the basic ideas underlying statistical learning theory and SVM are reviewed and the potential of the SVM for feature classification and multiple regression (modelling) problems is demonstrated by applying the method to two different cases of model induction from empirical data. The relative performance of the SVM is then analysed by comparing its results with that of ANNs on the same data sets.

Model trees as an alternative to neural networks in rainfall—runoff modelling

Abstract This paper investigates the comparative performance of two data-driven modelling techniques, namely, artificial neural networks (ANNs) and model trees (MTs), in rainfall—runoff transformation. The applicability of these techniques is studied by predicting runoff one, three and six hours ahead for a European catchment. The result shows that both ANNs and MTs produce excellent results for 1-h ahead prediction, acceptable results for 3-h ahead prediction and conditionally acceptable result for 6-h ahead prediction. Both techniques have almost similar performance for 1-h ahead prediction of runoff, but the result of the ANN is slightly better than the MT for higher lead times. However, the advantage of the MT is that the result is more understandable and allows one to build a family of models of varying complexity and accuracy.

Neural networks and M5 model trees in modelling water level-discharge relationship

Reliable estimation of discharge in a river is the crucial component of efficient flood management and surface water planning. Hydrologists use historical data to establish a relationship between water level and discharge, which is known as a rating curve. Once a relationship is established it can be used for predicting discharge from future measurements of water level only. Successful applications of machine learning in water management inspired the exploration of applicability of these approaches in modelling this complex relationship. In the present paper, models of the water level-discharge relationship are built with an artificial neural network (ANN) and an M5 model tree. The relevant inputs are selected by computing average mutual information. The predictive accuracy of this model is compared with a traditional rating curve built with the same data. It is concluded that the ANN- and M5 model tree-based models are superior in accuracy than the traditional model.

Two decades of anarchy? Emerging themes and outstanding challenges for neural network river forecasting

This paper traces two decades of neural network rainfall-runoff and streamflow modelling, collectively termed ‘river forecasting’. The field is now firmly established and the research community involved has much to offer hydrological science. First, however, it will be necessary to converge on more objective and consistent protocols for: selecting and treating inputs prior to model development; extracting physically meaningful insights from each proposed solution; and improving transparency in the benchmarking and reporting of experimental case studies. It is also clear that neural network river forecasting solutions will have limited appeal for operational purposes until confidence intervals can be attached to forecasts. Modular design, ensemble experiments, and hybridization with conventional hydrological models are yielding new tools for decision-making. The full potential for modelling complex hydrological systems, and for characterizing uncertainty, has yet to be realized. Further gains could also emerge from the provision of an agreed set of benchmark data sets and associated development of superior diagnostics for more rigorous intermodel evaluation. To achieve these goals will require a paradigm shift, such that the mass of individual isolated activities, focused on incremental technical refinement, is replaced by a more coordinated, problem-solving international research body.

River flow forecasting using artificial neural networks

Abstract River flow forecasting is required to provide basic information on a wide range of problems related to the design and operation of river systems. The availability of extended records of rainfall and other climatic data, which could be used to obtain stream flow data, initiated the practice of rainfall-runoff modelling. While conceptual or physically-based models are of importance in the understanding of hydrological processes, there are many practical situations where the main concern is with making accurate predictions at specific locations. In such situation it is preferred to implement a simple “black box” (data-driven, or machine learning) model to identify a direct mapping between the inputs and outputs without detailed consideration of the internal structure of the physical process. Artificial neural networks (ANNs) is probably the most successful machine learning technique with flexible mathematical structure which is capable of identifying complex non-linear relationships between input and output data without attempting to reach understanding as to the nature of the phenomena. In this study the applicability of ANNs for downstream flow forecasting in the Apure river basin (Venezuela) was investigated. Two types of ANN architectures, namely multi-layer perceptron network (MLP) and a radial basis function network (RBF) were implemented. The performances of these networks were compared with a conceptual rainfall-runoff model and they were found to be slightly better for this river flow-forecasting problem.

2006 Special issue: Machine learning approaches for estimation of prediction interval for the model output

A novel method for estimating prediction uncertainty using machine learning techniques is presented. Uncertainty is expressed in the form of the two quantiles (constituting the prediction interval) of the underlying distribution of prediction errors. The idea is to partition the input space into different zones or clusters having similar model errors using fuzzy c-means clustering. The prediction interval is constructed for each cluster on the basis of empirical distributions of the errors associated with all instances belonging to the cluster under consideration and propagated from each cluster to the examples according to their membership grades in each cluster. Then a regression model is built for in-sample data using computed prediction limits as targets, and finally, this model is applied to estimate the prediction intervals (limits) for out-of-sample data. The method was tested on artificial and real hydrologic data sets using various machine learning techniques. Preliminary results show that the method is superior to other methods estimating the prediction interval. A new method for evaluating performance for estimating prediction interval is proposed as well.

A framework for uncertainty analysis in flood risk management decisions

Abstract Modern flood risk management involves responding sustainably to flood risk with portfolios of structural and non‐structural measures. Under these circumstances of multi‐attribute choice between portfolios of options, the motivation for uncertainty analysis becomes more compelling than ever. Uncertainty analysis is required in order to understand the implications for decision makers of limited data, model uncertainties, changes in the flooding system over the long term, incommensurate scales of appraisal and potentially conflicting decision objectives. In recognition of the importance of uncertainty analysis as an integral aspect of sustainable flood risk management, a new framework for uncertainty analysis within flood risk management decisions has been established. The proliferation of methods for uncertainty analysis can be placed within the coherent framework. As well as estimating the amount of uncertainty associated with key decision variables, the framework supports the decision making process by identifying the most influential sources of uncertainty, and the implications of uncertainty for the preference ordering between options. The challenges posed by severe uncertainty about the potential for long term changes in the flooding system are discussed and robustness analysis is advocated in response to these uncertainties.

Experiments with AdaBoost.RT, an improved boosting scheme for regression

The application of boosting technique to regression problems has received relatively little attention in contrast to research aimed at classification problems. This letter describes a new boosting algorithm, AdaBoost.RT, for regression problems. Its idea is in filtering out the examples with the relative estimation error that is higher than the preset threshold value, and then following the AdaBoost procedure. Thus, it requires selecting the suboptimal value of the error threshold to demarcate examples as poorly or well predicted. Some experimental results using the M5 model tree as a weak learning machine for several benchmark data sets are reported. The results are compared to other boosting methods, bagging, artificial neural networks, and a single M5 model tree. The preliminary empirical comparisons show higher performance of AdaBoost.RT for most of the considered data sets.

A novel method to estimate model uncertainty using machine learning techniques

A novel method is presented for model uncertainty estimation using machine learning techniques and its application in rainfall runoff modeling. In this method, first, the probability distribution of the model error is estimated separately for different hydrological situations and second, the parameters characterizing this distribution are aggregated and used as output target values for building the training sets for the machine learning model. This latter model, being trained, encapsulates the information about the model error localized for different hydrological conditions in the past and is used to estimate the probability distribution of the model error for the new hydrological model runs. The M5 model tree is used as a machine learning model. The method is tested to estimate uncertainty of a conceptual rainfall runoff model of the Bagmati catchment in Nepal. In this paper the method is extended further to enable it to predict an approximation of the whole error distribution, and also the new results of comparing this method to other uncertainty estimation approaches are reported. It can be concluded that the method generates consistent, interpretable and improved model uncertainty estimates.

On the encapsulation of numerical-hydraulic models in artificial neural network

The optimal control of hydraulic networks often necessitates making a considerable number of very rapid simulations of flows, such as is not practical using existing, computationally-demanding, numerical-hydraulic models. However, the site-specific knowledge and data that is encapsulated in any such numerical model can be encapsulated in its turn in an artificial neural network (ANN), and this can provide much faster simulations. In this study, a number of possible types and configurations of ANNs are investigated for their suitability to this class of application. When regarded from a hydroinformatics point of view, this study becomes one of identifying the most suitable ANN encapsulations of numerical-hydraulic encapsulations of generic hydraulic knowledge and site-specific data.