Graeme C. Dandy

NEURAL NETWORKS FOR THE PREDICTION AND FORECASTING OF WATER RESOURCES VARIABLES: A REVIEW OF MODELLING ISSUES AND APPLICATIONS

Abstract Artificial Neural Networks (ANNs) are being used increasingly to predict and forecast water resources variables. In this paper, the steps that should be followed in the development of such models are outlined. These include the choice of performance criteria, the division and pre-processing of the available data, the determination of appropriate model inputs and network architecture, optimisation of the connection weights (training) and model validation. The options available to modellers at each of these steps are discussed and the issues that should be considered are highlighted. A review of 43 papers dealing with the use of neural network models for the prediction and forecasting of water resources variables is undertaken in terms of the modelling process adopted. In all but two of the papers reviewed, feedforward networks are used. The vast majority of these networks are trained using the backpropagation algorithm. Issues in relation to the optimal division of the available data, data pre-processing and the choice of appropriate model inputs are seldom considered. In addition, the process of choosing appropriate stopping criteria and optimising network geometry and internal network parameters is generally described poorly or carried out inadequately. All of the above factors can result in non-optimal model performance and an inability to draw meaningful comparisons between different models. Future research efforts should be directed towards the development of guidelines which assist with the development of ANN models and the choice of when ANNs should be used in preference to alternative approaches, the assessment of methods for extracting the knowledge that is contained in the connection weights of trained ANNs and the incorporation of uncertainty into ANN models.

Genetic Algorithms Compared to Other Techniques for Pipe Optimization

The genetic algorithm technique is a relatively new optimization technique. In this paper we present a methodology for optimizing pipe networks using genetic algorithms. Unknown decision variables are coded as binary strings. We investigate a three-operator genetic algorithm comprising reproduction, crossover, and mutation. Results are compared with the techniques of complete enumeration and nonlinear programming. We apply the optimization techniques to a case study pipe network. The genetic algorithm technique finds the global optimum in relatively few evaluations compared to the size of the search space. INTRODUCTION The construction and maintenance of pipelines for water supply costs many millions of dollars every year. As funds for the development of new infrastructure become increasingly scarce, there is an increasing desire to achieve the highest level of effectiveness for each dollar spent. Traditionally, the design of water distribution networks has been based on experience. However, there is now a significant (and growing) body of literature devoted to optimization of pipe networks. Much of the research to date has applied deterministic optimization techniques (including linear programming, dynamic programming, and nonlinear programming) to the problems of network design. A new and developing field involves the application of stochastic optimization techniques (such as genetic algorithms and simulated annealing) to large combinatorial problems. This paper applies genetic algorithms to the problem of designing pipe networks and compares its performance with the techniques of complete enumeration and nonlinear programming. PIPE NETWORK OPTIMIZATION PROBLEM In its simplest form, the problem of pipe network design for gravity systems is usually formulated in the following way. For a given layout of pipes and specified demands at the nodes, find the combination of pipe sizes that gives the minimum cost, subject to the following constraints: 1. Continuity of flow must be maintained at all junctions or nodes in the network. 2. The head loss in each pipe is a known function of the flow in the pipe, its diameter, length, and hydraulic properties. 3. The total head loss around a loop must equal zero or the head loss along a path between two reservoirs must equal the elevation difference. 4. Minimum and maximum pressure head limitations must be satisfied at certain nodes in the network. 5. Minimum and maximum diameter constraints may apply to certain pipes in the network. In addition, there may be existing pipes in the system with known diameters. One may usually assume steady state flow conditions in the network, although more than one loading condition may need to be considered. Extensions of the problem allow for valves, pumps, and storage tanks to be sized or selected. Goulter (1987) suggested that the minimum cost design for a given layout and single loading case is a branched network (i.e., a network with no loops). In practice, loops are an essential feature of actual distribution systems as they provide an alternative flow path if there is pipe failure or for maintenance. One can achieve a degree of redundancy in pipe network optimization by ensuring that the layout has appropriate loops and by specifying minimum diameters for all pipes. DETERMINISTIC SOLUTION TECHNIQUES A large literature exists on the optimization of pipe networks. Lansey and Mays (1989b) provide a comprehensive review of the published literature up to 1988. The following review will concentrate on the more recent papers. The traditional method for designing pipe networks is by trial and error guided by experience. In design of pipe networks, designers often make use of commercial simulation packages such as KYPIPE (Wood 1980), WATSYS [or WATERMAX in the United States (Olde 1985)] or WATER (Fowler 1990). A common technique is to ensure for each pipe in the system that the slope of the hydraulic grade line lies within reasonable bounds. Monbaliu et al. (1990) have proposed a type of gradient search technique to achieve an efficient design. Initially, they set all pipes at their minimum diameters and a simulation package was used to determine the pressures at all nodes in the network. If the minimum pressure constraints were not satisfied, the pipe with the maximum head loss per unit length was increased to the next available size and a further simulation was carried out. They repeated this process until all pressure constraints were satisfied. They obtained nearoptimal solutions in two test cases. Enumeration Complete enumeration is one approach for the optimization of pipe networks. The technique simulates every possible combination of discrete pipe sizes. One selects the cheapest cost network that satisfies the pressure constraints. The main drawback of this technique is the amount of computer time involved. For example, a relatively small system with eight pipes and eight possible sizes for each has 16,777,216 possible solutions. Gessler (1985) has proposed the use of selective enumeration of a severely pruned search space to optimize the design of a pipe network. One has to base the pruning of the search space on experience. Unfortunately, the global optimum may be eliminated in the process of pruning. Loubser and Gessler (1990) suggested guidelines for pruning the search space to reduce the amount of computational effort involved in enumeration; these included: (1) Grouping sets of pipes and assuming that a single diameter will be used for each group; (2) progressively storing the lowest cost solution which satisfies the constraints and eliminating all other solutions of higher cost; and (3) checking on combinations that violate the constraints. One eliminates all combinations that include the same or smaller pipe sizes. The use of guidelines 2 and 3 removes the need to check for hydraulic feasibility of particular networks since this is computationally demanding. Despite these aids, one requires a considerable amount of computer time for large networks and there is no guarantee that the optimal solution will remain in the pruned search space after applying these heuristics. Linear Programming A number of researchers have used linear programming to optimize a design of a pipe network. Researchers have developed two principal approaches (Alperovits and Shamir 1977; Quindry et al. 1979). These are reviewed in Lansey and Mays (1989b). Nonlinear Programming One can apply a number of nonlinear optimization packages to the network design problem. They include MINOS (Murtagh and Saunders 1987), GINO (Liebman et al. 1986), and GAMS (Brooke et al. 1988). All these packages use a constrained generalized reduced gradient technique to identify a local optimum for the network problem. Constraints can be included explicitly in the model. Examples include the continuity equations, head losses around loops or between reservoirs, minimum and maximum pressure limitations, and minimum and maximum diameters. Costs can be expressed as any nonlinear function of pipe diameter and length. The limitations of the technique are as follows: (1) Because the pipe diameters are continuous variables the optimal values will not necessarily conform to the available pipe sizes; thus, rounding of the final solution is required; (2) only a local optimum is obtained; and (3) there is a limitation on the number of constraints and hence the size of network that can be handled. Researchers have reported a number of applications of nonlinear optimization to pipe network problems (EI-Bahrawy and Smith 1985, 1987; Su et al. 1987; Lansey and Mays 1989a; Lansey et al. 1989; Duan et al. 1990). EI-Bahrawy and Smith (1985) applied MINOS to the design of water collection and distribution systems. Their model included a preprocessor to set up the data files and a postprocessor to round off the pipe sizes to commercial diameters. The model for distribution systems included pumps, check valves, and pressure-reducing valves. They obtained the optimal solution to a 33pipe network in a reasonable amount of computer time. E1-Bahrawy and Smith (1987) applied the aforementioned optimization model to a number of case studies. They demonstrated its ability to: (1) Handle pumps and valves; (2) to find the optimal location of booster pumps and their optimal lifts; and (3) to address the optimal layout problem. Su et al. (1987) used nonlinear programming to optimize looped pipe networks. In addition they included reliability constraints. They based the optimization model on the generalized reduced gradient (GRG) technique. A steady-state simulation model [KYPIPE, Wood (1980)] was used at each iteration to calculate pressure heads throughout the system. A separate model was used to compute the reliability of both system and nodes. They defined reliability as the probability of the design pressure being maintained at appropriate nodes in the system, given the possibility of some pipes being unavailable because of breakage. The model cannot include other elements such as pumps, valves, and storage tanks. The inclusion of constraints on reliability usually produced looped networks. Lansey et al. (1989) considered the optimal design of pipe networks where there is uncertainty in the nodal demands, Hazen-Williams coefficients and the minimum nodal heads. They used a chanceconstrained approach to convert the probabilistic constraints into deterministic ones. The constraints included the probability of the system being able to satisfy the specified nodal demands and heads. The GRG technique identified the optimum pipe sizes. The method tended to produce branched pipe networks. Lansey and Mays (1989a) used nonlinear programming to find the optimum design and layout of pipe networks. Their model was able to simulate pumps, tanks, and multiple loading cases. They embedded a simulation package [KYPIPE, Wood (1980)] in the model to ensure that the continuity and head loss constraints were satisfied. A GRG

An Improved Genetic Algorithm for Pipe Network Optimization

An improved genetic algorithm (GA) formulation for pipe network optimization has been developed. The new GA uses variable power scaling of the fitness function. The exponent introduced into the fitness function is increased in magnitude as the GA computer run proceeds. In addition to the more commonly used bitwise mutation operator, an adjacency or creeping mutation operator is introduced. Finally, Gray codes rather than binary codes are used to represent the set of decision variables which make up i the pipe network design. Results are presented comparing the performance of the traditional or simple GA formulation and the improved GA formulation for the New York City tunnels problem. The case study results indicate the improved GA performs significantly better than the simple GA. In addition, the improved GA performs better than previously used traditional optimization methods such as linear, dynamic, and nonlinear programming methods and an enumerative search method. The improved GA found a solution for the New York tunriels problem which is the lowest-cost feasible discrete size solution yet presented in the literature.

The Use of Artificial Neural Networks for the Prediction of Water Quality Parameters

This paper presents the use of artificial neural networks (ANNs) as a viable means of forecasting water quality parameters. A review of ANNs is given, and a case study is presented in which ANN methods are used to forecast salinity in the River Murray at Murray Bridge (South Australia) 14 days in advance. It is estimated that high salinity levels in the Murray cause

Review: Methods used for the development of neural networks for the prediction of water resource variables in river systems: Current status and future directions

US 22 million damage per year to water users in Adelaide. Previous studies have shown that the average salinity of the water supplied to Adelaide could be reduced by about 10% if pumping from the Murray were to be scheduled in an optimal manner. This requires forecasts of salinity several weeks in advance. The results obtained were most promising. The average absolute percentage errors of the independent 14-day forecasts for four different years of data varied from 5.3% to 7.0%. The average absolute percentage error obtained as part of a real-time forecasting simulation for 1991 was 6.5%.

The effect of internal parameters and geometry on the performance of back-propagation neural networks: an empirical study

Over the past 15 years, artificial neural networks (ANNs) have been used increasingly for prediction and forecasting in water resources and environmental engineering. However, despite this high level of research activity, methods for developing ANN models are not yet well established. In this paper, the steps in the development of ANN models are outlined and taxonomies of approaches are introduced for each of these steps. In order to obtain a snapshot of current practice, ANN development methods are assessed based on these taxonomies for 210 journal papers that were published from 1999 to 2007 and focus on the prediction of water resource variables in river systems. The results obtained indicate that the vast majority of studies focus on flow prediction, with very few applications to water quality. Methods used for determining model inputs, appropriate data subsets and the best model structure are generally obtained in an ad-hoc fashion and require further attention. Although multilayer perceptrons are still the most popular model architecture, other model architectures are also used extensively. In relation to model calibration, gradient based methods are used almost exclusively. In conclusion, despite a significant amount of research activity on the use of ANNs for prediction and forecasting of water resources variables in river systems, little of this is focused on methodological issues. Consequently, there is still a need for the development of robust ANN model development approaches.

Non-linear variable selection for artificial neural networks using partial mutual information

Abstract Artificial neural networks of the back-propagation type are being used increasingly for modelling environmental systems. One of the most difficult, and least understood, tasks in the design of back-propagation networks is the choice of adequate internal network parameters and appropriate network geometries. Although some guidance is available for the choice of these values, they are generally determined using a trial and error approach. This paper describes the effect of geometry and internal parameters on network performance for a particular case study. Although the information obtained from the tests carried out in this research is specific to the problem considered, it provides users of back-propagation networks with a valuable guide on the behaviour of networks under a wide range of operating conditions. The results obtained indicate that learning rate, momentum, the gain of the transfer function, epoch size and network geometry have a significant impact on training speed, but not on generalisation ability. The type of transfer and error function used was found to have a significant impact on learning speed as well as generalisation ability.

Estimating residential water demand in the presence of free allowances

Artificial neural networks (ANNs) have been widely used to model environmental processes. The ability of ANN models to accurately represent the complex, non-linear behaviour of relatively poorly understood processes makes them highly suited to this task. However, the selection of an appropriate set of input variables during ANN development is important for obtaining high-quality models. This can be a difficult task when considering that many input variable selection (IVS) techniques fail to perform adequately due to an underlying assumption of linearity, or due to redundancy within the available data. This paper focuses on a recently proposed IVS algorithm, based on estimation of partial mutual information (PMI), which can overcome both of these issues and is considered highly suited to the development of ANN models. In particular, this paper addresses the computational efficiency and accuracy of the algorithm via the formulation and evaluation of alternative techniques for determining the significance of PMI values estimated during selection. Furthermore, this paper presents a rigorous assessment of the PMI-based algorithm and clearly demonstrates the superior performance of this non-linear IVS technique in comparison to linear correlation-based techniques.

Use of artificial neural networks for modelling cyanobacteria Anabaena spp. in the River Murray, South Australia

The main aim of this paper is to quantify the impact of a rate structure which includes a free allowance on residential water consumption. The analysis here differs from most previous research in that it explicitly recognizes that water consumption above the free allowance, being sensitive to price, responds less to social and climatic factors than consumption below the free allowance. While the model specification and estimation procedures used here should be of interest to those interested in the demand for water, the results (using data for Adelaide, South Australia) on the determinants of water consumption should be of interest to water authorities.

Neural network based modelling of environmental variables: A systematic approach

The use of artificial neural networks (ANNs) for modelling the incidence of cyanobacteria in rivers was investigated by forecasting the occurrence of a species group of Anabaena in the River Murray at Morgan, Australia. The networks of backpropagation type were trained on 7 years of weekly data for eight variables, and their ability to provide a 4-week forecast was evaluated for a 28-week period. They were relatively successful in providing a good forecast of both the incidence and magnitude of a growth peak of the cyanobacteria within the limits required for water quality monitoring. The use of lagged versus unlagged inputs was evaluated in the implementation and performance of the networks. Lagged inputs proved far superior in providing a fit to the actual data. Sensitivity analysis of input variables was performed to evaluate their relative significance in determining the forecast values. The analysis indicated that for this data set for the River Murray, flow and temperature were the predominant variables in determining the onset and duration of cyanobacterial growth. Water colour was the next most important variable in determining the magnitude of the population growth peak. Plant nutrients nitrogen, phosphorus and iron, and turbidity were less important for this time period.