Bahaa Khalil

Statistical approaches used to assess and redesign surface water-quality-monitoring networks

An up-to-date review of the statistical approaches utilized for the assessment and redesign of surface water quality monitoring (WQM) networks is presented. The main technical aspects of network design are covered in four sections, addressing monitoring objectives, water quality variables, sampling frequency and spatial distribution of sampling locations. This paper discusses various monitoring objectives and related procedures used for the assessment and redesign of long-term surface WQM networks. The appropriateness of each approach for the design, contraction or expansion of monitoring networks is also discussed. For each statistical approach, its advantages and disadvantages are examined from a network design perspective. Possible methods to overcome disadvantages and deficiencies in the statistical approaches that are currently in use are recommended.

Bootstrap rank-ordered conditional mutual information (broCMI): A nonlinear input variable selection method for water resources modeling

The input variable selection problem has recently garnered much interest in the time series modeling community, especially within water resources applications, demonstrating that information theoretic (nonlinear)-based input variable selection algorithms such as partial mutual information (PMI) selection (PMIS) provide an improved representation of the modeled process when compared to linear alternatives such as partial correlation input selection (PCIS). PMIS is a popular algorithm for water resources modeling problems considering nonlinear input variable selection; however, this method requires the specification of two nonlinear regression models, each with parametric settings that greatly influence the selected input variables. Other attempts to develop input variable selection methods using conditional mutual information (CMI) (an analog to PMI) have been formulated under different parametric pretenses such as k nearest-neighbor (KNN) statistics or kernel density estimates (KDE). In this paper, we introduce a new input variable selection method based on CMI that uses a nonparametric multivariate continuous probability estimator based on Edgeworth approximations (EA). We improve the EA method by considering the uncertainty in the input variable selection procedure by introducing a bootstrap resampling procedure that uses rank statistics to order the selected input sets; we name our proposed method bootstrap rank-ordered CMI (broCMI). We demonstrate the superior performance of broCMI when compared to CMI-based alternatives (EA, KDE, and KNN), PMIS, and PCIS input variable selection algorithms on a set of seven synthetic test problems and a real-world urban water demand (UWD) forecasting experiment in Ottawa, Canada.

Short-term forecasting of groundwater levels under conditions of mine-tailings recharge using wavelet ensemble neural network models

Several groundwater-level forecasting studies have shown that data-driven models are simpler, faster to develop, and provide more accurate and precise results than physical or numerical-based models. Five data-driven models were examined for the forecasting of groundwater levels as a result of recharge via tailings from an abandoned mine in Quebec, Canada, for lead times of 1 day, 1 week and 1 month. The five models are: a multiple linear regression (MLR); an artificial neural network (ANN); two models that are based on de-noising the model predictors using the wavelet-transform (W-MLR, W-ANN); and a W-ensemble ANN (W-ENN) model. The tailing recharge, total precipitation, and mean air temperature were used as predictors. The ANN models performed better than the MLR models, and both MLR and ANN models performed significantly better after de-noising the predictors using wavelet-transforms. Overall, the W-ENN model performed best for each of the three lead times. These results highlight the ability of wavelet-transforms to decompose non-stationary data into discrete wavelet-components, highlighting cyclic patterns and trends in the time-series at varying temporal scales, rendering the data readily usable in forecasting. The good performance of the W-ENN model highlights the usefulness of ensemble modeling, which ensures model robustness along with improved reliability by reducing variance.RésuméPlusieurs études de prévision de niveaux d’eaux souterraines ont montré que les modèles pilotés par les données sont plus simples, plus rapides à développer et fournissent des résultats plus précis que les modèles numériques à base physique. Cinq modèles pilotés par les données ont été examinés pour la prévision des niveaux piézométriques résultant de la recharge via des terrils d’une mine abandonnée au Québec, Canada, pour des temps compris entre le jour, la semaine et le mois. Les cinq modèles sont: une régression linéaire multiple (RLM); un réseau de neurones artificiels (RNA); deux modèles qui sont basés sur le débruitage des prédicteurs de modèle utilisant la transformée des ondelettes (W-RLM, W-RNA); et un modèle d’ensemble d’ondelettes et de réseaux neuronaux artificiels (W-ERNA). La recharge via le terril, les précipitations totales, et la température moyenne de l’air sont utilisés comme prédicteurs. Les modèles RNA sont plus performants que les modèles RLM, et les deux modèles RLM et RNA fournissent des résultats significativement meilleurs après avoir ôté le bruit du signal du prédicteur en utilisant les transformées d’ondelettes. Globalement, le modèle W-ERNA obtient les meilleurs résultats pour les trois horizons temporels. Ces résultats mettent en évidence la capacité des transformées d’ondelettes pour décomposer les données non stationnaires en composantes discrètes d’ondelettes, soulignant les caractéristiques et les tendances cycliques dans les séries temporelles pour les différentes échelles de temps, rendant les données facilement utilisables pour les prévisions. La bonne performance du modèle W-ERNA souligne l’utilité de la modélisation d’ensemble, qui assure la robustesse du modèle avec une fiabilité améliorée en réduisant la variance.ResumenVarios estudios de pronósticos de niveles de agua subterráneas muestran que los modelos controlados por datos son más simples, más rápidos para desarrollar, y proporcionan resultados más precisos y exactos que los modelos de bases físicas o numéricas. Se examinaron cinco modelos controlados por datos para el pronóstico de niveles de agua subterránea como un resultado de la recarga en la escombrera de una mina abandonada en Quebec, Canadá, para tiempos de 1 día, 2 semana y 1 mes. Los cinco modelos son: una regresión lineal múltiple (MLR); una red neuronal artificial (ANN); dos modelos que están basado en modelos predictores de eliminación de ruidos usando la transformada de wavelet (W-MLR, W-ANN); y un conjunto de W con un modelo ANN (W-ENN). Se usaron la recarga de la escombrera, la precipitación total y la temperatura media del aire como predictores. Se obtuvieron mejores resultados con los modelos ANN que con los modelos MLR, y los modelos MLR y ANN mejoraron significativamente después de la eliminación de ruidos usando transformadas de wavelet. En general, el modelo W-ENN es mejor para cada uno de los tres tiempos. Estos resultados resaltan la capacidad de las transformadas de wavelet para descomponer datos no estacionarios en componentes discretas, resaltando los patrones cíclicos y las tendencias en las series de tiempo variando las escalas temporales, haciendo que los datos sean fácilmente utilizables en el pronóstico. La buena performance de modelo W-ENN resalta la utilidad del modelado conjunto, que asegura la robustez del modelo junto con una mayor fiabilidad al reducir la varianza.摘要若干项地下水位预测研究显示,依照数据处理的模型比基于物理或基于数值的模型可更简单、更快速地构建,能提供更准确的结果。检验了5个预测加拿大魁北克省废弃矿山尾矿排泄后周期为一天、一周、一个月地下水位的依据数据处理的模型。5个模型是:多元线性回归模型;人工神经网络模型;两个基于采用微波转换对模型预测因子除燥的模型;及微波总体神经网络模型。尾矿排泄、总降水量和平均气温作为预测因子。人工神经网络模型比多元线性回归模型表现要好,多远线性回归模型和人工神经网络模型经过采用微波转换对预测因子除燥后效果好很多。总之,微波总体人工神经网络模型对三个周期的每个时间段预测的最好。这些结果突出了微波转换分解非稳定数据到分立微波成分中的能力,强调了不同时间尺度下时间序列中的周期模式和趋势,使数据很容易滴用于预测。微波总体人工神经网络模型的良好表现凸显了总体模拟的有用性,这种总体模拟可通过减少差异确保模型的稳健性和改进的可靠性。ResumoVários estudos de previsão de níveis de águas subterrâneas têm mostrado que os modelos baseados em dados são mais simples, mais rápidos de desenvolver e fornecem resultados mais exatos e precisos do que os modelos físicos ou modelos baseados em métodos numéricos. Foram examinados cinco modelos baseados em dados, para a previsão de níveis de águas subterrâneas resultantes de recarga através dos rejeitos de uma mina abandonada no Quebec, Canadá, para prazos de resposta de 1 dia, 1 semana e 1 mês. Os cinco modelos são: uma regressão linear múltipla (MLR); uma rede neuronal artificial (ANN); dois modelos baseados na eliminação de ruído dos preditores do modelo utilizando a transformada de onduletas (W-MLR, W-ANN); e um modelo neuronal W-conjunto de onduletas ANN (W-ENN). Foram utilizados como preditores a recarga nos rejeitados mineiros, a precipitação total e a temperatura média do ar. Os modelos ANN tiveram melhor desempenho do que os modelos MLR, e ambos os modelos MLR e ANN tiveram um desempenho significativamente melhor após a redução de ruído dos preditores utilizando transformadas de onduletas. Em geral, o modelo W-ENN gerou melhores resultados para cada um dos três prazos de resposta. Os resultados destacam a capacidade das transformadas de onduletas de decompor dados não estacionários em componentes discretos de onduletas, destacando padrões cíclicos e tendências nas séries temporais nas diferentes escalas temporais, tornando os dados imediatamente utilizáveis na previsão. O bom desempenho do modelo W-ENN põe em evidência a utilidade da modelação conjunta, que garante a robustez do modelo a par de uma fiabilidade acrescida, através da redução da variância.

Comparison of Record-Extension Techniques for Water Quality Variables

The extension of records at monthly, weekly or daily time steps at a short-record gauge from another continuously measured gauge is termed “record extension”. Ordinary least squares regression (OLS) of the flows, or any hydrological or water quality variable, is a traditional and still common record-extension technique. However, its purpose is to generate optimal estimates of each daily (or monthly) record, rather than the population characteristics, for which the OLS tends to underestimate the variance. The line of organic correlation (LOC) was developed to correct this bias. On the other hand, the Kendall-Theil robust line (KTRL) method has been proposed as an analogue of OLS, its advantage being its robustness in the presence of extreme values. In this study, four record-extension techniques are described, and their properties are explored. These techniques are OLS, LOC, KTRL and a new technique (KTRL2), which includes the advantage of LOC in reducing the bias in estimating the variance and the advantage of KTRL in being robust in the presence of extreme values. A Monte-Carlo study is conducted to examine these four techniques for bias, standard error of moment estimates and full range of percentiles. An empirical examination is made of the preservation of historic water quality concentration characteristics using records from the Nile Delta water quality monitoring network in Egypt. The Monte-Carlo study showed that the OLS and KTRL techniques are shown to have serious deficiencies as record-extension techniques, while the LOC and KTRL2 techniques show results that are nearly similar. Using real water quality records, the KTRL2 is shown to lead to better results than the other techniques.

Comparison of OLS, ANN, KTRL, KTRL2, RLOC, and MOVE as Record-Extension Techniques for Water Quality Variables

In this study, nine record extension techniques were explored: ordinary least squares (OLS), maintenance of variance extension techniques (MOVE1, MOVE2, MOVE3, and MOVE4), Kendall–Theil robust line (KTRL), artificial neural network (ANN), and two recently developed techniques (RLOC and KTRL2). The first technique is the robust line of organic correlation (RLOC), which is a modified version of MOVE1 with the advantage of being robust in the presence of outliers and/or deviation from normality. The second technique is a modified version of the KTRL (KTRL2) that has the advantage of being able to maintain the variance in the extended records. Water quality data from the Nile Delta monitoring network in Egypt were used to conduct an empirical experiment. The nine record extension techniques were used to extend the Chloride records using Electric Conductivity as a predictor. A comparison was carried out between the nine techniques to assess their ability to provide extended records that preserve different statistical characteristics of the observed records. Results showed that the RLOC and KTRL2 are better than other techniques in preserving the characteristics of the entire distribution. However, the ANN and KTRL techniques are superior in estimating individual water quality records. The RLOC or KTRL2 techniques are recommended for extending records of discontinued water quality variables in the Nile Delta, while the ANN and KTRL techniques are recommended for the substitution of missing values. In addition, a Monte Carlo experiment was conducted to assess the impact of the presence of outliers on the performance of the MOVE techniques as well as the KTRL2 and RLOC. Results of the Monte Carlo experiment showed that, in the presence of outliers, the KTRL2 and RLOC techniques outperform the MOVE techniques.

A Novel Record-Extension Technique for Water Quality Variables Based on L-Moments

Extension of hydrological or water quality records at short-gauged stations using information from another long-gauged station is termed record extension. The ordinary least squares regression (OLS) is a traditional and commonly used record-extension technique. However, OLS is more appropriate for the substitution of scattered missing values than for record-extension as the OLS provides extended records with underestimated variance. Underestimation of the variance of the extended records leads to underestimation of high percentiles and overestimation of low percentiles given that the data is normally distributed. The Maintenance of Variance Extension techniques (MOVE) have the advantage of maintaining the variance in the extended records. However, the OLS and MOVE techniques are sensitive to the presence of outliers. Two new record-extension techniques with the advantage of being robust in the presence of outliers were recently proposed by the authors: the robust line of organic correlation (RLOC) and modified version of the Kendall-Theil Robust line (KTRL2). In this study a new robust technique is proposed. The new regression technique based on L-moments (LMOM) is a modified version of the RLOC and uses the same intercept as that of RLOC and KTRL2 while the estimated slope is based on the second L-moment. An empirical examination of the preservation of the water quality variable characteristics was carried out using water quality records from the Nile Delta water quality monitoring network in Egypt. A comparison between nine record-extension techniques (OLS, MOVE1 to MOVE4, KTRL, KTRL2, RLOC and LMOM) was performed to examine the extended records for bias and standard error in their statistical moment estimates and over the full range of percentiles. Results showed that the proposed LMOM technique outperforms other techniques by producing extended records that preserve variance as well as extreme percentiles.

Analyzing the Relationship between El-Niño Southern Oscillation and Streamflow in Ontario and Quebec Using Wavelet Transforms

Abstract. The impacts of El NiA±o Southern Oscillation (ENSO) on streamflow variability have been studied extensively in many different parts of the world. Due to the non-stationary nature of streamflow data, wavelet transform approaches are useful for analyzing these types of data. In the Canadian context, applying the wavelet transform approaches, especially those that involve the use of continuous wavelet transform (CWT) and wavelet coherence (WTC) to analyze the influence of ENSO on streamflow variability, is still rare. This present study assessed the influence of ENSO activity on streamflow in Ontario and Quebec. Monthly data from a total of 8 stations were used. Spearman correlation was used to determine the amount of time taken by streamflow to respond to ENSO activity. The wavelet spectra of streamflow CWT revealed significant periodicities at the 6-month and 12-month wavebands throughout the period covered by the data – this observation is consistent from one station to another. The CWT spectra of the SST data showed significant regions of up to 8 years, most of which occur after 1960. The influence of ENSO on streamflow as seen in the WTC spectra occur both up to and over 8 years. Areas of significant regions observed for periodicities of over 8 years mostly occur after 1960. Spearman correlation analyses indicate that the influence of ENSO is significant for all stations except for Sydenham River. Correlations between SST and reconstructed streamflow data also indicate that the influence of ENSO occurs both at low and high time scales.

Reconstruction of Information for Short-gauged Water Quality Parameters Using a Robust Version of the Line of Organic Correlation Technique

Missing data are a common problem in water quality management and operational hydrology. In such situations, filling in missing observations is warranted. However, sometimes the extension of hydrological or water quality time series at short-gauged stations is required. Records can be extended in time at short-gauged stations by exploiting the correlation between the station of interest and a nearby long-gauged station. Ordinary least squares regression (OLS) is a traditional and commonly used record extension technique. However, its purpose is to generate optimal estimates of each record rather than of the population characteristics, for which OLS tends to underestimate the variance in the extended records. This leads to underestimation of high percentiles and overestimation of low percentiles, given that the data is normally distributed. The development of the line of organic correlation (LOC) technique, also known as the maintenance of variance extension technique (MOVE), is aimed at correcting this bias. The LOC is preferable when the probability distribution of the estimates, and not just an individual estimate, is of interest. Given that water resources data in general, and water quality data in particular, are characterized by the presence of outliers, positive skewness and non-normal distribution of data, a robust record extension technique is more appropriate. In this study, three record extension techniques were investigated, and their properties were explored: OLS, LOC and a new technique proposed in this paper, the robust line of organic correlation technique (RLOC). RLOC includes the advantage of the LOC in reducing the bias in estimating the variance, but at the same time it is also robust to the presence of outliers. An empirical examination of the preservation of the characteristics of the water quality parameters was carried out using water quality records from the Nile Delta drainage system monitoring network in Egypt. Ten years of monthly records of Electric Conductivity and Chloride measured at eleven locations were used in this study. A comparison between the three record extension techniques was performed to examine the extended records for bias and standard error of the estimate of statistical moments and over the full range of percentiles. The results of this study show that the proposed RLOC technique outperforms the OLS and LOC techniques by producing extended records that preserve variability as well as high and low percentiles. As such, it is recommended that the newly proposed RLOC technique be further investigated using simulated records with specific characteristics and data sets from other geographical areas.