Robust night flow analysis in water distribution networks: A BiLSTM deep autoencoder approach

Many water utilities, particularly in the developing countries, continue to operate inefficient water distribution systems (WDSs) with a significant amount of water and revenue losses. Various factors, manageable to different extents, contribute to water losses, such as poor infrastructure, high pressures, illegal water use, etc. Whilst the problem of water losses in WDSs is global in scale, solutions need to be tailored to local circumstances due to the various causes of water loss and the mechanisms available to manage them. This paper investigates the potentials of the available pressure management methodologies and their implementation in developing countries, using a case study of a district metering area (DMA) in Kotež-Serbia. The minimal night flow method was applied for assessment of real losses. A particular focus is on assessment of water savings due to reduction of pressures. A total of three methods for estimation of water savings are described and tested against data measured in the DMA under initial and reduced pressures: (i) the method based on Leakage Index (LI) calculations, (ii) the PRESMAC model and (iii) a newly-developed method which is based on the assumption that both leakage and consumption are pressure dependent. The results indicate that the third method leads to the most accurate prediction of the total amount of water savings under reduced pressures, with only 6% difference between measured and estimated volume of saved water. Keywords : water supply, water losses, minimum night flow, pressure, consumption, water savings

<p>Quantification of the Water Losses (WL) components in Water Distribution Networks (WDNs) is a vital task towards their reduction. However, current WL estimation methods rely on semi-empirical approaches with high uncertainty levels, which usually lead to inaccurate estimates of the lost volume. Here, we compare the probabilistic Minimum Night Flow (MNF) estimation method introduced by Serafeim et al. (2021) to the Water Balance components analysis, introduced by the International Water Association (IWA). The strong point of the Serafeim et al. (2021) approach is that it uses statistical metrics to filter out noise effects in the flow timeseries used for MNF estimation, leading to more accurate estimation of the low flows during night hours. The effectiveness of the applied methods is tested via a large-scale, real world application to the 4 largest Pressure Management Areas (PMAs) of the WDN of the city of Patras, the third largest city in Greece (see Serafeim at al., 2022). Although methodologically different, the two approaches lead to very similar results, substantiating the robustness of the Serafeim at al. (2021) approach which allows for reliable confidence interval estimation of the observed Minimum Night Flows, making it particularly suited for engineering applications.</p><p><strong>Acknowledgements</strong></p><p>The research work was supported by the Hellenic Foundation for Research and Innovation (H.F.R.I.) under the “First Call for H.F.R.I. Research Projects to support Faculty members and Researchers and the procurement of high-cost research equipment grant” (Project Number: 1162).</p><p><strong>References</strong></p><p>Serafeim, A.V., Kokosalakis, G., Deidda, R., Karathanasi I. and Langousis A (2021) Probabilistic estimation of minimum night flow in water distribution networks: large-scale application to the city of Patras in western Greece, <em>Stoch. Environ. Res. Risk. Assess.</em>, https://doi.org/10.1007/s00477-021-02042-9</p><p>Serafeim, A.V.; Kokosalakis, G.; Deidda, R.; Karathanasi, I.; Langousis, A. (2022) Probabilistic Minimum Night Flow Estimation in Water Distribution Networks and Comparison with the Water Balance Approach: Large-Scale Application to the City Center of Patras in Western Greece, <em>Water</em>, <em><strong>14</strong></em>, 98, https://doi.org/10.3390/w14010098</p>

The residential minimum water demand characterisation is of fundamental importance for water distribution system management. During the minimum consumption, indeed, maximum pressures are on network pipes, and at the same time, tank levels rise. The water consumption analysis during the period of low demand and high pressure is thus of great interest for leakage estimation due to the increase in water loss with pressure. In order to contribute to the study of and forecast the daily minimum residential water demand, some probability distributions were tested by means of statistical inferences on a data set collected from different District Metering Areas (DMAs), showing that the stochastic minimum flow demand is defined by the Log-Normal (LN), Gumbel (Gu) and Log-Logistic (LL) distributions, as an extreme minimum value. With reference to the analysed DMAs, the parameters of such statistical distributions were estimated and the relationships are provided as a function only of the supplied users for different DMAs. The data were analysed with 1 h intervals of discretisation, with the aim of providing a useful guide to water utilities, which usually manage water distribution system data with such a resolution time. Indeed, once the minimum residential flow consumption at a 1 h interval was estimated as a function of the user number, by subtracting it to the inflow measured, it is possible to estimate the leakages rate at the DMA.

This study was conducted in the Konyaalti Water Distribution Network in Antalya, Turkey. The study area was divided into 18 district metered areas (DMAs) for better management of water losses. Water levels in reservoirs, flow rates, and water pressures were monitored on-line by the SCADA data system. A hydraulic model was calibrated and verified for each DMA using data provided by SCADA. The model results revealed that a number of DMAs exhibited high pressures, greater than 3.5 bars, and high minimum night flow (MNF) throughout the year. Also, the Infrastructure Leakage Index (ILI) for the study area was greater than 20, indicating high water losses. As a result of these findings, a pressure reducing valve (PRV) was installed at DMA No. 2 as an example and set at 3.0 bars resulting in considerable reduction in water losses. The optimum pressure level for setting the PRV was chosen using the hydraulic model. The same model was used to predict water savings due to pressure reduction. The predicted water savings were verified using long periods of flow rates and water pressure profiles. The predicted and measured water savings showed good agreement. The study concluded that hydraulic modelling is essential for applying appropriate pressure management strategies.

Reducing the inlet water pressure of the district metering area (DMA) can be achieved by fixed outlet pressure control and intelligently flow modulated pressure control respectively to reduce leakage in water distribution system. In order to weaken temperature factor influence on the benefit analysis from pressure management field study, a new methodology is proposed in this paper which is based on some theories, such as regression forecasting theory, minimum night flow (MNF)method, Bursts and Background Estimates (BABE) and Fixed and Variable Area Discharges (FAVAD) concepts. The methodology allows assessment of the reduction of water supply and the pressure reduction rate in the critical point by two pressure control techniques. Moreover, the differences of two pressure management are comprehensively analyzed, and the benefits yielded by flow modulated pressure control are more than the other.

Leakages are undesirable events in water distribution networks (WDNs) with mostly unknown incident time making it extremely difficult to accurately label leaks in distribution systems. This study proposes a one-dimensional convolutional neural network (1D CNN) deep autoencoder (AE) that is trained in a semi-supervised fashion to detect and localize leaks using multivariate timeseries data in a bid to mitigate the adverse impact of random noise. A leakage identification framework based on Pruned Exact Linear Time (PELT) multiple change point detector and alarm correlation is proposed. Additionally, a novel localization framework is proposed based on the relative percentage deviation of 1D CNN deep AE reconstruction error of pressure sensors in a district metering area (DMA). The leak localization framework outputs a sensor priority list based on the proximity of the pressure sensor to the leaky pipe thereby significantly reducing the search area for a leakage. The proposed leakage identification and localization framework is validated on a benchmark WDN, the L-TOWN. The results indicate the ability of the proposed method to identify 16 of the 19 leaks in 2019 and accurately localize 13 of the 16 identified leaks within the maximum allowable leak search radius of 300 m without recourse to a hydraulic model. Minimal identification lag was recorded for abrupt leakages whiles incipient leaks require longer identification time due to gradual leak growth rate.

As one of the most important component of Non-Revenue Water (NRW), apparent loss exists in all kinds of water distribution system but hardly can be assessed due to its complexity and invisibility. Aimed to analyze the different component of apparent loss on the basis of the characteristic of flow meters in district metered areas (DMA), an effective methodology to calculate the three primary parts of NRW is presented in this paper. In particular, the real loss assessment can be carried out by Minimum Night Flow (MNF) analysis; while the unauthorized consumption can be identified by consuming variation curve; finally, the metering error is calculated by the genetic algorithm with the integrated model based on IWA water balance audit. Application of this method in water distribution network of a steel-making enterprise improves its significance in controlling NRW of DMA.

Reducing pipe leakage is one of the top priorities for water companies, with many investing in higher quality sensor coverage to improve flow forecasting and detection of leaks. Most research on this topic is focused on leakage detection through the analysis of sensor data from district metered areas (DMAs), aiming to identify bursts after their occurrence. This study is a step towards the development of ‘self-healing’ water infrastructure systems. In particular, machine learning and deep learning-based algorithms are applied to forecasting the anomalous water flow experienced during bursts (new leakage) in DMAs at various temporal scales, thereby aiding in the health monitoring of water distribution systems. This study uses a dataset of over 2,000 DMAs in North Yorkshire, UK, containing flow time series recorded at 15-minute intervals for a period of one year. Firstly, the method of isolation forests is used to identify anomalies in the dataset, which are cross referenced with entries in the water mains repair log, indicating the occurrence of bursts. Going beyond leakage detection, this research proposes a hybrid deep learning framework named FLUIDS (Forecasting Leakage and Usual flow Intelligently in water Distribution Systems). A recurrent neural network (RNN) is used for mean flow forecasting, which is then combined with forecasted residuals obtained through real-time Kalman filtering. While providing expected day-to-day flow demands, this framework also aims to issue sufficient early warning for any upcoming anomalous flow or possible leakages. For a given forecast period, the FLUIDS framework can be used to compute the probability of flow exceeding a pre-defined threshold, thus allowing decision-making for any necessary interventions. This can inform targeted repair strategies that best utilize resources to minimize leakages and disruptions. The FLUIDS framework is statistically assessed and compared against the state-of-practice minimum night flow (MNF) methodology. Based on the statistical analyses, it is concluded that the proposed framework performs well on the unobserved test dataset for both regular and leakage water flows.

Water pressure management in a water distribution network (WDN) is a key component applied to achieve desirable water quality as well as a trouble-free operation of the network. This paper presents a hybrid, two-stage approach, to provide optimal separation of a WDN into District Metered Areas (DMAs), improving both water age and pressure. The first stage aims to divide the WDN into smaller areas via the Geometric Partitioning method, which is based on Recursive Coordinate Bisection (RCB). Subsequently, the Student’s t-mixture model (SMM) is applied to each area, providing an optimal placement of isolation valves and separating the network in DMAs. The model is evaluated on a realistic network generated through Watergems and is compared against one variation of it implemented, including the Gaussian Mixture Model (GMM) as well as the Genetic Algorithm (GA) approach, obtaining impressive performance. The implementation of both stages was deployed in a MATLAB environment through the Epanet toolkit. The proposed system is very promising, especially for large size WDNs due to the decreased running time and noteworthy reduction of pressure and water age.

Water losses in watersupply and distribution systems have been a serious problem in Turkey and allover the world causing economic costs and environmental drawbacks. Total waterlosses is the summation of real (physical) and apparent (commercial) losses. Mainactivities to reduce water losses include digitizing the components of network usingGeographical Information Systems (GIS), dividing large water distributionnetworks (WDNs) into District Metered Areas (DMAs), monitoring, carrying outyearly water balance, hydraulic modelling, pressure management, active leakagecontrol, recording water consumption of the users, reducing meter errors andillegal water consumption. The Ministry of Forestry and Water Affairs in Turkeyissued a new regulation for water losses reduction in 2014 based on the methodsused to determine and reduce the losses. This is followed by issuing practicaldescription of the technical procedure to reduce the losses in 2015. The newlegislation aims to reduce total water losses in all water authorities below 25%within certain time periods. All the water authorities are required to carryout yearly water balance and report them to the ministry together with forms tofollow up the efforts towards water losses reduction. In order to apply theprescribed techniques in a successful way, first of all water losses problemshould be understood well, technical and human resource capacity should beimproved, and sufficient financial resources should be allocated. Also, thewater utilities should have a strategic plan and a strong commitment to reducewater losses.

Leakages are a relevant issue in water distribution networks with severe effects on costs and water savings. While there are several solutions for detecting leakages by analyzing of the minimum night flow and the pressure inside manageable areas of the network (DMAs), the problem of estimating the time-instant when the leak occurred has been much less considered. However, an estimate of the leakage time-instant is useful for the diagnosis operations, as it may clarify the leak causes. We here address this problem by combining two change-point methods (CPMs) in a hierarchy: at first, a CPM analyses the minimum night flow providing an estimate of the day when the leakage started. Such an estimate is then refined by a second CPM, which analyzes the residuals between the pressure measurements and a network model in a neighborhood of the estimated leakage day. The proposed approach was tested on data from a DMA of a big European city, both on artificially injected and real leakages. Results show the feasibility of the proposed solution, also when leakages are very small.KeywordsLeakages in Water Distribution NetworksChange-Point EstimationNonstationarity Detection

Most water distribution networks (WDNs) have expanded traditionally and are based on population growth, regardless of the concept of district-metered areas (DMAs). Pressure management in existing and old WDNs has many advantages, including water leakage control, ease of operation, and WDN management in emergencies. Implementing pressure management policies, such as zoning, is a practical solution for identifying and reducing non-revenue water. This study has investigated the separate and sequential optimization effects of creating DMAs and district-pressure areas (DPAs) for pressure management in four scenarios. The proposed method has been applied to two Rural and Bellingham WDNs, and the results of each scenario have been compared by cost, reliability, and nodal pressures. The results showed that the sequential optimization consideration of DPAs and DMAs (scenario 4) caused the highest reliability and cost. The scenario increased reliability to 67 and 79% in Rural and Bellingham WDNs, respectively. Also, the results showed that creating a separate DMA (scenario 2) in the WDN has significantly lower costs than other scenarios. The results of this study, which compares pressure management scenarios, can lead to choosing the best scenario for redesigning WDNs. The proposed approach minimizes leaks and bursts in WDNs, enhancing customer satisfaction and supporting sustainable city development.

In urban water infrastructure (UWI), information and communication technology (ICT) are presently concentrated on central facilities (e.g., treatment plant) or are installed at main points in the urban drainage or water distribution network, e.g., inlet points of district meter areas (DMA) or combined sewer overflow structures. In this regard, the Internet of Things (IoT) concept as part of smart city development enables a large-scale implementation of measuring equipment and allows the integration of decentralised elements into an overall controlled system. Consequently, reliable and suitable ICT is a key element for the exchange of measurement and control data as well as for the success of these systems. From a water engineering perspective, it is often difficult to choose the right ICT for the intended UWI application. Likewise, from an IoT technology perspective, it is often unclear what kind of UWI applications are feasible, making them difficult to efficiently implement. Aim of this work is to develop a first-decision making tool, which can be used by network operators, researcher, and stakeholders to support supervisory control and data acquisition (SCADA) development and to realise an efficient ICT system in the field of network-based UWI. In contrast to existing recommendations, our approach is based on a comprehensive review of required spatial and temporal resolution of measurement and control data for a wide range of different network-based UWI applications. Subsequently, this enables a targeted coordination with the properties of communication technologies (e.g., data rate, range, and quality of service) and leads to a significantly improved and integrative decision-making tool.Subsequently, we tested the functionality of the framework on two exemplary applications in UWI, namely (1) determining suitable communication technologies for an early warning system for leakage detection and localisation in water distribution networks (e.g., (W)M-Bus for water meters, LoRaWAN for water pressure sensors, and GPRS at inlet points of DMA), and (2) identifying feasible applications for an existing LoRaWAN network (e.g., monitoring micro-climate and automatic irrigation at nature-based solutions). Results from the framework application have been evaluated through a literature review on used communication technologies, and are found to be consistent with real-word applications. As conclusion, different communication technologies are necessary to satisfy different requirements associated with an integrative approach for “smart water cities”. Selected for possible publication in a Special Issue in the Journal of Hydroinformatics.

Abstract. This paper proposes a model based detection and localisation method to deal with abnormal quality levels based on the chlorine measurements and chlorine sensitivity analysis in a water distribution network. A fault isolation algorithm which correlates on line the residuals (generated by comparing the available chlorine measurements with their estimations using a model) with the fault sensitivity matrix is used. The proposed methodology has been applied to a District Metered Area (DMA) in the Barcelona network.

This research article presents a data-driven approach for detecting bursts in water distribution networks (WDNs). The framework uses spatiotemporal information from monitoring pressure and unsupervised learning model. This approach employs three stages: (1) benchmark dataset acquisition, (2) spatiotemporal information analysis, and (3) burst detection model construction. First, the benchmark datasets were the normal dataset initially obtained by the clustering algorithm. Second, spatiotemporal information features are extracted from multimoment time windows from multiple sensors, including the distance and shape features. Third, burst detection was performed based on the isolation forest technique. A WDN is used to evaluate the performance of the method. Results show that the method can effectively detect the burst.