Three-dimensional convolutional neural network for leak detection and localization in smart water distribution systems

The integration of small-scale Photovoltaics (PV) systems (such as rooftop PVs) decreases the visibility of power systems, since the real demand load is masked. Most rooftop systems are behind the metre and cannot be measured by household smart meters. To overcome the challenges mentioned above, this paper proposes an online solar energy disaggregation system to decouple the solar energy generated by rooftop PV systems and the ground truth demand load from net measurements. A 1D Convolutional Neural Network (CNN) Bidirectional Long Short-Term Memory (BiLSTM) deep learning method is used as the core algorithm of the proposed system. The system takes a wide range of online information (Advanced Metering Infrastructure (AMI) data, meteorological data, satellite-driven irradiance, and temporal information) as inputs to evaluate PV generation, and the system also enables online and offline modes. The effectiveness of the proposed algorithm is evaluated by comparing it to baselines. The results show that the proposed method achieves good performance under different penetration rates and different feeder levels. Finally, a transfer learning process is introduced to verify that the proposed system has good robustness and can be applied to other feeders.

Over large areas, land cover classification has conventionally been undertaken using satellite time series. Typically temporal metric percentiles derived from single pixel location time series have been used to take advantage of spectral differences among land cover classes over time and to minimize the impact of missing observations. Deep convolutional neural networks (CNNs) have demonstrated potential for land cover classification of single date images. However, over large areas and using time series their application is complicated because they are sensitive to missing observations and they may misclassify small and spatially fragmented surface features due to their spatial patch-based implementation. This study demonstrates, for the first time, a one-dimensional (1D) CNN single pixel time series land classification approach that uses temporal percentile metrics and that does not have these issues. This is demonstrated for all the Conterminous United States (CONUS) considering two different 1D CNN structures with 5 and 8 layers, respectively. CONUS 30 m land cover classifications were derived using all the available Landsat-5 and -7 imagery over a seven-month growing season in 2011 with 3.3 million 30 m land cover class labelled samples extracted from the contemporaneous CONUS National Land Cover Database (NLCD) 16 class land cover product. The 1D CNNs and, a conventional random forest model, were trained using 10%, 50% and 90% samples, and the classification accuracies were evaluated with an independent 10% proportion. Temporal metrics were classified using 5, 7 and 9 percentiles for each of five Landsat reflective wavelength bands and their eight band ratios. The CONUS and detailed 150 × 150 km classification results demonstrate that the approach is effective at scale and locally. The 1D CNN classification land cover class boundaries were preserved for small axis dimension features, such as roads and rivers, with no stripes or anomalous spatial patterns. The 8-layer 1D CNN provided the highest overall classification accuracies and both the 5-layer and 8-layer 1D CNN architectures provided higher accuracies than the random forest by 1.9% - 2.8% which as all the accuracies were > 83% is a meaningful increase. The CONUS overall classification accuracies increased marginally with the number of percentiles (86.21%, 86.40%, and 86.43% for 5, 7 and 9 percentiles, respectively) using the 8-layer 1D-CNN. Class specific producer and user accuracies were quantified, with lower accuracies for the developed land, crop and pasture/hay classes, but no systematic pattern among classes with respect to the number of temporal percentiles used. Application of the trained model to a different year of CONUS Landsat ARD showed moderately decreased accuracy (80.79% for 7 percentiles) that we illustrate is likely due to different intra-annual surface variations between years. These encouraging results are discussed with recommended research for deep learning using temporal metric percentiles.

Recently, three dimensional (3D) convolutional neural networks (CNNs) have emerged as dominant methods to capture spatiotemporal representations in videos, by adding to pre-existing 2D CNNs a third, temporal dimension. Such 3D CNNs, however, are anti-causal (i.e., they exploit information from both the past and the future frames to produce feature representations, thus preventing their use in online settings), constrain the temporal reasoning horizon to the size of the temporal convolution kernel, and are not temporal resolution-preserving for video sequence-to-sequence modelling, as, for instance, in action detection. To address these serious limitations, here we present a new 3D CNN architecture for the causal/online processing of videos. Namely, we propose a novel Recurrent Convolutional Network (RCN), which relies on recurrence to capture the temporal context across frames at each network level. Our network decomposes 3D convolutions into (1) a 2D spatial convolution component, and (2) an additional hidden state 1 × 1 convolution, applied across time. The hidden state at any time t is assumed to depend on the hidden state at t - 1 and on the current output of the spatial convolution component. As a result, the proposed network: (i) produces causal outputs, (ii) provides flexible temporal reasoning, (iii) preserves temporal resolution. Our experiments on the largescale large Kinetics and MultiThumos datasets show that the proposed method performs comparably to anti-causal 3D CNNs, while being causal and using fewer parameters.

When using convolutional neural networks (CNNs) for segmentation of organs and lesions in medical images, the conventional approach is to work with inputs and outputs either as single slice [two-dimensional (2D)] or whole volumes [three-dimensional (3D)]. One common alternative, in this study denoted as pseudo-3D, is to use a stack of adjacent slices as input and produce a prediction for at least the central slice. This approach gives the network the possibility to capture 3D spatial information, with only a minor additional computational cost. In this study, we systematically evaluate the segmentation performance and computational costs of this pseudo-3D approach as a function of the number of input slices, and compare the results to conventional end-to-end 2D and 3D CNNs, and to triplanar orthogonal 2D CNNs. The standard pseudo-3D method regards the neighboring slices as multiple input image channels. We additionally design and evaluate a novel, simple approach where the input stack is a volumetric input that is repeatably convolved in 3D to obtain a 2D feature map. This 2D map is in turn fed into a standard 2D network. We conducted experiments using two different CNN backbone architectures and on eight diverse data sets covering different anatomical regions, imaging modalities, and segmentation tasks. We found that while both pseudo-3D methods can process a large number of slices at once and still be computationally much more efficient than fully 3D CNNs, a significant improvement over a regular 2D CNN was only observed with two of the eight data sets. triplanar networks had the poorest performance of all the evaluated models. An analysis of the structural properties of the segmentation masks revealed no relations to the segmentation performance with respect to the number of input slices. A post hoc rank sum test which combined all metrics and data sets yielded that only our newly proposed pseudo-3D method with an input size of 13 slices outperformed almost all methods. In the general case, multislice inputs appear not to improve segmentation results over using 2D or 3D CNNs. For the particular case of 13 input slices, the proposed novel pseudo-3D method does appear to have a slight advantage across all data sets compared to all other methods evaluated in this work.

In this study, we present deep learning‐based approaches to automatic segmentation and applicator reconstruction with high accuracy and efficiency in the planning computed tomography (CT) for cervical cancer brachytherapy (BT). A novel three‐dimensional (3D) convolutional neural network (CNN) architecture was proposed and referred to as DSD‐UNET. The dataset of 91 patients received CT‐based BT of cervical cancer was used to train and test DSD‐UNET model for auto‐segmentation of high‐risk clinical target volume (HR‐CTV) and organs at risk (OARs). Automatic applicator reconstruction was achieved with DSD‐UNET‐based segmentation of applicator components followed by 3D skeletonization and polynomial curve fitting. Digitization of the channel paths for tandem and ovoid applicator in the planning CT was evaluated utilizing the data from 32 patients. Dice similarity coefficient (DSC), Jaccard Index (JI), and Hausdorff distance (HD) were used to quantitatively evaluate the accuracy. The segmentation performance of DSD‐UNET was compared with that of 3D U‐Net. Results showed that DSD‐UNET method outperformed 3D U‐Net on segmentations of all the structures. The mean DSC values of DSD‐UNET method were 86.9%, 82.9%, and 82.1% for bladder, HR‐CTV, and rectum, respectively. For the performance of automatic applicator reconstruction, outstanding segmentation accuracy was first achieved for the intrauterine and ovoid tubes (average DSC value of 92.1%, average HD value of 2.3 mm). Finally, HDs between the channel paths determined automatically and manually were 0.88 ± 0.12 mm, 0.95 ± 0.16 mm, and 0.96 ± 0.15 mm for the intrauterine, left ovoid, and right ovoid tubes, respectively. The proposed DSD‐UNET method outperformed the 3D U‐Net and could segment HR‐CTV, bladder, and rectum with relatively good accuracy. Accurate digitization of the channel paths could be achieved with the DSD‐UNET‐based method. The proposed approaches could be useful to improve the efficiency and consistency of treatment planning for cervical cancer BT.

Cardiac magnetic resonance (CMR) images play a growing role in diagnostic imaging of cardiovascular diseases. MRI is arguably the most comprehensive imaging modality for noninvasive and nonionizing imaging of the heart and great vessels and, hence, most suited for population imaging cohorts. Ensuring full coverage of the left ventricle (LV) is a basic criterion of CMR image quality. Complete LV coverage, from base to apex, precedes accurate cardiac volume and functional assessment. Incomplete coverage of the LV is identified through visual inspection, which is time-consuming and usually done retrospectively in large imaging cohorts. In this chapter, we propose a novel automatic method to check the coverage of LV from CMR images by using Fisher discriminative and dataset invariance (FDDI) three-dimensional (3D) convolutional neural networks (CNN) independently of image-acquisition parameters, such as imaging device, magnetic field strength, variations in protocol execution, etc. The proposed model is trained on multiple cohorts of different provenance to learn the appearance and identify missing basal and apical slices. To address this, a two-stage framework is proposed. First, the FDDI 3D CNN extracts high-level features in the common representation from different CMR datasets using adversarial approach; then these image features are used to detect missing basal and apical slices. Compared with the traditional 3D CNN strategy, the proposed FDDI 3D CNN can minimize the within-class scatter and maximize the between-class scatter, which can be adapted to other CMR image data for LV coverage assessment.

Recently, several convolutional neural networks have been proposed not only for 2D images, but also for 3D and 4D volume segmentation. Nevertheless, due to the large data size of the latter, acquiring a sufficient amount of training annotations is much more strenuous than in 2D images. For 4D time-series tomograms, this is usually handled by segmenting the constituent tomograms independently through time with 3D convolutional neural networks. Inter-volume information is therefore not utilized, potentially leading to temporal incoherence. In this paper, we attempt to resolve this by proposing two hidden Markov model variants that refine 4D segmentation labels made by 3D convolutional neural networks working on each time point. Our models utilize not only inter-volume information, but also the prediction confidence generated by the 3D segmentation convolutional neural networks themselves. To the best of our knowledge, this is the first attempt to refine 4D segmentations made by 3D convolutional neural networks using hidden Markov models. During experiments we test our models, qualitatively, quantitatively and behaviourally, using prespecified segmentations. We demonstrate in the domain of time series tomograms which are typically undersampled to allow more frequent capture; a particularly challenging problem. Finally, our dataset and code is publicly available.

BackgroundRecently, deep learning via convolutional neural networks (CNNs) has largely superseded conventional methods for proton (1H)‐MRI lung segmentation. However, previous deep learning studies have utilized single‐center data and limited acquisition parameters.PurposeDevelop a generalizable CNN for lung segmentation in 1H‐MRI, robust to pathology, acquisition protocol, vendor, and center.Study typeRetrospective.PopulationA total of 809 1H‐MRI scans from 258 participants with various pulmonary pathologies (median age (range): 57 (6–85); 42% females) and 31 healthy participants (median age (range): 34 (23–76); 34% females) that were split into training (593 scans (74%); 157 participants (55%)), testing (50 scans (6%); 50 participants (17%)) and external validation (164 scans (20%); 82 participants (28%)) sets.Field Strength/Sequence1.5‐T and 3‐T/3D spoiled‐gradient recalled and ultrashort echo‐time 1H‐MRI.Assessment2D and 3D CNNs, trained on single‐center, multi‐sequence data, and the conventional spatial fuzzy c‐means (SFCM) method were compared to manually delineated expert segmentations. Each method was validated on external data originating from several centers. Dice similarity coefficient (DSC), average boundary Hausdorff distance (Average HD), and relative error (XOR) metrics to assess segmentation performance.Statistical TestsKruskal–Wallis tests assessed significances of differences between acquisitions in the testing set. Friedman tests with post hoc multiple comparisons assessed differences between the 2D CNN, 3D CNN, and SFCM. Bland–Altman analyses assessed agreement with manually derived lung volumes. A P value of <0.05 was considered statistically significant.ResultsThe 3D CNN significantly outperformed its 2D analog and SFCM, yielding a median (range) DSC of 0.961 (0.880–0.987), Average HD of 1.63 mm (0.65–5.45) and XOR of 0.079 (0.025–0.240) on the testing set and a DSC of 0.973 (0.866–0.987), Average HD of 1.11 mm (0.47–8.13) and XOR of 0.054 (0.026–0.255) on external validation data.Data ConclusionThe 3D CNN generated accurate 1H‐MRI lung segmentations on a heterogenous dataset, demonstrating robustness to disease pathology, sequence, vendor, and center.Evidence Level4.Technical EfficacyStage 1.

The past several years have seen both an explosion in the use of Convolutional Neural Networks (CNNs) and accelerators to make CNN inference practical. In the architecture community, the lion share of effort has targeted CNN inference for image recognition. The closely related problem of video recognition has received far less attention as an accelerator target. This is surprising, as video recognition is more computationally intensive than image recognition, and video traffic is predicted to be the majority of internet traffic in the coming years. This paper fills the gap between algorithmic and hardware advances for video recognition by providing a design space exploration and flexible architecture for accelerating 3D Convolutional Neural Networks (3D CNNs)—the core kernel in modern video understanding. When compared to (2D) CNNs used for image recognition, efficiently accelerating 3D CNNs poses a significant engineering challenge due to their large (and variable over time) memory footprint and higher dimensionality. To address these challenges, we design a novel accelerator called Morph, that can adaptively support different spatial and temporal tiling strategies depending on the needs of each layer of each target 3D CNN. We codesign a software infrastructure alongside the Morph hardware to find good-fit parameters to control the hardware. Evaluated on state-of-the-art 3D CNNs, Morph achieves up to 2.7× (1.9× average) reduction in energy consumption and improves performance/watt up to 4.4× (3× average) compared to a baseline 3D CNN accelerator, with an area overhead of 2%. Morph further achieves a 11.6× average energy reduction on 3D CNNs when compared to Eyeriss, a popular 2D CNN accelerator, while reducing efficiency compared to Eyeriss on a 2D CNN by 71%.

Non-small cell lung carcinoma (NSCLC) is the most common type of pulmonary cancer, one of the deadliest malignant tumors worldwide. Given the increased emphasis on the precise management of lung cancer, identifying various subtypes of NSCLC has become pivotal for enhancing diagnostic standards and patient prognosis. In response to the challenges presented by traditional clinical diagnostic methods for NSCLC pathology subtypes, which are invasive, rely on physician experience, and consume medical resources, we explore the potential of radiomics and deep learning to automatically and non-invasively identify NSCLC subtypes from computed tomography (CT) images. An integrated model is proposed that investigates both radiomic features and deep learning features and makes comprehensive decisions based on the combination of these two features. To extract deep features, a three-dimensional convolutional neural network (3D CNN) is proposed to fully utilize the 3D nature of CT images while radiomic features are extracted by radiomics. These two types of features are combined and classified with multi-head attention (MHA) in our proposed model. To our knowledge, this is the first work that integrates different learning methods and features from varied sources in histological subtype classification of lung cancer. Experiments are organized on a mixed dataset comprising NSCLC Radiomics and Radiogenomics. The results show that our proposed model achieves 0.88 in accuracy and 0.89 in the area under the receiver operating characteristic curve (AUC) when distinguishing lung adenocarcinoma (ADC) and lung squamous cell carcinoma (SqCC), indicating the potential of being a non-invasive way for predicting histological subtypes of lung cancer.

The convolutional neural network (CNN) is a potent and popular neural network types and has been crucial to deep learning in recent years. A standard CNN which is known as 2-dimensions CNN was first proposed to solve image classification problems at first. With advancements in science and technology, as well as the growth of the internet, tasks involving video analysis have received increased attention in recent years. As a result, it was suggested that a 3-dimensions convolutional neural network (3D CNN) be applied to tackle 3D image processing or video analysis. 3D CNN is primarily used in the field of human activity recognition, video classification, medical imaging detection, and so on. This paper offers a succinct introduction to the general architecture and representative models of 3D CNN, contrasts the variations between 2D CNN and 3D CNN, explores the widely used models derived from 3D CNN, and displays the application outcomes of 3D CNN.

Objective: The objective of this study is to develop a model for locating leaks in small-scale water distribution networks, based on the integration of a hydraulic simulator with multicriteria analysis. Theoretical Framework: The water supply system for human consumption is defined as an installation consisting of a set of civil works, materials and equipment, from the collection to the building connections, intended for the production and collective supply of drinking water, through the distribution network. However, real losses, also known as physical losses, denote the entire volume of water made available in the distribution that does not reach the end consumer, requiring a tool to assist professionals in locating these leaks. Method: The methodology adopted for this research consisted of a literature review and the development of the Leak Location Model (LLM) that was built for the hypothetical network and for the discretized hypothetical network. The PSO algorithm was adopted for optimization. Later, in the model validation stage, for all simulated cases containing one, two or three leaks, the observed sections were compared with the three best alternatives indicated by the LLM to calculate the functions f1 and f2, and subsequently the objective function (fo). Results and Discussion: The results obtained revealed that the application of the multicriteria method as a tool for locating leaks in small water distribution networks. It was found that the evaluation of the model satisfactorily described the decision-making problem (average error less than 30%) when it comes to the occurrence of a leak in the network. Research Implications: The leak location model (LLM) is promising for the sector, encompassing the detection of possible leaks in the distribution network that reflect environmental effects. LLM is especially relevant given the lack of resources to assist both regulatory bodies and companies responsible for water distribution. Originality/Value: The study contributes to the area, mainly due to the lack of archived data as a previous history for comparing leak location models for small networks.

Mobile devices are becoming an important carrier for deep learning tasks, as they are being equipped with powerful, high-end mobile CPUs and GPUs. However, it is still a challenging task to execute 3D Convolutional Neural Networks (CNNs) targeting for real-time performance, besides high inference accuracy. The reason is more complex model structure and higher model dimensionality overwhelm the available computation/storage resources on mobile devices. A natural way may be turning to deep learning weight pruning techniques. However, the direct generalization of existing 2D CNN weight pruning methods to 3D CNNs is not ideal for fully exploiting mobile parallelism while achieving high inference accuracy. This paper proposes RT3D, a model compression and mobile acceleration framework for 3D CNNs, seamlessly integrating neural network weight pruning and compiler code generation techniques. We propose and investigate two structured sparsity schemes i.e., the vanilla structured sparsity and kernel group structured (KGS) sparsity that are mobile acceleration friendly. The vanilla sparsity removes whole kernel groups, while KGS sparsity is a more fine-grained structured sparsity that enjoys higher flexibility while exploiting full on-device parallelism. We propose a reweighted regularization pruning algorithm to achieve the proposed sparsity schemes. The inference time speedup due to sparsity is approaching the pruning rate of the whole model FLOPs (floating point operations). RT3D demonstrates up to 29.1x speedup in end-to-end inference time comparing with current mobile frameworks supporting 3D CNNs, with moderate 1%~1.5% accuracy loss. The end-to-end inference time for 16 video frames could be within 150 ms, when executing representative C3D and R(2+1)D models on a cellphone. For the first time, real-time execution of 3D CNNs is achieved on off-the-shelf mobiles.

An exhaustive measurement of re-sampling detection in lossy compressed images using deep learning approach

The protection of smart meters (SMs) from cyberattacks is of utmost importance because SMs in advanced metering infrastructure (AMI) are physically unprotected and produce a large amount of sensitive data. Due to scalability, the SMs are small-sized and low-cost devices having low computational capabilities. The algorithms that are designed to complete the security requirements of SMs should be lightweight. To address this issue, this paper proposes a lightweight security solution to address the man-in-the-middle attack, data tempering, and blockchain-based data provenance. Received signal strength indicator (RSSI) is used to generate link fingerprints, which are used along with pseudo-random nonce to secure AMI. The proposed algorithm detects the involvement of adversarial node or meter tempering by computing other values along with 0 and 1 as the average of consecutive RSSI and difference between the RSSI of connected static SMs. Pearson correlation coefficient ( $\rho $ ) of 0.9102 is achieved when no adversarial node is present in between the connected SMs having mobility in one or both SMs. Negative or approximately equal to zero values of $\rho $ are computed when the adversary is present in the AMI or any of the SM in the AMI is forged. For blockchain-based data provenance, all the hash values of the packet header are 100% matched with the hash functions present at the data concentrator unit (DCU), which shows no adversary’s involvement in AMI. For cases when the adversary is in the AMI, hash functions show no match with the hash values present at the DCU.