A Semantic Interpretation Method for Deep Neural Networks Based on Knowledge Graphs

We present a physics‐informed deep neural network (DNN) method for estimating hydraulic conductivity in saturated and unsaturated flows governed by Darcy's law. For saturated flow, we approximate hydraulic conductivity and head with two DNNs and use Darcy's law in addition to measurements of hydraulic conductivity and head to train these DNNs. For unsaturated flow, we approximate unsaturated conductivity function and capillary pressure with DNNs and train these DNNs using measurements of capillary pressure and the Richards equation. Because it is difficult to measure unsaturated conductivity in the field, we assume that no measurements of unsaturated conductivity are available. The proposed approach enforces the partial differential equation (PDE) (Darcy or Richards equation) constraints by minimizing the PDE residual at select points in the simulation domain. We demonstrate that physics constraints increase the accuracy of DNN approximations of sparsely observed functions and allow for training DNNs when no direct measurements of the functions of interest are available. For the saturated conductivity estimation problem, we show that the physics‐informed DNN method is more accurate than the state‐of‐the‐art maximum a posteriori probability method. For the unsaturated flow in homogeneous porous media, we find that the proposed method can accurately estimate the pressure‐conductivity relationship based on the capillary pressure measurements only, even in the presence of measurement noise.

SUMMARY Traditional seafloor mapping relies on shipborne soundings which have limited spatial coverage. The Surface Water and Ocean Topography (SWOT) wide-swath altimetry satellite holds the potential for predicting more detailed seafloor topography. In this study, we integrate SWOT gravity data with single-beam shipborne depths to construct seafloor topography models in the Northwestern Pacific using the deep neural network (DNN) method. Compared to shipborne depth checkpoints, the root mean square (RMS) error of the differences between topography model predicted by DNN method and shipborne depths is approximately 97.5 m, improving by 19.5 per cent and 9.9 per cent compared to the gravity-geologic (GGM) method and the Smith and Sandwell (SAS) method respectively. Compared to traditional data, the integration of SWOT gravity data universally enhances prediction accuracy. Furthermore, the DNN method effectively demonstrates superior capability in balancing the characterization of overall structures with the retention of authentic topography features, which we demonstrated in the Mariana region of the NW Pacific Ocean. However, limited by spatial heterogeneity and physical mechanisms, accurate prediction of such complex, fine-scale topography using gravity data remains a significant challenge.

With the advancement of machine learning methods, audio sentiment analysis has become an active research area in recent years. For example, business organizations are interested in persuasion tactics from vocal cues and acoustic measures in speech. A typical approach is to find a set of acoustic features from audio data that can indicate or predict a customer's attitude, opinion, or emotion state. For audio signals, acoustic features have been widely used in many machine learning applications, such as music classification, language recognition, emotion recognition, and so on. For emotion recognition, previous work shows that pitch and speech rate features are important features. This thesis work focuses on determining sentiment from call center audio records, each containing a conversation between a sales representative and a customer. The sentiment of an audio record is considered positive if the conversation ended with an appointment being made, and is negative otherwise. In this project, a data processing and machine learning pipeline for this problem has been developed. It consists of three major steps: 1) an audio record is split into segments by speaker turns; 2) acoustic features are extracted from each segment; and 3) classification models are trained on the acoustic features to predict sentiment. Different set of features have been used and different machine learning methods, including classical machine learning algorithms and deep neural networks, have been implemented in the pipeline. In our deep neural network method, the feature vectors of audio segments are stacked in temporal order into a feature matrix, which is fed into deep convolution neural networks as input. Experimental results based on real data shows that acoustic features, such as Mel frequency cepstral coefficients, timbre and Chroma features, are good indicators for sentiment. Temporal information in an audio record can be captured by deep convolutional neural networks for improved prediction accuracy.

Multi-label learning based deep transfer neural network for facial attribute classification

With the expansion of power grid, the amount of knowledge in power system is exploding. In order to organize, manage and utilize massive knowledge effectively, knowledge graph technology is introduced into the field of power system. In order to further improve the application of knowledge graph technology in power monitoring system. Firstly, this paper analyzes the power system knowledge graph and its advantages in power system knowledge management. Then, the construction method of power system knowledge graph is designed, focusing on the comprehensive analysis of the relationship between defect level and risk level, and drawing heat map. Combined with the characteristics of power system knowledge graph, the typical application scenarios of knowledge graph technology in the field of power system can be intelligently expanded by applying the mature and stable graph database method in the industry and adding the deep learning Convolutional Neural Network (CNN) method innovatively. Finally, on the basis of analyzing the current research hot-spots, the key problems in the application of knowledge graph in power system and the possible research directions in the future are pointed out.

This paper proposes a new training method of deep neural networks (DNNs) for statistical parametric speech synthesis. DNNs are recently used as acoustic models that represent mapping functions from linguistic features to acoustic features in statistical parametric speech synthesis. There are problems to be solved in conventional DNN-based speech synthesis: 1) the inconsistency between the training and synthesis criteria; and 2) the over-smoothing of the generated parameter trajectories. In this paper, we introduce the parameter trajectory generation process considering the global variance (GV) into the training of DNNs. A unified framework which consistently uses the same criterion in both training and synthesis can be obtained and the model parameters are optimized for parameter generation considering the GV in the proposed method. Experimental results show that the proposed method outperforms the conventional method in the naturalness of synthesized speech.

Numerical solution of Generalized Burger–Huxley & Huxley’s equation using Deep Galerkin neural network method

Deep neural networks such as convolutional neural networks (CNNs) have achieved a great success in a broad range of fields. Spiking neural networks (SNNs) are designed as a solution for realizing ultra-low power consumption using spikebased neuromorphic hardware. To enable the mapping between conventional artificial neural networks (ANNs) and spike-based neuromorphic hardware, direct conversion from conventional ANNs into SNNs has recently been proposed. However, performance loss is hard to avoid after conversion. To reduce the loss, we analyze the encoding methods of SNNs and the optimization methods for converting. We use the rate coding and approximate the output of an activation function of CNNs by the number of spikes produced by a neuron within a given time window in SNNs. We propose a method for generating fixed uniform spike train whose number of spikes is as exact as we expect, and present an optimization method to reduce the loss by rescaling the threshold of each layer in SNNs. For evaluation, we train three different CNNs with MNIST and SVHN datasets separately and convert all of them into SNNs. The networks achieve a max accuracy of 99.17% on MNIST and of 93.36% on additional training dataset of SVHN. The results show the proposed fixed and uniform spike train not only outperforms the Poisson distributed spike train but also costs less time to generate. Our threshold rescaling method greatly improves the performance of SNNs.

Artificial Intelligence (AI) and, in particular, Machine Learning (ML), have emerged to be utilized in various applications due to their capability to learn how to solve complex problems. Over the past decade, rapid advances in ML have presented Deep Neural Networks (DNNs) consisting of a large number of neurons and layers. DNN Hardware Accelerators (DHAs) are leveraged to deploy DNNs in the target applications. Safety-critical applications, where hardware faults/errors would result in catastrophic consequences, also benefit from DHAs. Therefore, the reliability of DNNs is an essential subject of research. In recent years, several studies have been published accordingly to assess the reliability of DNNs. In this regard, various reliability assessment methods have been proposed on a variety of platforms and applications. Hence, there is a need to summarize the state-of-the-art to identify the gaps in the study of the reliability of DNNs. In this work, we conduct a Systematic Literature Review (SLR) on the reliability assessment methods of DNNs to collect relevant research works as much as possible, present a categorization of them, and address the open challenges. Through this SLR, three kinds of methods for reliability assessment of DNNs are identified, including Fault Injection (FI), Analytical, and Hybrid methods. Since the majority of works assess the DNN reliability by FI, we characterize different approaches and platforms of the FI method comprehensively. Moreover, Analytical and Hybrid methods are propounded. Thus, different reliability assessment methods for DNNs have been elaborated on their conducted DNN platforms and reliability evaluation metrics. Finally, we highlight the advantages and disadvantages of the identified methods and address the open challenges in the research area. We have concluded that Analytical and Hybrid methods are light-weight yet sufficiently accurate and have the potential to be extended in future research and to be utilized in establishing novel DNN reliability assessment frameworks.

Although enormous progress has been made by Deep Neural Networks (DNN) in basic perception tasks, they are long criticized for lack of reasoning quality and interpretability. Raven's Progressive Matrices (RPMs) are standard tests for assessing human Intelligence Quotient, also acting as a tool to evaluate Artificial Intelligence. Existing methods of pure DNN combining perception and reasoning are difficult to confirm DNN's ability of logical reasoning. Hybrid DNN methods reasoning by algorithms, not DNN. Existing methods use end-to-end training, while perception and reasoning are detached modules functionally and neurologically. Here we propose a method of separating visual perception by a perfect convolutional neural network perception module abstracting RPMs to panel-level interpretable encoding and inferring it with our Recurrent Neural Network reasoning model. With a trained perfect perception module, our model attaches great performance on the RPM dataset Impartial-RAVEN of 96.72% and RAVEN of 90.77%. Our method tests the abstract reasoning quality of DNN s and also provides a platform for further research in neuroscience.

The evaluation of the learning process is an effective way to realize personalized online learning. Real-time evaluation of learners’ cognitive level during online learning helps to monitor learners’ cognitive state and adjust learning strategies to improve the quality of online learning. However, most of the existing cognitive level evaluation methods use manual coding or traditional machine learning methods, which are time-consuming and laborious. They cannot fully mine the implicit cognitive semantic information in unstructured text data, making the cognitive level evaluation inefficient. Therefore, this study proposed the bidirectional gated recurrent convolutional neural network combined with an attention mechanism (AM-BiGRU-CNN) deep neural network cognitive level evaluation method, and based on Bloom’s taxonomy of cognition objectives, taking the unstructured interactive text data released by 9167 learners in the massive open online course (MOOC) forum as an empirical study to support the method. The study found that the AM-BiGRU-CNN method has the best evaluation effect, with the overall accuracy of the evaluation of the six cognitive levels reaching 84.21%, of which the F1-Score at the creating level is 91.77%. The experimental results show that the deep neural network method can effectively identify the cognitive features implicit in the text and can be better applied to the automatic evaluation of the cognitive level of online learners. This study provides a technical reference for the evaluation of the cognitive level of the students in the online learning environment, and automatic evaluation in the realization of personalized learning strategies, teaching intervention, and resources recommended have higher application value.

This paper describes an unsupervised adaptation method of deep neural networks (DNNs) regarding discriminative sound source localization (SSL). DNNs-based SSL and its unsupervised adaptation fail under different conditions from those during training. The estimations sometimes include incoherent unpredictable errors due to the NN's non-linearity. We propose an eliminative posterior probability constraint using a model-based SSL for unsupervised DNNs adaptation. This constraint forces the probability of “less possible candidates” to become zero to eliminate incoherent errors. The candidates are indicated by a model-based SSL method because it can estimate the azimuth of the sound source with moderate accuracy and explicit reasoning. As a result, the localization performance of adapted DNNs improved more than that of model-based SSL. Experimental results showed that our method improved localization correctness of 1D azimuth and 3D regions by a maximum of 13.3 and 5.9 points compared with the model-based SSL.

Some industrial processes take place in confined settings only observable by sensors, e.g. infrared (IR) cameras. Drying processes take place while a material is transported by means of a conveyor through a black box equipped with internal IR cameras. While such sensors deliver data at high rates, this is beyond what human operators can analyze and calls for automation. Inspired by numerous implementations monitoring techniques that analyse IR images using deep learning, this paper shows how they can be applied to the confined microwave drying of porous foams, with benchmarking their effectiveness at condition monitoring to conduct fault detection. Convolutional neural networks, derived transfer learning, and deep residual neural network methods are already regarded as cutting-edge and are studied here, using a set of conventional approaches for comparative evaluation. Our comparison shows that state-of-the-art deep learning techniques significantly benefit condition monitoring, providing an increase in fault finding accuracy of up to 48% over conventional methods. Nevertheless, we also found that derived transfer learning and deep residual network techniques do not in our case yield increased performance over normal convolutional neural networks.

Fault detection (FD) is important for health monitoring and safe operation of dynamical systems. Previous studies use model-based approaches which are sensitive to system specifics, attenuating the robustness. Data-driven methods have claimed accurate performances which scale well to different cases, but the algorithmic structures and enclosed operations are “black,” jeopardizing its robustness. To address these issues, exemplifying the FD problem of aircraft air data sensors, we explore to develop a robust (accurate, scalable, explainable, and interpretable) FD scheme using a typical data-driven method, i.e., deep neural networks (DNN). To guarantee the scalability, aircraft inertial reference unit measurements are adopted as equivalent inputs to the DNN, and a database associated with 6 different aircraft/flight conditions is constructed. Convolutional neural networks (CNN) and long-short time memory (LSTM) blocks are used in the DNN scheme for accurate FD performances. To enhance robustness of the DNN, we also develop two new concepts: “large structure” which corresponds to the parameters that can be objectively optimized (e.g., CNN kernel size) via certain metrics (e.g., accuracy) and “small structure” that conveys subjective understanding of humans (e.g., class activation mapping in CNN) within a certain context (e.g., object detection). We illustrate the optimization process we adopted in devising the DNN large structure, which yields accurate (90%) and scalable (24 diverse cases) performances. We also interpret the DNN small structure via class activation mapping, which yields promising results and solidifies the robustness of DNN. Lessons and experiences we learned are also summarized in the paper, which we believe is instructive for addressing the FD problems in other similar fields.

Raindrop size distributions (DSD) and rain rate have been estimated from polarimetric radar data using different approaches with the accuracy depending on the errors both in the radar measurements and the estimation methods. Herein, a deep neural network (DNN) technique was utilized to improve the estimation of the DSD and rain rate by mitigating these errors. The performance of this approach was evaluated using measurements from a two-dimensional video disdrometer (2DVD) at the Kessler Atmospheric and Ecological Field Station in Oklahoma as ground truth with the results compared against conventional estimation methods for the period 2006–17. Physical parameters (mass-/volume-weighted diameter and liquid water content), rain rate, and polarimetric radar variables (including radar reflectivity and differential reflectivity) were obtained from the DSD data. Three methods—physics-based inversion, empirical formula, and DNN—were applied to two different temporal domains (instantaneous and rain-event average) with three diverse error assumptions (fitting, measurement, and model errors). The DSD retrievals and rain estimates from 18 cases were evaluated by calculating the bias and root-mean-squared error (RMSE). DNN produced the best performance for most cases, with up to a 5% reduction in RMSE when model errors existed. DSD and rain estimated from a nearby polarimetric radar using the empirical and DNN methods were well correlated with the disdrometer observations; the rain-rate estimate bias of the DNN was significantly reduced (3.3% in DNN vs 50.1% in empirical). These results suggest that DNN has advantages over the physics-based and empirical methods in retrieving rain microphysics from radar observations.