Tensorization of neural networks for improved privacy and interpretability

In this paper we introduce an advanced supervised training method for neural networks. It is based on Jacobian rank deficiency and it is formulated, in some sense, in the spirit of the Gauss-Newton algorithm. The Levenberg-Marquardt algorithm, as a modified Gauss-Newton, has been used successfully in solving nonlinear least squares problems including neural-network training. It outperforms (in terms of training accuracy, convergence properties, overall training time, etc.) the basic backpropagation and its variations with variable learning rate significantly, however, with higher computation and memory complexities within each iteration. The new method developed in this paper is aiming at improving convergence properties, while reducing the memory and computation complexities in supervised training of neural networks. Extensive simulation results are provided to demonstrate the superior performance of the new algorithm over the Levenberg-Marquardt algorithm.

In recent years, with the transformation of our country’s economic structure, the consumption level of residents has gradually increased, and the volume of various consumer credit businesses of commercial banks has increased sharply, which has made the bank’s risk management and control work facing huge challenges. Traditional risk assessment methods mainly rely on the market experience of credit personnel. The results of risk assessment are greatly affected by personal subjective factors. In addition, in the face of increasing data volume and business volume, traditional assessment methods are inefficient, have long business cycles, and are accurate. It is difficult to guarantee. Therefore, this article aims to study the application of electricity data in personal credit risk assessment of commercial banks. On the basis of analyzing the causes of personal credit risk, the characteristics of personal credit risk and the application of electricity data in personal credit risk assessment, a personal credit assessment model is established through the data mining method of neural network, and the model is predicted. The prediction results show that the overall prediction accuracy rate of the model on the training data set is 86%, and the prediction accuracy rate of the test data set is 78%. The neural network model has high prediction accuracy, low data requirements and good results.

Electrocardiogram (ECG) is often used together with a spectral Doppler ultrasound to separate heart cycles by determining the end-diastole locations. However, the ECG signal is not always recorded. In such cases, the cardiac cycles can be estimated manually from the ultrasound data retrospectively. We present a deep learning-based method for automatic detection of the end-diastoles in spectral Doppler spectrograms. The method uses a combination of a convolutional neural network (CNN) for extracting features and a recurrent neural network (RNN) for modeling temporal relations. In echocardiography, there are three Doppler spectrogram modalities, continuous wave, pulsed wave, and tissue velocity Doppler. Both the training and test data sets include all three modalities. The model was tested on 643 spectrograms coming from different hospitals than in the training data set. For the purposes described in this work, a valid end-diastole detection is defined as a prediction being closer than 60 ms to the reference value. We will refer to these as true detections. Similarly, a prediction farther away is defined as nonvalid or false detections. The method automatically rejects spectrograms where the detection of an end-diastole has low confidence. When setting the algorithm to reject 1.9%, the method achieved 97.7% true detections with a mean error of 14 ms and had 2.5% false detections on the remaining spectrograms.

Performance evaluation of neural networks in modeling exhaust gas aftertreatment reactors

This article has been retracted by the publisher.

Land cover and its dynamic information is the basis for characterizing surface conditions, supporting land resource management and optimization, and assessing the impacts of climate change and human activities. In land cover information extraction, the traditional convolutional neural network (CNN) method has several problems, such as the inability to be applied to multispectral and hyperspectral satellite imagery, the weak generalization ability of the model and the difficulty of automating the construction of a training database. To solve these problems, this study proposes a new type of deep convolutional neural network based on Landsat-8 Operational Land Imager (OLI) imagery. The network integrates cascaded cross-channel parametric pooling and average pooling layer, applies a hierarchical sampling strategy to realize the automatic construction of the training dataset, determines the technical scheme of model-related parameters, and finally performs the automatic classification of remote sensing images. This study used the new type of deep convolutional neural network to extract land cover information from Qinhuangdao City, Hebei Province, and compared the experimental results with those obtained by traditional methods. The results show that: (1) The proposed deep convolutional neural network (DCNN) model can automatically construct the training dataset and classify images. This model performs the classification of multispectral and hyperspectral satellite images using deep neural networks, which improves the generalization ability of the model and simplifies the application of the model. (2) The proposed DCNN model provides the best classification results in the Qinhuangdao area. The overall accuracy of the land cover data obtained is 82.0%, and the kappa coefficient is 0.76. The overall accuracy is improved by 5% and 14% compared to the support vector machine method and the maximum likelihood classification method, respectively.

Comparison of the Kriging and neural network methods for modeling foF2 maps over North China region

In this paper we use the method of MRF and neural network to solve the problem of parameters estimation in non-rigid 3D movement. Firstly, the method of MRF is used for modeling the local motion correlation of each feature point, and the 3D coordinates of each feature point are obtained. Then the method of neural network is used for clustering the feature points according to their motion situation. When the neural network reaches stabilization, we can get the motion parameters of each feature point. Finally, we correct the neighborhoods of each feature point according to motion parameters. The experimental results show that our algorithm can correctly estimate the non-rigid motion parameters.

Machine-learning algorithms are widely applied to facilitate human tasks. For instance, in seismology, they can help deliver high-resolution seismic catalogs including very small magnitude events that usually remain undetected by human analysts and by standard monitoring procedures. The new frontier of modern seismology is to exploit deep neural networks (DNN) to automatically detect P- and S-wave arrival times to obtain good-quality seismic event locations in terms of hypocentral errors. Increasing the number of events and seismic phases is essential to build complete earthquake catalogs to be used in seismological analyses (such as seismic hazard estimation, seismic tomography, fault zone structure determination, rupture mechanism study, etc.). Machine Learning methods are being integrated into large Research Infrastructures (RIs), like the European Plate Observing System (EPOS ERIC), which brings together 10 different scientific domains in Solid Earth Sciences. In this contribution, we present results from a specific Sponsored Research Activity promoted by the RI EPOS and dedicated to ML-driven methods for phase picking in seismic time series.To ensure the correct recognition of seismic waves, neural networks trained on large and representative training datasets are essential. Here, we investigate the influence of the training dataset on the DNN PhaseNet performance, applying the method to three case studies. In the first case, the DNN trained with the Italian seismicity dataset (INSTANCE) is used to build a catalogue on the Fucino basin (Central Italy) study area; in the second case, we use the AQ2009 training dataset (based on the L’Aquila 2009 aftershocks) to analyse the 2016-2017 Amatrice-Visso-Norcia seismic sequence; and finally, the CREW training dataset (that contains P- and S-waves reflected on the mantle Earth and recorded at large epicentral distance) to detect P- and S- waves of teleseismic (regional) data acquired by the Adria Array project network. The use of different training datasets greatly improves the performance of the neural network in recognizing P and S phases, reducing the number of false positives and providing more accurate and precise P and S-phase arrival times. 

Game agents are efficiently implemented through different AI techniques, such as neural network, reinforcement learning, and evolutionary search. Although there are many works for each approach, we present a critical analysis and comparison between them, suggesting a common benchmark and parameter configurations. The evolutionary strategy implements the NeuroEvolution of Augmenting Topologies algorithm, while the reinforcement learning agent leverages Q-Learning and Proximal Policy Optimization. We formulate and empirically compare this set of solutions using the Flappy Bird game as a test scenario. We also compare different representations of state and reward functions for each method. All methods were able to generate agents that can play the game, where the NEAT algorithm had the best results, reaching the goal of never losing.

Neural network analysis provides for a powerful means for non-linear regression analysis that can be applied to a wide variety of problems. This paper describes the application of neural networks to two welding areas. The general approach for the development of the back propagation neural networks is described, including the method that was used to identify the optimum neural network architecture and the best neural network. The first application considers the weld profile shape in pulsed laser aluminum welds and the prediction of the profiles as a function of weld process conditions. It is shown that the neural network predictions are reasonably accurate in most cases, in spite of the fact that the training data set was quite small. The second application considers the prediction of Ferrite Number in stainless steel welds as a function of weld composition. For this application an extensive training data set was available. The neural network that was developed was compared to conventional means for predicting Ferrite Number and it was shown that the neural network was considerably more accurate than other currently available methods.

The article presents the issue of monitoring the sociopsychological state of society in the period of epidemics by means of neural network algorithms and methods. Publications of sociologists, psychologists, and philologists, who have created a number of methods for in-depth analysis of emotions and tonality of texts in the Internet media, including cognitive and interpretive decoding, are devoted to substantiating approaches and methods for studying the content of Internet content. The purpose of the study is to substantiate the methods and computer tools for studying the socio-psychological state of society in crisis situations, in particular epidemics, based on neural network technologies based on Internet resources. The article considers methodological approaches and particular methods of their computer implementation. It is shown that for the computer analysis of the psychological state of society in the context of the epidemic, it is necessary to adapt the methodology for designing neural network technologies, as well as systems for collecting and textual analysis of the content of electronic and Internet resources. An effective approach to creating such systems is embedding, which uses a dense vector representation of tokens in a multidimensional space, the dimension of which should be selected experimentally in the process of training and testing the developed artificial neural networks (ANN). For contextual neural network analysis, a multiclass-oriented ANN with regularization layers of the form “SpatialDropout1D”can be used. The neural network architecture can be built on fully connected layers with an activation function of the “ReLU” type. The scientific and applied significance of the results of neural network analysis based on Internet resources is the possibility of obtaining classified assessments and segmentation of target information about the psychological state of society during periods of epidemics. This information can be used to effectively counter information threats to society.

In this chapter, effective learning approach of inverse kinematics using neural networks with efficient weights update ability has been presented for a serial link structure and industrial robot. Generally, in making neural networks learn a relation among multi inputs and outputs, a desired training data set prepared in advance is used. The training data set consists of multiple pairs of input and output vectors. The input layer receives each input vector for forward computation, and it is compared with the yielded vector from the output layer. The time required for the learning process of the neural networks depends on the number of total weights in the neural networks and that of the input-output pairs in the training data set.

ObjectiveConsidering the complex and nonlinear relationship between characteristic parameters of tumors (location and morphology) and lung dose‐volume parameters of patients with esophageal cancer (EPC) undergoing intensity‐modulated radiation therapy (IMRT), the model identification method of artificial neural network (ANN) was adopted to build the model to predict lung dose‐volume parameters under different characteristic parameters of tumors. The goal is to enhance the efficiency and quality of radiotherapy planning and reduce the risk of radiation‐induced lung injury (RILI) by providing pre‐planning dose predictions.Methods1). In the previous research work, a retrospective analysis was done on 103 cases of patients with EPC who were treated by radiotherapy, in which a linear regression model was employed and had proved the association between tumor morphology [including tumor relative length (L) and “tumor axial cross‐sectional area” (S)] and lung dose‐volume parameters (including V5, V10, V20 and V30 of lungs). 2). Building on earlier research, this study uses curve regression analysis to further explore the effect of tumor relative position (P) on lung dose‐volume parameters of patients with EPC undergoing IMRT. 3). A backpropagation (BP) neural network model was constructed with P, L, and S as inputs and lung dose‐volume parameters as outputs to establish their quantitative relationship. 4). Genetic algorithm (GA) was combined with BP neural network to optimize the initial weights and thresholds of BP neural network and avoid the model from falling into local optima, so as to improve the performance of BP neural network model.Results1). Quadratic regression models were applied to analyze the relationship of P with V5 and V10 of the lung, yielding adjusted R2 values of 0.177 and 0.081, respectively, with statistically significant regression coefficients (p < 0.01). The results suggest that as P increases, V5 and V10 initially increase, then decrease. Linear regression models were applied to analyze the relationship of P with V20 and V30 of the lung, yielding adjusted R2 values of 0.06 and 0.072, respectively, with significant regression coefficients (p < 0.01). The results show that both V20 and V30 decrease as P increases. 2). The finding of this study indicates that the modeling method of BP neural network can realize the prediction of lung dose‐volume parameters under different characteristic parameters of tumors. The GA‐BP model showed slight improvements in prediction accuracy (PA) and error index (EI) compared to the BP model alone. For both models, prediction accuracy is highest for V5, followed by V10 and V20, with V30 being the least accurate. Notably, both the BP and GA‐BP neural networks achieve outstanding recognition accuracy for V5, with EI and PA values of 0.121 and 88.61% for BP, and 0.113 and 89.00% for GA‐BP, respectively. 3). The re‐optimization of four test cases with under‐predicted whole‐lung V5 and V10 values resulted in substantial improvements over the original plans. Conclusion: The ANN model developed in this study can effectively predict lung dose‐volume parameters for patients with EPC undergoing IMRT. It provides pre‐planning guidance that enhances the efficiency and quality of radiotherapy planning and reducing the risk of RILI.

With the wide application of satellite communication networks, it is of great significance to evaluate the efficiency of a satellite communication network rationally. This paper proposed an operational evaluation method for satellite communication networks based on AHP-BP algorithm. First, the satellite communication network is evaluated in a rough and fast way in which Analytic Hierarchy Process (AHP) is used to label the data for Back-Propagation (BP) neural networks, and the training data set is derived. Then, the real training data set is built by combining the runtime data and the experience of experts. BP algorithm is used again to retrain the model based on the real training data set for correcting the evaluation result. The simulation result indicates that the AHP-BP algorithm is suitable for evaluating the satellite communication networks with its high accuracy.