A fuzzy neural network to SAR image classification

Online news recommendation aims to continuously select a pool of candidate articles that meet the temporal dynamics of user preferences. Most of the existing methods assume that all user-item interaction history are equally importance for recommendation, which is not alway applied in real-word scenario since the user-item interactions are sometime full of stochasticity and contingency. In addition, previous work on session-based algorithms only considers user sequence behaviors within current session without incorporating users’ historical interests or pointing out users’ main purposes within such session. In this paper, we propose a novel neural network framework, dynamic attention-integrated neural network, to tackle the problems. Specifically, we propose a dynamic neural network to model users’ dynamic interests over time in a unified framework for personalized news recommendations. News article semantic embedding, user interests modelling, session-based public behavior mining and an attention scheme that used to learn the attention score of user and item interaction within sessions are four key factors for online sequences mining and recommendation strategy. Experimental results on three real-world datasets show significant improvements over several baselines and state-of-the-art methods on session-based neural networks.

A spatial–temporal graph neural network framework for automated software bug triaging

A dynamic neural network (DNN) and a new computationally efficient functional link artificial neural network (CEFLANN) combination optimised with differential evolution (DE) is presented in this paper to predict financial time series like stock price indices and stock return volatilities of two important Indian stock markets, namely the Reliance Industries Limited (RIL), and NIFTY from one day ahead to one month in advance. The DNN comprises a set of 1st order IIR filters for processing the past inputs and their functional expansions and its weights are adjusted using a sliding mode strategy known for its fast convergence and robustness with respect to chaotic variations in the inputs. Extensive computer simulations are carried out to predict simultaneously the stock market indices and return volatilities and it is observed that the simple IIR–based DNN–FLANN model hybridised with DE produces better forecasting accuracies in comparison to the more complicated neural architectures.

This paper deals with the adaptive nonlinear identification and trajectory tracking via dynamic multilayer neural network (NN) with different timescales. Two NN identifiers are proposed for nonlinear systems identification via dynamic NNs with different timescales including both fast and slow phenomenon. The first NN identifier uses the output signals from the actual system for the system identification. In the second NN identifier, all the output signals from nonlinear system are replaced with the state variables of the NNs. The online identification algorithms for both NN identifier parameters are proposed using Lyapunov function and singularly perturbed techniques. With the identified NN models, two indirect adaptive NN controllers for the nonlinear systems containing slow and fast dynamic processes are developed. For both developed adaptive NN controllers, the trajectory errors are analyzed and the stability of the systems is proved. Simulation results show that the controller based on the second identifier has better performance than that of the first identifier.

An increasing number of applications of dynamic neural networks has been developed for digital signal processing (DSP), dynamic neural networks are feedforward neural networks with commonly used scalar synapses replaced by linear filters. This provides feedforward neural networks with the capability of performing dynamic mappings, which depend on past input values, dynamic neural networks are suitable for time series prediction, nonlinear system identification, and signal processing applications. Their most popular types are Finite Impulse Response (FIR) neural networks, which are obtained by replacing synapses with finite impulse response filters. Due to their guaranteed stability characteristic and easy to minimize error surface they have been used with great success in many applications such as signal enhancement, noise cancellation, classification of input patterns, system identification, prediction, and control.. Most of the works on system identification using neural networks are based on multilayer feedforward neural networks with backpropagation learning or more efficient variations of this algorithm, an elegant method for training layered networks. This paper is based on work in a Dynamic System Modeling (DYSMO) and as an application for speed control of DC motor drive.

Concerned with the slow learning problems of multilayer perceptrons, which utilise computationally intensive training algorithms like the backpropagation learning algorithm that can get trapped in local minima, this paper presents a novel type of higher-order recurrent neural networks, which is called the Dynamic Ridge Polynomial Neural Networks (DRPNN). The aim of the proposed network is to improve the performance of the Ridge Polynomial Neural Network by accommodating recurrent links structure. The network is tested for the prediction of non-linear and non-stationary financial signals. Three exchange rate time-series, which are the exchange rate time series between the British Pound and the Euro, the US dollar and the Euro as well as the exchange rate between the Japanese Yen and the Euro, are used in the simulation process. Simulation results showed that DRPNN generate higher profit returns with fast convergence when used to predict noisy financial time series.

The classification capability of a dynamic threshold neural network

This paper proposes a new type of neural network called the Dynamic Threshold Neural Network (DTNN) which is theoretically and experimentally superior to a conventional sigmoidal multilayer neural network in classification capability. Given a training set containing 4k + 1 patterns in Rn, to successfully learn this training set, the upper bound on the number of free parameters for a DTNN is (k + 1)(n + 2) + 2(k + 1), while the upper bound for a sigmoidal network is 2k(n + 1) + (2k + 1). We also derive a learning algorithm for the DTNN in a similar way to the derivation of the backprop learning algorithm. In simulations on learning the Two-Spirals problems, our DTNN with 30 neurons in one hidden layer takes only 3200 epochs on average to successfully learn the whole training set, while the single-hidden-layer feedforward sigmoidal neural networks have never been reported to successfully learn the given training set even though more hidden neurons are used.

One of the difficulties encountered in the application of reinforcement learning methods to real-world problem is the generalization of large-scale or continuous state space. In order to solve the curse of dimensionality problem caused by discretizing continuous state space, a kind of Q-learning method for continuous state space based on a dynamic Elman neural network was proposed in this paper. The inputs and the output of Elman network are the system state-action pair and the corresponding Q-value. That is, Elman network is used to estimate the Q-value of state-action pair on-line. Eligibility trace for connecting weights is introduced by borrowing ideas from the eligibility trace mechanism of state in temporal difference algorithm to improve the learning speed of neural network. Computer simulations on mountain car control illustrate the performance and applicability of the proposed Q-learning scheme.

The article is devoted to the problem of optimization of search request ranking algorithms in the digital information retrieval system. The algorithm of functioning of the neural network ranking unit based on Hopfield neural network is built. The ability to generate a ranked list of pages found as a result of the request in the digital information retrieval system can be provided by solving two problems of integer optimization: the problem of assignment of combinatorial sets of criteria for assessing the relevance of web page search and the problem of sorting of numbers—relevance values. The architecture of the neural network model based on the dynamic Hopfield neural network with binary output function designed for combinatorial optimization of the final list of documents found in the digital information retrieval system was synthesized. Promising variants of neural network models with binary output function of neurons for synthesis of the optimal evaluation plan with a combinatorial set of criteria by solving the problem of assignment were built. It has been proven that the built models differ in the rules for determining the coefficients of synaptic connections and external shifts; each of the created rules can be used independently or in different combinations with one another. In the course of analytical research, it was found that the optimization formulation of the problem of sorting of relevance values of search pages is identical to the problem of assignment of combinatorial groups of evaluation criteria provided that the elements of the performance matrix of the latter are defined as linear combinations of relevance values.

Several mathematical approaches to studying analytically the dynamics of neural networks rely on mean-field approximations, which are rigorously applicable only to networks of infinite size. However, all existing real biological networks have finite size, and many of them, such as microscopic circuits in invertebrates, are composed only of a few tens of neurons. Thus, it is important to be able to extend to small-size networks our ability to study analytically neural dynamics. Analytical solutions of the dynamics of small-size neural networks have remained elusive for many decades, because the powerful methods of statistical analysis, such as the central limit theorem and the law of large numbers, do not apply to small networks. In this article, we critically review recent progress on the study of the dynamics of small networks composed of binary neurons. In particular, we review the mathematical techniques we developed for studying the bifurcations of the network dynamics, the dualism between neural activity and membrane potentials, cross-neuron correlations, and pattern storage in stochastic networks. Then, we compare our results with existing mathematical techniques for studying networks composed of a finite number of neurons. Finally, we highlight key challenges that remain open, future directions for further progress, and possible implications of our results for neuroscience.

Intelligent manufacturing of ultra-precision optical surfaces is urgently desired but rather difficult to achieve due to the complex physical interactions involved. The development of data-oriented neural networks provides a new pathway, but existing networks cannot be adapted for optical fabrication with a high number of feature dimensions and a small specific dataset. In this Letter, for the first time to the best of our knowledge, a novel Fourier convolution-parallel neural network (FCPNN) framework with library matching was proposed to realize multi-tool processing decision-making, including basically all combination processing parameters (tool size and material, slurry type and removal rate). The number of feature dimensions required to achieve supervised learning with a hundred-level dataset is reduced by 3-5 orders of magnitude. Under the guidance of the proposed network model, a 260 mm × 260 mm off-axis parabolic (OAP) fused silica mirror successfully achieved error convergence after a multi-process involving grinding, figuring, and smoothing. The peak valley (PV) of the form error for the OAP fused silica mirror decreased from 15.153λ to 0.42λ and the root mean square (RMS) decreased from 2.944λ to 0.064λ in only 25.34 hours. This network framework has the potential to push the intelligence level of optical manufacturing to a new extreme.

The analysis of acoustic signals in the water environment is a complex task, complicated by the small number of available datasets and Neural networks are a relevant and powerful tool for classifying acoustic signals in the water environment. Taking into account the current problems in this area, it is advisable to create a neural network framework that can work with acoustic noise, in which the target signal is noisy with background noise of the water environment, which corresponds to real conditions. To solve this problem, a framework of several neural networks can be used, which as a result perform the tasks of noise suppression and subsequent classification. The ability to suppress background noise will improve classification accuracy by filtering out spectral components that are not typical of watercraft. The presence of artifacts uncharacteristic of the target object complicates the classification process, as unnecessary characteristics of the water environment lead to the neural network learning patterns that are not typical for watercraft and reduce classification accuracy. Testing a neural network for noise suppression requires a dataset with a sufficient signal-to-noise ratio that corresponds to real water environment signals. Also, training such neural networks often requires sets of pairs of clean and noisy samples, where the neural network will suppress noise from the noisy samples, and have examples of clean samples as a reference for comparing the work done. The process of obtaining water environment datasets by recording real noises is a rather costly and complex process that does not guarantee satisfactory results. Therefore, the task of creating water environment noise with a given signal-to-noise ratio and the presence of specific vessel noise and background noise in the required specified ratio is relevant. The aim of this work is to create a dataset for training a neural network to suppress background noise in the aquatic environment. Methodology. The software for creating the dataset and the neural network code were developed using Python in the Microsoft Visual Studio Code environment. Scientific novelty. The approach to creating an aquatic environment dataset from two datasets was improved, and a direction for further work to achieve better results was proposed. Conclusions. The proposed approach to dataset creation showed a lower signal-to-noise ratio compared to the approach described in the article. Future plans for the development and improvement of the dataset are described.

Neural networks are efficient and effective in predicting system states in hydrology. However, most current approaches lack hydrological flow partitioning, do not allow for training on measurements of multiple variables, or lack capability to tightly integrate physically-based components. To address these shortcomings I propose and evaluate an approach referred to below as Dynamical System Neural Network (DSNN). DSNN is a feedforward neural network with an architecture that resembles the organisation in components of the real-world system it represents. In hydrology, the DSNN represents each water flow (e.g. seepage, snow melt) by a collection of input, hidden, and output neural layers, where each input is the state of a hydrological storage (e.g. groundwater storage influencing seepage) or other variable (e.g. air temperature influencing snow melt). These components are interconnected to form a single neural network of the complete dynamical system considered, where all storages and flows are explicitly quantified. If physical understanding of a flow and its parameterization is available, a known formulation can be used as a replacement of a neural network component. The DSNN is applied forward in time, backpropagating gradients over all timesteps. It can be run in spatially lumped or semi-distributed mode. To demonstrate the approach, a DSNN is presented of the Austrian Dorfertal (Kals) Alpine catchment containing snow and subsurface water storages and associated flows including streamflow. The DSNN is trained, validated, and tested on daily streamflow over ~40 years. To explore the capability of the DSNN in estimating the magnitude and dynamics of internal system storages (snow water equivalent, subsurface water storage) and flows (evapotranspiration, sublimation, snowmelt, seepage), the DSNN is first trained and tested with streamflow data generated by a conceptual model. The DSNN turns out to be capable of reproducing - with a satisfactory level of precision - the system states and fluxes calculated by the conceptual model, with decreasing performance when measurement error is added to the artificially generated streamflow data before training. To explore its predictive performance, the DSNN is applied on measured streamflow data for the Dorfertal, comparing multiple DSNN setups that represent all flows as neural network components or only a subset of flows where remaining flows are represented with a standard conceptual model (e.g. linear reservoir). Preliminary results indicate that in predictive performance, in most setups, the DSNN outperforms a standard conceptual model trained on the same streamflow data, with NSE values for testing of 0.74 and 0.71, respectively. This preliminary result indicates DSNN to be a promising approach for blending process-based and neural network based modelling as well as for training (i.e. calibration) of neural network models on measurements of multiple hydrological variables as these are all explicitly represented by the DSNN and can thus be incorporated in the loss function (e.g. streamflow, snow depth, groundwater, evapotranspiration).

Diabetic retinopathy (DR) is an increasingly common eye disorder that gradually damages the retina. Identification at the early stage can significantly reduce the severity of vision loss. Deep learning techniques provide detection for retinal images based on data size and quality, as the error rate increases with low-quality images and unbalanced data classes. This paper proposes a hybrid intelligent framework of a conventional neural network and a fuzzy inference system to measure the stages of DR automatically, Diabetic Retinopathy Stage Measurement using Conventional Neural Network and Fuzzy Inference System (DRSM-CNNFIS). The fuzzy inference used human experts’ rules to overcome data dependency problems. At first, the Conventional Neural Network (CNN) model was used for feature extraction, and then fuzzy rules were used to measure diabetic retinopathy stage percentage. The framework is trained using images from Kaggle datasets (Diabetic Retinopathy Detection, 2022). The efficacy of this framework outperformed the other models with regard to accuracy, macro average precision, macro average recall, and macro average F1 score: 0.9281, 0.7142, 0.7753, and 0.7301, respectively. The evaluation results indicate that the proposed framework, without any segmentation process, has a similar performance for all the classes, while the other classification models (Dense-Net-201, Inception-ResNet ResNet-50, Xception, and Ensemble methods) have different levels of performance for each class classification.