Input Similarity from the Neural Network Perspective

In this paper, we propose and demonstrate a low-complexity fiber nonlinearity impairments compensation (NLC) scheme based on simple recurrent neural network (SRNN), combining with the triplets derived from the first-order perturbation solution of the coupled nonlinear Schrodinger equation. Benefitting from the time memory of SRNN, the proposed NLC can reduce the computational complexity without sacrificing the compensation performance compared with the schemes based on memoryless feedforward neural networks. To verify the feasibility of the proposed NLC, the simulation systems of single channel and three channel 30 GBaud polarization division multiplexing (PDM) 16 quadrature amplitude modulation (QAM) over 2400 km standard single mode fiber transmission are constructed respectively. Comprehensive numerical simulation results demonstrate that although the NLC performance of proposed scheme is inferior to that of digital back propagation (DBP) with much higher complexity and impracticality, it can still achieve the competitive NLC performance with lower computational complexity compared to the schemes based on deep neural network (DNN) and artificial neural network (ANN). Moreover, it is further proved that using triplets as the feature required for NLC can cause the pattern of pseudo random bit sequence to be destroyed, resulting in the neural network-based NLC scheme not suffering from the problem of overestimation for performance. Additionally, the time memory of SRNN enables the number of triplets in training set required by the SRNN-based NLC to be reduced by 169% compared to that of DNN and ANN. The experimental results of 16 GBaud PDM-16QAM wavelength division multiplexing coherent systems demonstrate that compared to with ANN and DNN, the comparable NLC performance can also be achieved by the proposed scheme with lower complexity.

Deep neural networks are used increasingly for perception and decision-making in UAVs. For example, they can be used to recognize objects from images and decide what actions the vehicle should take. While deep neural networks can perform very well at complex tasks, their decisions may be unintuitive to a human operator. When a human disagrees with a neural network prediction, due to the black box nature of deep neural networks, it can be unclear whether the system knows something the human does not or whether the system is malfunctioning. This uncertainty is problematic when it comes to ensuring safety. As a result, it is important to develop technologies for explaining neural network decisions for trust and safety. This paper explores a modification to the deep neural network classification layer to produce both a predicted label and an explanation to support its prediction. Specifically, at test time, we replace the final output layer of the neural network classifier by a k-nearest neighbor classifier. The nearest neighbor classifier produces 1) a predicted label through voting and 2) the nearest neighbors involved in the prediction, which represent the most similar examples from the training dataset. Because prediction and explanation are derived from the same underlying process, this approach guarantees that the explanations are always relevant to the predictions. We demonstrate the approach on a convolutional neural network for a UAV image classification task. We perform experiments using a forest trail image dataset and show empirically that the hybrid classifier can produce intuitive explanations without loss of predictive performance compared to the original neural network. We also show how the approach can be used to help identify potential issues in the network and training process.

We propose a hybrid prediction system of neural network (NN) and memory based learning (MBR). NN and MBR are frequently applied to data mining with various objectives. NN and MBR can be directly applied to classification and regression without additional transformation mechanisms. They also have strength in learning the dynamic behavior of the system over a period of time. In our hybrid system of NN and MBR, the feature weight set which is calculated from the trained NN plays the core role in connecting both learning strategies and the explanation on prediction can be given by obtaining and presenting the most similar examples from the case base. Experimental results show that the hybrid system has a high potential in solving data mining problems.

Problem statement. Deep neural networks are successfully used in channel coding to improve the quality of decoding. However, modern neural channel decoders cannot achieve high decoding performance and low complexity at the same time. To overcome this problem, it is necessary to develop a decoder with a double residual neuron. By integrating residual input and residual learning into the neural channel decoder architecture, it will significantly improve decoding performance while maintaining low complexity. Goal. Development of a decoder architecture with a dual residual neural network, which provides high decoding performance in channel coding with low model complexity and computational costs. Results. The simulation results show that the proposed decoder with a double residual neural network provides a significant increase in decoding performance. Compared to a neural decoder based on a trust propagation algorithm, a decoder with a dual residual neural network provides an additional 0.5-1.8 dB encoding gain compared to different channel codes. Compared to the hypergraph neural decoder, which has the best error correction capability among all existing neural channel decoders, the dual residual neural network decoder also provides similar or even better decoding performance for different channel codes. Practical significance. The results obtained can be used to improve decoding efficiency and improve performance in channel coding. The work was carried out with the financial support of the Ministry of Science and Higher Education of the Russian Federation within the framework of the state assignment "youth laboratory" No. FZGM-2024-0003.

The identification of emergent structures in dynamical systems is a major challenge in complex systems science. In particular, the formation of intermediate-level dynamical structures is of particular interest for what concerns biological as well as artificial systems. In this work, we present a set of measures aimed at identifying groups of elements that behave in a coherent and coordinated way and that loosely interact with the rest of the system (the so-called “relevant sets”). These measures are based on Shannon entropy, and they are an extension of a measure introduced for detecting clusters in biological neural networks. Even if our results are still preliminary, we have evidence for showing that our approach is able to identify and partially characterise the relevant sets in some artificial systems, and that this way is more powerful than usual measures based on statistical correlation. In this work, the two measures that contribute to the cluster index, previously adopted in the analysis of neural networks, i.e. integration and mutual information, are analysed separately in order to enhance the overall performance of the so-called dynamical cluster index. Although this latter variable already provides useful information about highly integrated subsystems, the analysis of the different parts of the index are extremely useful to better characterise the nature of the sub-systems.

The aim of our work was to make a metrological analysis of neural network measuring systems for diagnosing a person’s condition. The approach methodology was based on the concept of the proposed class of measuring systems on neural networks for diagnosing of the state of complex objects. They are differ than the software (or software-hardware) implementation of the measuring system with the neural network environment, which is using at present time; considering a neural network as a super numerous channel measuring system; approach to the diagnosis of the state of the object as system measurements of the integral parameter; categorically-operational metrological description of neural network measuring functional converters. We proposed to consider neural networks as ordinary measuring instrument. The description of their operation can be form from the point of view of classical measurements with traditional metrological estimates. The procedure for bringing the neural network measurement methods to the classical ones is most constructed on the basis of formalisms of general algebra, graphs, theory of categories and functors, system and operational approaches. The metrological description of neural networks is based on a consistent description and analysis of measurement errors and their characteristics, which is formed from the accepted representation of the measurement procedure as a sequence of transformations of the input action. We presented some examples of the metrological analysis of neural network measuring systems for diagnosing a person’s condition (acupuncture diagnostics of the bio-object condition, monitoring system of the main vital functions of a person). The categorical operational representation of the metrological description makes it possible to obtain traditional metrological estimates of the process of neural network measurements, to detail various types of errors in measuring transformations (for neurons, neuronal layers and measuring channels), to analyse the influence of the neural network structure on the measurement error, to perform the synthesis of neural network measuring systems with specified metrological characteristics. We can form a quantitative express diagnostics of internal organs, functional systems and the organism as a whole using neural network diagnostic system.

Many techniques have been developed for wireless signal recognition in many fifth generation (5G) enabled derivatives. Many harsh constraints, such as the large amount of model parameters and complex signal characteristics, drives intelligent recognition method in real-world settings. In this paper, we propose a generalizable, practical method for raw IQ signal recognition. Specifically, deep complex-valued convolutional neural network models, including a Complex-valued Visual Geometry Group (VGG) (CxVGG) model and a Complex-valued ResNet (CxRN) model, are proposed for handling raw signal IQ data. We examine the merit of complex-valued neural networks (CxNN) and validate their performance with experiments using two public datasets. With an SNR of 10dB, the proposed algorithm achieves a recognition accuracy of 96% on the RadioML2018.10a dataset. When performing drone recognition, CvNN can achieve a recognition accuracy of 99%. Our experimental results verify that deep complex-valued neural network models can achieve considerably improved accuracy with lower computation complexity and fewer model parameters than their real-valued counterparts.

Thyroid nodules with a predominant follicular structure are often diagnosed as indeterminate at fine-needle aspiration biopsy (FNAB). We studied 453 patients with a thyroid nodule diagnosed as indeterminate at FNAB by using a feed-forward artificial neural network (ANN) analysis to integrate cytologic and clinical data, with the goal of subgrouping patients into a high-risk and in a low-risk category. Three hundred seventy-one patients were used to train the network and 82 patients were used to validate the model. The cytologic smears were blindly reviewed and classified in a high-risk and a low-risk subgroup on the basis of standard criteria. Neural network analysis subdivided the 371 lesions of the first series into a high-risk group (cancer rate of approximately 33% at histology) and a low-risk group (cancer rate of 3%). Only cytologic parameters contributed to this classification. Analysis of the receiver operating characteristic (ROC) curves demonstrated that the ANN model discriminated with higher sensitivity and specificity between benign and malignant nodules compared to standard cytologic criteria (p < 0.001). This value did not show degradation when ANN predictions were applied to the validation series of 82 nodules. In conclusion, neural network analysis of cytologic data may be a useful tool to refine the risk of cancer in patients with lesions diagnosed as indeterminate by FNAB.

Forecasting financial time series using a low complexity recurrent neural network and evolutionary learning approach

Time-varying quadratic programming (TV-QP) is widely used in artificial intelligence, robotics, and many other fields. To solve this important problem, a novel discrete error redefinition neural network (D-ERNN) is proposed. By redefining the error monitoring function and discretization, the proposed neural network is superior to some traditional neural networks in terms of convergence speed, robustness, and overshoot. Compared with the continuous ERNN, the proposed discrete neural network is more suitable for computer implementation. Unlike continuous neural networks, this article also analyzes and proves how to select the parameters and step size of the proposed neural networks to ensure the reliability of the network. Moreover, how to achieve the discretization of the ERNN is presented and discussed. The convergence of the proposed neural network without disturbance is proven, and bounded time-varying disturbances can be resisted in theory. Furthermore, the comparison results with other related neural networks show that the proposed D-ERNN has a faster convergence speed, better antidisturbance ability, and lower overshoot.

Language understanding is multimodal. During human communication, messages are conveyed not only by words in textual form, but also through speech patterns, gestures or facial emotions of the speakers. Therefore, it is crucial to fuse information from different modalities to achieve a joint comprehension. With the rapid progress in the deep learning field, neural networks have emerged as the most popular approach for addressing multimodal data fusion [1, 6, 7, 12]. While these models can effectively combine multimodal features by learning from data, they nevertheless lack an explicit exhibition of how different modalities are related to each other, due to the inherent low interpretability of neural networks [2]. In the meantime, Quantum Theory (QT) has given rise to principled approaches for incorporating interactions between textual features into a holistic textual representation [3, 5, 8, 10], where the concepts of superposition andentanglement have been universally exploited to formulate interactions. The advantages of those models in capturing complicated correlations between textual features have been observed. We hereby propose the research on quantum-inspired multimodal data fusion, claiming that the limitation of multimodal data fusion can be tackled by quantum-driven models. In particular, we propose to employ superposition to formulate intra-modal interactions while the interplay between different modalities is expected to be captured by entanglement measures. By doing so, the interactions within multimodal data may be rendered explicitly in a unified quantum formalism, increasing the performance and interpretability for concrete multimodal tasks. It will also expand the application domains of quantum theory to multimodal tasks where only preliminary efforts have been made [11]. We therefore aim at answering the following research question: RQ. Can we fuse multimodal data with quantum-inspired models? To answer this question, we propose to fuse multimodal data with complex-valued neural networks, motivated by the theoretical link between neural networks and quantum theory [4] and advances in complex-valued neural networks [9]. Our model begins with a separate complex-valued embedding learned for each unimodal data based on the existing works [5, 10] which inherently assumes superposition between intra-modal features. Then we construct a many-body system in entangled state for multimodal data, where cross-modality interactions are naturally reflected by entanglement measures. Quantum measurement operators are applied to the entanglement state to address a concrete multimodal task at hand. The whole process is instrumented by a complex-valued neural network, which is able to learn how multimodal features are combined from data, and at the same time explain the combination by means of quantum superposition and entanglement measures. We plan to examine our proposed models on CMU-MOSI [12] and CMU-MOSEI [1] which are benchmarking multimodal sentiment analysis datasets. The dataset targets at classifying sentiment into 2, 5 or 7 classes with the input of textual, visual and acoustic features. We expect to see comparable effectiveness to state-of-the-art models, and we will explore superposition and entanglement measures to better understand the inter-modal interactions.

The goal of this article is to use operating data-based approaches for automating the manufacturing of submicron flash memory devices in semiconductor. This novel technique which combines the neural network and information inductive analysis has recently been proposed. It is used in this article to generate the recipe for plasma etching process design. Traditional plasma etching variables such as pressure, gas flows, temperature, rf power, etc., are used to build the neural network for predicting etching rate of polysilicon and field oxide, and the uniformity of field oxide. The information inductive analysis based on the information entropy and fuzzy clustering analysis is then utilized to look for the candidate points in each optimal region whose response surface is constructed based on the neural network model. With only a few runs, the best optimal condition getting close to the design requirements is found. Since the complexity of plasma modeling and design at the equipment level is presently ahead of theoretical method from a fundamental physical standpoint, the proposed method can effectively cope with nonlinear characteristics in the plasma etching process, giving good design directions and taking advantage of traditional statistical approaches. Using the proposed method within four runs and a total of 26 experimental points, the recipe that meets all specifications is found.

The increasing hardware failures caused by the shrinking semiconductor technologies pose substantial influence on the neural accelerators and improving the resilience of the neural network execution becomes a great design challenge especially to mission-critical applications such as self-driving and medical diagnose. The reliability analysis of the neural network execution is a key step to understand the influence of the hardware failures, and thus is highly demanded. Prior works typically conducted the fault analysis of neural network accelerators with simulation and concentrated on the prediction accuracy loss of the models. There is still a lack of systematic fault analysis of the neural network acceleration system that considers both the accuracy degradation and system exceptions such as system stall and early termination.In this work, we implemented a representative neural network accelerator and fault injection modules on a Xilinx ARM-FPGA platform and conducted fault analysis of the system using four typical neural network models. We had the system open-sourced on github. With comprehensive experiments, we identify the system exceptions based on the various abnormal behaviours of the FPGA-based neural network acceleration system and analyze the underlying reasons. Particularly, we find that the probability of the system exceptions dominates the reliability of the system and they are mainly caused by faults in the DMA, control unit and instruction memory of the accelerators. In addition, faults in these components also incur moderate accuracy degradation of the neural network models other than the system exceptions. Thus, these components are the most fragile part of the accelerators and need to be hardened for reliable neural network execution.

Security analysis and prediction of multi-relay networks over Fisher–Snedecor [formula omitted] fading channels

With the scaling technology, soft error induced bit upsets are increasingly threatening the processor reliability. Processor designers require effective tools or methodologies to estimate the often-used metric Architectural Vulnerability Factor (AVF). This paper presents an ensemble learning based AVF calculation methodology for fast reliability assessment. Instead of the entire feature set, only partial non-critical attributes are selected to build the predictive model so that many performance counters can be removed or shut down for saving memory space and power consumption. Millions of data collected from a cycle-accurate simulator sim-SODA, are trained by the latest learning methods in Tensorflow. The SPEC2000 results demonstrate the instanced ensemble learning-random forest and Ada-boost perform nearly perfect accuracy, better than linear regression, and neural network.