Fundamental neural structures, operations, and asymptotic performance criteria in decentralized binary hypothesis testing

Two fundamental distributed decision network structures are considered: the first system consists of finite number of sensors, each collecting asymptotically many data, while the second one employs asymptotically many sensors, each collecting a single datum. For binary hypothesis testing, the Neyman-Pearson criterion is utilized and justified via information theoretic arguments. An asymptotic relative efficiency performance measure is used to establish tradeoffs between the two structures, by comparing the performance characteristics of the decentralized detection systems to their centralized counterparts. >

Wireless sensor networks are prone to node misbehavior arising from tampering by an adversary (Byzantine attack), or due to other factors such as node failure resulting from hardware or software degradation. In this paper, we consider the problem of decentralized detection in wireless sensor networks in the presence of one or more classes of misbehaving nodes. Binary hypothesis testing is considered where the honest nodes transmit their binary decisions to the fusion center (FC), while the misbehaving nodes transmit fictitious messages. The goal of the FC is to identify the misbehaving nodes and to detect the state of nature. We identify each class of nodes with an operating point (false alarm and detection probabilities) on the receiver operating characteristic (ROC) curve. Maximum likelihood estimation of the nodes' operating points is then formulated and solved using the expectation maximization (EM) algorithm with the nodes' identities as latent variables. The solution from the EM algorithm is then used to classify the nodes and to solve the decentralized hypothesis testing problem. Numerical results compared with those from the reputation-based schemes show a significant improvement in both classification of the nodes and hypothesis testing results. We also discuss an inherent ambiguity in the node classification problem which can be resolved if the honest nodes are in majority.

<p>Deep neural networks have achieved tremendous success in many domains (e.g., computer vision~\cite{Alexnet12,vggnet15,fastrcnn15}, speech recognition~\cite{hinton2012deep,dahl2012context}, natural language processing~\cite{dahl2012context,collobert2011natural}, games~\cite{silver2017mastering,silver2016mastering}), however, there are still many challenges in deep learning comunity such as how to speed up training large deep neural networks, how to compress large nerual networks for mobile/embed device without performance loss, how to automatically design the optimal network structures for a certain task, and how to further design the optimal networks with improved performance and certain model size with reduced computation cost.</p> <p>To speed up training large neural networks, we propose to use multinomial sampling for dropout, i.e., sampling features or neurons according to a multinomial distribution with different probabilities for different features/neurons. To exhibit the optimal dropout probabilities, we analyze the shallow learning with multinomial dropout and establish the risk bound for stochastic optimization. By minimizing a sampling dependent factor in the risk bound, we obtain a distribution-dependent dropout with sampling probabilities dependent on the second order statistics of the data distribution. To tackle the issue of evolving distribution of neurons in deep learning, we propose an efficient adaptive dropout (named evolutional dropout) that computes the sampling probabilities on-the-fly from a mini-batch of examples.</p> <p>To compress large neural network structures, we propose a simple yet powerful method for compressing the size of deep Convolutional Neural Networks (CNNs) based on parameter binarization. The striking difference from most previous work on parameter binarization/quantization lies at different treatments of $1\times 1$ convolutions and $k\times k$ convolutions ($k>1$), where we only binarize $k\times k$ convolutions into binary patterns. By doing this, we show that previous deep CNNs such as GoogLeNet and Inception-type Nets can be compressed dramatically with marginal drop in performance. Second, in light of the different functionalities of $1\times 1$ (data projection/transformation) and $k\times k$ convolutions (pattern extraction), we propose a new block structure codenamed the pattern residual block that adds transformed feature maps generated by $1\times 1$ convolutions to the pattern feature maps generated by $k\times k$ convolutions, based on which we design a small network with $\sim 1$ million parameters. Combining with our parameter binarization, we achieve better performance on ImageNet than using similar sized networks including recently released Google MobileNets.</p> <p>To automatically design neural networks, we study how to design a genetic programming approach for optimizing the structure of a CNN for a given task under limited computational resources yet without imposing strong restrictions on the search space. To reduce the computational costs, we propose two general strategies that are observed to be helpful: (i) aggressively selecting strongest individuals for survival and reproduction, and killing weaker individuals at a very early age; (ii) increasing mutation frequency to encourage diversity and faster evolution. The combined strategy with additional optimization techniques allows us to explore a large search space but with affordable computational costs.</p> <p>To further design the optimal networks with improved performance and certain model size under reduced computation cost, we propose an ecologically inspired genetic approach for neural network structure search , that includes two types of succession: primary and secondary succession as well as accelerated extinction. Specifically, we first use primary succession to rapidly evolve a community of poor initialized neural network structures into a more diverse community, followed by a secondary succession stage for fine-grained searching based on the networks from the primary succession. Accelerated extinction is applied in both stages to reduce computational cost. In addition, we also introduce the gene duplication to further utilize the novel block of layers that appeared in the discovered network structure.</p>

Robust decentralized detection by asymptotically many sensors

Deep neural networks DNN are currently the basis of many modern AI ap-plications has been widely applied in various domains. As more safety-oriented fields autonomous driving, medical diagnosis, etc. begin to use DNN, people have put forward new requirements for DNN. Not only the ac-curacy of DNN and other objective indicators be excellent, but also have ro-bustness and ability to handle various corner cases. It is important to test the adequacy of the deep neural network model, design appropriate evaluation indicators, build a complete test evaluation system. However, deep neural network computing like a black box, a slight disturbance to the input may cause errors in the final output of the model. Therefore, it is important to test the adequacy of the deep neural network model, design appropriate eval-uation indicators, build a complete test evaluation system. We prove that there are differences in the internal structure of neural networks for different types of input. Based on this discovery, we proposed Multi-Layer test criteria based on the neural network structure. To quantify and analyze the changes in the internal structure of neural networks under different types of input, this paper pro poses an algorithm for mapping the deep neural network to tree structure data. Finally, a Multi-Layer test criteria based on the neural network structure is proposed to guide the generation of test cases, which can generate high-quality test cases.

Real-time detection of the pavement environment is an important part of autonomous driving technology. This paper presents a real-time vehicle detection system based on embedded devices. Based on the existing yolov3-tiny neural network structure, this paper proposes a new neural network structure - the YOLO v3-live. Tailoring the network layer structure of YOLOv3-tiny, and quantify the network parameters in the network. Reducing the complexity of computing in embedded devices, making the proposed neural network structure more suitable for embedded devices. The new structure is tested, before quantization the YOLO v3-live's detection precision can achieve 87.79% mAP on the test set, after quantization of network parameters can achieve 69.79% mAP and detection speed can achieve 28 FPS.

Dynamic neural network control for non-linear systems: optimal neural network structure and stability analysis

Currently, there is a great need for mobile monitoring systems used to detect heart rate abnormalities in citizens working or living in hard-to-reach areas. Due to problems with transmitting data from such areas, it is desirable to carry out the entire information processing process at the study site. But, despite the limited hardware costs for the implementation of such systems, it is necessary to ensure high reliability of monitoring results. Purpose is to develop a structural diagram of a mobile heart rate analysis device, select the architecture of the neural network of the conversion and segmentation subsystem of the mobile device, determine the structure of the neural network of the conversion and segmentation subsystem, train the neural network of the conversion and segmentation subsystem, conduct software modeling of the neural network of the conversion and segmentation subsystem, implementation of a neural network conversion and segmentation subsystem based on field programmable gate arrays (FPGAs). A structural diagram of a mobile heart rate analysis device has been developed. The choice of the neural network architecture of the mobile device conversion and segmentation subsystem is justified. The structure of the neural network of the transformation and segmentation subsystem is determined. The neural network of the transformation and segmentation subsystem was trained. A software simulation of the neural network of the transformation and segmentation subsystem was carried out, confirming the high reliability of the proposed neural network structure. A neural network conversion and segmentation subsystem has been implemented in the hardware description language VHDL based on an FPGA. The developed structural diagram of the mobile device and the neural network structure of the conversion and segmentation subsystem can be used when implementing mobile heart rate monitoring means.

This paper brings forward two kinds of PD control schemes of adaptive neural-variable structure for uncertain robot trajectory tracking. The first scheme consists of a PD feedback and a dynamic compensator which is composed of RBF neural network and variable structure. The adaptive laws of Network weights are based on Lyapunov function method. This controller can guarantee stability of closed-loop system and asymptotic convergence of tracking errors. The second scheme substitutes the integrated controller consisting of neural network and variable structure for the hybrid controller by way of smooth function. This integrated controller can reduce chattering of variable structure control input, overcome the deficiencies of local generalization neural networks and improve control precision and convergence speed. In addition, This controller is still able to ensure the system maintains good robustness and stability in the case of neural network disabled. The simulation results have showed the effectiveness of two kinds of control schemes, and that the second scheme is more advantageous.

A new neural network structure known as the fuzzy differentiable Cerebellar Model Articulation Controller (FD_CMAC) was presented in this study. The FD_CMAC mentioned in this study involved embedding the differentiable functions (such as Gaussian function) into the hypercube of the receptive field space, thereby forming non-fixed values and differentiable blocks. In addition, in the network structure, the fuzzy inference engine was introduced to replace the index addressing matrix of the association memory. Additionally, the fuzzy membership function was adopted to describe the mapping of the physical memory. The new network structure not only retained the features of CMAC, but also adopted the differentiable characteristics. The traditional weight equipartition is no longer the only method in the learning mechanism. Instead, the steepest descent method was adopted to derive at optimized parameters or weight memory contents. Confirmed through the simulation results, the network structure proposed in this study was not only feasible, but also possessed good approximation effects and convergence performance.

Proposes a new neural network structure to design a robust controller for non-exactly known nonlinear systems. Based on input-output data, the neural network structure provides a global affine model of the controlled system which is compatible with well known nonlinear control techniques. An affine neural tracking controller is developed and applied to an inverted pendulum submitted to random perturbations. The performance of the controller and the main advantage of the proposed approach are discussed.

The classical result of Blahut, which characterizes achievable error exponents in binary hypothesis testing, is generalized in two different directions. First, in M-ary hypothesis testing, the tradeoff of all M(M-1) types of error exponents and corresponding optimal decision schemes are explored. Then, motivated by a power-constrained distributed detection scenario, binary hypothesis testing is revisited, and the tradeoff of power consumption versus error exponents is fully characterized. In the latter scenario, sensors are allowed to make random decisions as to whether they should remain silent and save power, or transmit and improve detection quality. It is then shown by an example that optimal sensor decisions may indeed be random

With the advent of the Internet of Things (IoT), the security of the network layer in the IoT is getting more and more attention. The traditional intrusion detection technologies cannot be well adapted in the complex Internet environment of IoT. For the deep learning algorithm of intrusion detection, a neural network structure may have fine detection accuracy for one kind of attack, but it may not have a good detection effect when facing other attacks. Therefore, it is urgent to design a self-adaptive model to change the network structure for different attack types. This paper presents an intrusion detection model based on improved genetic algorithm (GA) and deep belief network (DBN). Facing different types of attacks, through multiple iterations of the GA, the optimal number of hidden layers and number of neurons in each layer are generated adaptively, so that the intrusion detection model based on the DBN achieves a high detection rate with a compact structure. Finally, the NSL-KDD dataset was used to simulate and evaluate the model and algorithms. The experimental results show that the improved intrusion detection model combined with DBN can effectively improve the recognition rate of intrusion attacks and reduce the complexity of the neural network structure.

In order to accelerate the learning ability of neural network structure parameters and improve the prediction accuracy of deep learning algorithms, an evolutionary algorithm, based on a prior Gaussian mutation (PGM) operator, is proposed to optimize the structure parameters of a gated recurrent unit (GRU) neural network. In this algorithm, the sensitivity learning process of GRU model parameters into the Gaussian mutation operator, used the variance of the GRU model parameter training results as the Gaussian mutation variance to generate the optimal individual candidate set. Then, the optimal GRU neural network structure is constructed using the evolutionary algorithm of the prior Gaussian mutation operator. Moreover, the PGM-EA-GRU algorithm is applied to the prediction of stock market returns. Experiments show that the prediction model effectively overcomes the GRU neural network, quickly falling into a local optimum and slowly converging. Compared to the RF, SVR, RNN, LSTM, GRU, and EA-GRU benchmark models, the model significantly improves the searchability and prediction accuracy of the optimal network structure parameters. It also validates the effectiveness and the progressive nature of the PGM-EA-GRU model proposed in this paper with stock market return prediction.

Drilling is one of the most common and fundamental machining processes. Since approximately 40% of all the cutting operations are drilling in industry. It is most frequently performed, material removing process and is used as a preliminary step for many operations, such as reaming, tapping and boring. Because of their importance in nearly all production operations twist drills have been the subject of numerous investigations. Surface finish quality of a machined work piece is an issue of main concern to the manufacturing industry. The aim of the present work is to identify suitable parameters for the prediction of surface roughness. Back propagation neural networks were used for detection of surface roughness. Drill diameter, cutting speed, feed, and machining time were given as inputs to the neural network structure and surface roughness was estimated. Drilling experiments with 10 mm drill size were performed at three cutting speeds and feeds. The number of neurons were selected from 1,2,3,…, 20. The learning rate was selected as 0.01 and no smoothing factor was used. The best structure of neural networks were selected based on the criteria as the minimum of summation of square with the actual value of surface roughness. For statistical analysis, it was assumed that surface roughness depends on cutting speed, feed and machining time. For the mathematical analysis inverse coefficient matrix method was used for calculating the estimated values of surface roughness. Comparative analysis has been done between the actual values and the estimated values obtained by statistical analysis, mathematical analysis and neural network structure.