Polynomial Approximations for Several Neural Network Activation Functions

Purpose The authors argue that the current studies about malicious insiders confuse the fact that malicious attacks belong to two different categories, namely, those that launch instrumental attacks and expressive attacks. The authors collect malicious insider data from publicly available sources and use text-mining techniques to analyze the association between malicious insiders’ characteristics and the different types of attack. Design/methodology/approach The authors investigated the relationship between personality characteristics and different types of malicious attacks. For the personality characteristics, the authors use the same method as Liang et al. (2016), which extracted these characteristics based on a keyword-characteristic dictionary. For different types of malicious attacks, two raters rated each case based on criteria modified from criminology research to determine the degree of expressiveness and instrumentality. Findings The results show that malicious insiders who are manipulative or seeking personal gain tend to carry out instrumental attacks. Malicious insiders who are arrogant tend to conduct expressive attacks. Research limitations/implications This study uses third party articles to identify the personality characteristics of known malicious insiders. As such, not all personality characteristics may have been reported. Data availability was an issue. Practical implications Understanding if different personality characteristics lead different types of attacks can help managers identify employees who exhibit them and mitigate an attack before it occurs. Social implications Malicious insider attacks can have devastating results on businesses and employees. Help to identify potential malicious insiders before they act, may prevent undue harm. Originality/value This study used 132 cases of none malicious insiders to examine their attack objectives. No other study that the authors know of used that many cases.

This research mainly based on multilayer perceptron (MLP) neural networks technique of data mining to forecast the wheat crop yield at the district level. There are many statistical and simulation models available, but the proposed algorithm with new activation function provides promising results in a shorter time with more accuracy. Sigmoid and hyperbolic tangent activation functions are widely used in the neural network. The activation functions play an important role in the neural network learning algorithm. The main objective of the proposed work is to develop an amended MLP neural network with new activation function and revised random weights and bias values for crop yield estimation by using the different weather parameter datasets. MLP model has been tested by existing activation functions and newly created activation functions with different cases including weights and bias values. In this research study, we evaluate the result of different activation functions and recommend some new simple activation functions, named DharaSig, DharaSigm and SHBSig, to improve the performance of neural networks and accurate results. Also, three new activation functions created with little variations in the DharaSig function named DharaSig1, DharaSig2 and DharaSig3. In this research study, variable numbers of hidden layers are tested with the variable number of neurons per hidden layer for the agriculture dataset. Variable values of momentum, seed and learning rate are also used in this study. Experiments show that newly created activation functions provide better results compared to ‘sigmoid’ default neural network activation function for agriculture datasets.

In artificial neural networks (ANNs), the activation function most used in practice are the logistic sigmoid function and the hyperbolic tangent function. The activation functions used in ANNs have...

There are different types of adversarial attacks and defences for machine learning algorithms which makes assessing the robustness of an algorithm a daunting task. Moreover, there is an intrinsic bias in these adversarial attacks and defences to make matters worse. Here, we organise the problems faced: a) Model Dependence, b) Insufficient Evaluation, c) False Adversarial Samples, and d) Perturbation Dependent Results. Based on this, we propose a model agnostic adversarial robustness assessment method based on L0 and L∞ distance-based norms and the concept of robustness levels to tackle the problems. We validate our robustness assessment on several neural network architectures (WideResNet, ResNet, AllConv, DenseNet, NIN, LeNet and CapsNet) and adversarial defences for image classification problem. The proposed robustness assessment reveals that the robustness may vary significantly depending on the metric used (i.e., L0 or L∞). Hence, the duality should be taken into account for a correct evaluation. Moreover, a mathematical derivation and a counter-example suggest that L1 and L2 metrics alone are not sufficient to avoid spurious adversarial samples. Interestingly, the threshold attack of the proposed assessment is a novel L∞ black-box adversarial method which requires even more minor perturbation than the One-Pixel Attack (only 12% of One-Pixel Attack’s amount of perturbation) to achieve similar results. We further show that all current networks and defences are vulnerable at all levels of robustness, suggesting that current networks and defences are only effective against a few attacks keeping the models vulnerable to different types of attacks.

Intrusion detection System forms a vital component of internet security. To keep pace with the growing trends, there is a critical need to replace single layer detection technology with multi layer detection. Different types of Denial of Service (DoS) attacks thwart authorized users from gaining access to the networks and we tried to detect as well as alleviate some of those attacks. In this paper, we have proposed a novel cross layer intrusion detection architecture to discover the malicious nodes and different types of DoS attacks by exploiting the information available across different layers of protocol stack in order to improve the accuracy of detection. We have used cooperative anomaly intrusion detection with data mining technique to enhance the proposed architecture. We have implemented fixed width clustering algorithm for efficient detection of the anomalies in the MANET traffic and also generated different types of attacks in the network. The simulation of the proposed architecture is performed in OPNET simulator and we got the result as we expected.

An artificial neural network(ANN) is a type of computing system that mimics the functioning of the neural networks in a biological brain. In this paper, an ANN has been simulated on an Field Programmable Gate Array (FPGA) for the application of handwritten digit recognition using the sigmoid and hyperbolic tangent(tanh) activation functions. Both software and hardware simulations have been carried out, using python and Verilog HDL respectively. The two activation functions were implemented on an Artix-7 FPGA. A comparison has been made between the sigmoid and tanh activation functions based on speed, accuracy, and hardware required, and it has been inferred that the tanh function is best for the application of handwritten digit recognition as it has 3% higher accuracy and uses 5 less LUTs than the sigmoid activation function.

In a present-day world there are different types of attacks being launched on computing devices. World is experiencing more and more cyber-attacks and types of attacks are also increasing. For example, an IoT device in a home network can act as a botnet attacking other devices or there could be man in the middle attack. As time goes by more and more devices are being connected within any given network. All these devices will be vulnerable to attacks if any one of the devices is compromised within the network. This complicates Intrusion Detection in any given network. Manual detection and intervention is nearly impossible. Hence it is quintessential to detect different types of attacks with more confidence with less computation complexity and time. A lot of research work has already been done in this area where the attacks have been studied separately. In this paper we focus on detecting intrusions including IoT botnet attacks and other types of network attacks. To achieve this, we build a multiclass classification using supervised learning models along with the dimensionality reduction technique. Numerous studies on ML-based IDS have been using KDD or the upgraded versions of KDD dataset. In this study we used a new dataset, IoT network Intrusion Detection dataset.

Design and investigation of a chaotic neural network architecture for cryptographic applications

We present a simulation-based study of the impacts of different types of attacks in mobile ad hoc networks (MANETs). We consider three common types of attacks: blackhole attack, jellyfish attack and neighbor attack. Specifically, we study how the number of attackers and their positions affect the performance metrics of a connection such as packet delivery ratio, throughput, end-to-end delay, and delay jitter. The results enable us to suggest measures to minimize the impacts of the above types of attacks in MANETs.

Utilizing speech as the transmission medium in Internet of things (IoTs) is an effective way to reduce latency while improving the efficiency of human-machine interaction. In the field of speech recognition, Recurrent Neural Network (RNN) has significant advantages to achieve accuracy improvement on speech recognition. However, some of RNN-based intelligence speech recognition applications are insufficient in the privacy-preserving of speech data, and others with privacy-preserving are time-consuming, especially about model training and speech recognition. Therefore, in this paper we propose a novel Privacy-preserving Speech Recognition framework using Bidirectional Long short-term memory neural network, namely PSRBL. On the one hand, PSRBL designs new functions to construct security activation functions by combing with an additive secret sharing protocol, namely a secure piecewise-linear Sigmoid and a secure piecewise-linear Tanh respectively, to achieve privacy-preserving of speech data during speech recognition process running on edge servers. On the other hand, in order to reduce the time spent on both the training and the recognition of the speech model while keeping high accuracy during speech recognition process, PSRBL first utilizes secure activation functions to refit original activation functions in the bidirectional Long Short-Term Memory neural network (LSTM), and then makes full use of the left and the right context information of speech data by employing bidirectional LSTM. Experiments conducted on the speech dataset TIMIT show that our framework PSRBL performs well. Specifically compared with the state-of-the-art ones, PSRBL significantly reduces the time consumption on both the training and the recognition of the speech model under the premise that PSRBL and the comparisons are consistent in the privacy-preserving of speech data.

Robust Control Theory Based Stability Certificates for Neural Network Approximated Nonlinear Model Predictive Control

Model-free PID control tuning methods have been widely reported for the case with the time-varying uncertain systems. A model-free control system is introduced in this paper with PID controller. This control design could be considered as a contribution to kernel wavelet neural network PID controllers in model-free system with a straightforward tuning by the simultaneous perturbation stochastic approximation algorithm. It is particularly well suited to problems involving the measurement data without prior inner structure knowledge of the unknown system. On the other hand, neural network activation functions are bounded which cause the outputs of neural network to be limited. Therefore, the T-S Fuzzy control method on basis of kernel wavelet neural network (KWNN) is introduced to improve the control parameter tuning adaptivity in terms of boundedness of the KWNN excitation function. Finally, simulation results are presented to exhibit the effectiveness of the proposed IPWR control system.

This paper considers the interconnection of a continuous time linear time-invariant system and a multivariable magnitude nonlinearity. A number of different quadratic constraints are established for the magnitude nonlinearity and then used to derive stability criteria based on quadratic and Lurie-type Lyapunov functions. The new stability criteria are cast as matrix inequalities and in some cases solved using semi-definite programming. Connections are made between the magnitude nonlinearity and neural network activation functions, such as the rectified linear unit (ReLU) and leaky ReLU, effectively allowing the stability criteria derived here to be used to analyse interconnections of dynamical systems and neural networks. Using the positive homogeneity property, shared by the (leaky) ReLU and magnitude functions, mild conditions are also established to show that the existence of a unique equilibrium point is sufficient for local and global stability to be equivalent. Finally, the new global stability criteria are tested on several numerical examples, including a Hopfield network with 100 states and neurons, and compare favourably with competing criteria from the literature.

Application of Neural Networks in Computer Security

Activation functions play increasingly important roles in deep learning. The activation functions give the neural network the ability to learn complex patterns by introducing non-linearity. Complex-valued activation functions for complex-valued neural networks are extended from real-valued activation functions like sigmoid, tanh, rectified linear unit, and exponential linear unit. Recently, the most widely used complex-valued activation function is the real-imaginary-type activation function (RIAF), which applies separate real-valued activation functions on both the real and the imaginary parts of a neuron. This separate activation method ignores the internal relationship between the real and imaginary parts of the complex input and loses the integrity of activation. In this work, a complex-valued activation function called Gaussian-type activation function (GTAF) is proposed to activate the real and imaginary parts of a neuron jointly and enhance the learning ability and learning speed. GTAF consists of an ordinary real-valued activation function and a Gaussian function and is suitable for any given complex-valued neural network. Two polar-SAR image classification experiments on complex-valued convolutional neural networks show that GTAF converges faster than RIAF and has higher accuracy.