Personality Detection from Text using Convolutional Neural Network

Alzheimer's disease (AD) is a progressive brain disease. The goal of this study is to provide a new computer-vision based technique to detect it in an efficient way. The brain-imaging data of 98AD patients and 98 healthy controls was collected using data augmentation method. Then, convolutional neural network (CNN) was used, CNN is the most successful tool in deep learning. An 8-layer CNN was created with optimal structure obtained by experiences. Three activation functions (AFs): sigmoid, rectified linear unit (ReLU), and leaky ReLU. The three pooling-functions were also tested: average pooling, max pooling, and stochastic pooling. The numerical experiments demonstrated that leaky ReLU and max pooling gave the greatest result in terms of performance. It achieved a sensitivity of 97.96%, a specificity of 97.35%, and an accuracy of 97.65%, respectively. In addition, the proposed approach was compared with eight state-of-the-art approaches. The method increased the classification accuracy by approximately 5% compared to state-of-the-art methods.

This paper considers the interconnection of a continuous time linear time-invariant system and a multivariable magnitude nonlinearity. A number of different quadratic constraints are established for the magnitude nonlinearity and then used to derive stability criteria based on quadratic and Lurie-type Lyapunov functions. The new stability criteria are cast as matrix inequalities and in some cases solved using semi-definite programming. Connections are made between the magnitude nonlinearity and neural network activation functions, such as the rectified linear unit (ReLU) and leaky ReLU, effectively allowing the stability criteria derived here to be used to analyse interconnections of dynamical systems and neural networks. Using the positive homogeneity property, shared by the (leaky) ReLU and magnitude functions, mild conditions are also established to show that the existence of a unique equilibrium point is sufficient for local and global stability to be equivalent. Finally, the new global stability criteria are tested on several numerical examples, including a Hopfield network with 100 states and neurons, and compare favourably with competing criteria from the literature.

The convolutional neural networks (CNN), including AlexNet, GoogleNet, VGGNet, etc. extract features for many computer vision problems which are very discriminative. The trained CNN model over one dataset performs reasonably well whereas on another dataset of similar type the hand-designed feature descriptor outperforms the same trained CNN model. The Rectified Linear Unit (ReLU) layer discards some values in order to introduce the non-linearity. In this paper, it is proposed that the discriminative ability of deep image representation using trained model can be improved by Average Biased ReLU (AB-ReLU) at the last few layers. Basically, AB-ReLU improves the discriminative ability in two ways: 1) it exploits some of the discriminative and discarded negative information of ReLU and 2) it also neglects the irrelevant and positive information used in ReLU. The VGGFace model trained in MatConvNet over the VGG-Face dataset is used as the feature descriptor for face retrieval over other face datasets. The proposed approach is tested over six challenging, unconstrained and robust face datasets (PubFig, LFW, PaSC, AR, FERET and ExtYale) and also on a large scale face dataset (PolyUNIR) in retrieval framework. It is observed that the AB-ReLU outperforms the ReLU when used with a pre-trained VGGFace model over the face datasets. The validation error by training the network after replacing all ReLUs with AB-ReLUs is also observed to be favorable over each dataset. The AB-ReLU even outperforms the state-of-the-art activation functions, such as Sigmoid, ReLU, Leaky ReLU and Flexible ReLU over all seven face datasets.

The application of deep learning technology has increased rapidly in recent years. Technologies in deep learning increasingly emulate natural human abilities, such as knowledge learning, problem-solving, and decision-making. In general, deep learning can carry out self-training without repetitive programming by humans. Convolutional neural networks (CNNs) are deep learning algorithms commonly used in wide applications. CNN is often used for image classification, segmentation, object detection, video processing, natural language processing, and speech recognition. CNN has four layers: convolution layer, pooling layer, fully connected layer, and non-linear layer. The convolutional layer uses kernel filters to calculate the convolution of the input image by extracting the fundamental features. The pooling layer combines two successive convolutional layers. The third layer is the fully connected layer, commonly called the convolutional output layer. The activation function defines the output of a neural network, such as 'yes' or 'no'. The most common and popular CNN activation functions are Sigmoid, Tanh, ReLU, Leaky ReLU, Noisy ReLU, and Parametric Linear Units. The organization and function of the visual cortex greatly influence CNN architecture because it is designed to resemble the neuronal connections in the human brain. Some of the popular CNN architectures are LeNet, AlexNet and VGGNet.

Price movement predictions of financial instruments using traditional time-series models with a predefined mathematical structure are common, but this method restricts their ability to learn latent patterns in the data. In recent times, artificial neural networks (ANN) have been able to learn complex hidden patterns from financial datasets using a highly nonlinear architecture. However, most experiments with neural networks require a lot of time to search a suitable network and subsequently train the network. The authors have developed a deep multilayer perceptron (MLP) classifier with a zero-centered piecewise linear unit activation that yielded better classification performance according to their accuracy metric and required consistently less training time compared to a similar MLP network with leaky rectified linear unit and tanh activation functions. The authors illustrate their technique with a large high-frequency dataset on selected bank shares from the Indian stock market. The authors also discuss the theoretical properties and advantages of their proposal. <b>TOPICS:</b>Statistical methods, simulations, big data/machine learning <b>Key Findings</b> • Time-series models have been used extensively to understand stock price movements. They are constrained by a predefined structure. Artificial neural network (ANNs), due to their universal approximation properties, offer more flexibility; however, they suffer from long training times. • In this article, the authors introduce a zero-centered piecewise linear unit (PLU), a hybrid activation function derived from tanh and leaky rectified linear unit (ReLU) functions that exhibits the zero-centered property from tanh and the nonzero gradient property from leaky ReLU. • The authors developed a multilayer perceptron (MLP) classifier with PLU activation, which yielded better classification performance according to the accuracy metric and required consistently less training time compared to a similar MLP network with leaky ReLU and tanh activation.

Recently, Deep learning has made a great deal of success in processing images, audios, and natural languages and so on. The activation function is one of the key factors in Deep learning. In this paper, according to characteristics of biological neurons, an improved Leaky Single-Peaked Triangle Linear Unit (LSPTLU) activation function is presented for the right-hand response unbounded of Rectified Linear Unit (ReLU) and Leaky ReLU (LReLU). LSPTLU is more in line with the biological neuron essence and achieves the excellent performance of equivalent or beyond ReLU and LReLU on different datsets, e.g., MNIST, Fashion-MNIST, SVHN, IMAGENET, CALTECH101 and CIFAR10 datasets.

A convolutional neural network is employed to retrieve the time-domain envelop and phase of few-cycle femtosecond pulses from transient-grating frequency-resolved optical gating (TG-FROG) traces. We use theoretically generated TG-FROG traces to complete supervised trainings of the convolutional neural networks, then use similarly generated traces not included in the training dataset to test how well the networks are trained. Accurate retrieval of such traces by the neural network is realized. In our case, we find that networks with exponential linear unit (ELU) activation function perform better than those with leaky rectified linear unit (LRELU) and scaled exponential linear unit (SELU). Finally, the issues that need to be addressed for the retrieval of experimental data by this method are discussed.

Different parameters of a fully convolutional network (FCN) are experimented to evaluate which combination predicts sound velocity models from a single configuration of seismic modeling. The evaluation is made considering some fixed parameters of the deep learning model, such as number of epochs, batch size and loss function, but with variations of the optimizer and activation function. The considered optimizers were RMSprop, Adam and Adamax, whilst the activations functions were the Rectified Linear Unit (ReLU), Leaky ReLU, Exponential Linear Unit (ELU) and Parametric ReLU (PReLU). Five metrics were used to evaluate the model during the testing stage: R2, Pearson's r, factor of two, mean absolute error and mean squared error. To the extent of these experiments, it was found that the optimizers have much more influence than the activation functions when determining the resolution of the output model. The best combination was the one using the PReLU activation function with the Adamax optimizer.

Optical fringe patterns filtering based on multi-stage convolution neural network

Several deep neural networks have been introduced for finger vein recognition over time, and these networks have demonstrated high levels of performance. However, most current state-of-the-art deep learning systems use networks with increasing layers and parameters, resulting in greater computational costs and complexity. This can make them impractical for real-time implementation, particularly on embedded hardware. To address these challenges, this article concentrates on developing a lightweight convolutional neural network (CNN) named FV-EffResNet for finger vein recognition, aiming to find a balance between network size, speed, and accuracy. The key improvement lies in the utilization of the proposed novel convolution block named the Efficient Residual (EffRes) block, crafted to facilitate efficient feature extraction while minimizing the parameter count. The block decomposes the convolution process, employing pointwise and depthwise convolutions with a specific rectangular dimension realized in two layers (n × 1) and (1 × m) for enhanced handling of finger vein data. The approach achieves computational efficiency through a combination of squeeze units, depthwise convolution, and a pooling strategy. The hidden layers of the network use the Swish activation function, which has been shown to enhance performance compared to conventional functions like ReLU or Leaky ReLU. Furthermore, the article adopts cyclical learning rate techniques to expedite the training process of the proposed network. The effectiveness of the proposed pipeline is demonstrated through comprehensive experiments conducted on four benchmark databases, namely FV-USM, SDUMLA, MMCBNU_600, and NUPT-FV. The experimental results reveal that the EffRes block has a remarkable impact on finger vein recognition. The proposed FV-EffResNet achieves state-of-the-art performance in both identification and verification settings, leveraging the benefits of being lightweight and incurring low computational costs.

Over the past few decades, convolutional neural networks (CNNs) have achieved outstanding results in addressing a broad scope of computer vision problems. Despite these improvements, fully convolutional Siamese neural networks (FCSNN) still hardly adapt to complex scenes, such as appearance change, scale change, similar objects interference, etc. The present study focuses on an enhanced FCSNN based on convolutional block hyperparameters optimization, a new activation function (ModReLU) and Gaussian low pass filter. The optimization of hyperparameters is an important task, as it has a crucial ascendancy on the tracking process performance, especially when it comes to the initialization of weights and bias. They have to work efficiently with the following activation function layer. Inadequate initialization can result in vanishing or exploding gradients. In the first method, we propose an optimization strategy for initializing weights and bias in the convolutional block to ameliorate the learning of features so that each neuron learns as much as possible. Next, the activation function normalizes the output. We implement the convolutional block hyperparameters optimization by setting the convolutional weights initialization to constant, the bias initialization to zero and the Leaky ReLU activation function at the output. In the second method, we propose a new activation, ModReLU, in the activation layer of CNN. Additionally, we also introduce a Gaussian low pass filter to minimize image noise and improve the structures of images at distinct scales. Moreover, we add a pixel-domain-based color adjustment implementation to enhance the capacity of the proposed strategies. The proposed implementations handle better rotation, moving, occlusion and appearance change problems and improve tracking speed. Our experimental results clearly show a significant improvement in the overall performance compared to the original SiamFC tracker. The first proposed technique of this work surpasses the original fully convolutional Siamese networks (SiamFC) on the VOT 2016 dataset with an increase of 15.42% in precision, 16.79% in AUPC and 15.93% in IOU compared to the original SiamFC. Our second proposed technique also reveals remarkable advances over the original SiamFC with 18.07% precision increment, 17.01% AUPC improvement and an increase of 15.87% in IOU. We evaluate our methods on the Visual Object Tracking (VOT) Challenge 2016 dataset, and they both outperform the original SiamFC tracker performance and many other top performers.

Well-known activation functions like ReLU or Leaky ReLU are non-differentiable at the origin. Over the years, many smooth approximations of ReLU have been proposed using various smoothing techniques. We propose new smooth approximations of a non-differentiable activation function by convolving it with approximate identities. In particular, we present smooth approximations of Leaky ReLU and show that they outperform several well-known activation functions in various datasets and models. We call this function Smooth Activation Unit (SAU). Replacing ReLU by SAU, we get 5.63%, 2.95%, and 2.50% improvement with ShuffleNet V2 (2.0x), PreActResNet 50 and ResNet 50 models respectively on the CIFAR100 dataset and 2.31% improvement with ShuffleNet V2 (1.0x) model on ImageNet-1k dataset.KeywordsSmooth Activation FunctionNeural Network

Currently, we are dealing with a very limited set of activation functions such as Sigmoid, ReLu, Leaky ReLu among others. These activation functions used in the existing digital brain neural network systems are chosen using assumption with the help of “trial and error” approach. However, they do not ethically and appropriately establish any relationship with the referenced AI datasets. Jamilu (2019) proposed that a digital brain should have at least 2000 to 100 billion distinct activation functions implying distinct artificial neurons satisfies Jameel’s criterion(s) for it to normally mimic the human brain. The objectives of this paper are to propose a theorem called “Digital Brain Completeness Theorem”, “superintelligent digital brain neural network systems” and why it is tremendously important to have an extremely huge distinct activation functions implying distinct artificial neurons in a digital brain just like in the case of its counterpart for it to function rationally.

Convolutional neural networks (CNNs) have become a useful tool for a wide range of applications such as text classification. However, CNNs are not always sufficiently accurate to be useful in certain applications. The selection of activation functions within CNN architecture can affect the efficacy of the CNN. However, there is limited research regarding which activation functions are best for CNN text classification. This study tested sixteen activation functions across three text classification datasets and six CNN structures, to determine the effects of activation function on accuracy, iterations to convergence, and Positive Confidence Difference (PCD). PCD is a novel metric introduced to compare how activation functions affected a network’s classification confidence. Tables were presented to compare the performance of the activation functions across the different CNN architectures and datasets. Top performing activation functions across the different tests included the symmetrical multi-state activation function, sigmoid, penalised hyperbolic tangent, and generalised swish. An activation function’s PCD was the most consistent evaluation metric during activation function assessment, implying a close relationship between activation functions and network confidence that has yet to be explored.

Machine learning is a branch of artificial intelligence and computer science. The purpose of machine learning is to predict new data by using the existing data. In this study, two different machine learning methods which are Polynomial Regression (PR) and Artificial Neural Network (ANN) are applied to the neutron transport problems which are albedo problem, the Milne problem, and the criticality problem. ANN applications contain two different activation functions, Leaky Relu and Elu. The training data set is calculated by using the HN method. PR and ANN results are compared with the literature data. The study is only based on the existing data; therefore, the study could be thought only data mining on the one-speed neutron transport problems for isotropic scattering.