Enhancement: SiamFC Tracker Algorithm Performance Based on Convolutional Hyperparameters Optimization and Low Pass Filter

Convolutional neural networks (CNNs) have become a useful tool for a wide range of applications such as text classification. However, CNNs are not always sufficiently accurate to be useful in certain applications. The selection of activation functions within CNN architecture can affect the efficacy of the CNN. However, there is limited research regarding which activation functions are best for CNN text classification. This study tested sixteen activation functions across three text classification datasets and six CNN structures, to determine the effects of activation function on accuracy, iterations to convergence, and Positive Confidence Difference (PCD). PCD is a novel metric introduced to compare how activation functions affected a network’s classification confidence. Tables were presented to compare the performance of the activation functions across the different CNN architectures and datasets. Top performing activation functions across the different tests included the symmetrical multi-state activation function, sigmoid, penalised hyperbolic tangent, and generalised swish. An activation function’s PCD was the most consistent evaluation metric during activation function assessment, implying a close relationship between activation functions and network confidence that has yet to be explored.

Convolutional Neural Network (CNN) is a recently used popular machine learning technique to classify images. However, choosing an optimum and efficient architecture is an inevitable challenge. The research goal was to implement CNN on face classification from low quality CCTV footage. The best model was gained from the hyperparameter optimization process used on CNN structure. The optimized hyperparameters were those connected to the structure network including activation function, the number of kernel, the size of kernel, and the number of nodes on the fully connected layers. Hyperparameter optimization strategy used was random grid coarse-to-fine search optimization approach. This approach combined random search, grid search, and coarse-to-fine technique that was easily and efficiently applied, yet worked well. Exhaustive-random search process was done by evaluating all selected activation functions and choosing another hyperparameters randomly. This was based on the assumption that activation functions were the most related hyperparameter to the model. The SELU activation function used in the next step was the one with the best average performance. Grid coarse-to-fine was conducted to optimize the number of kernel and the number of node on fully connected layer, while grid search was conducted to optimize the kernel size. This process aimed to locate optimal value gradually in hyperparameter which had high-dimensional space. Evaluation of the model resulted from the optimum hyperparameter was 97,56%.

Empirical-statistical Downscaling (SD) techniques are valuable tools able to generate high-resolution climate information needed to carry out impact studies. In this regard, Convolutional Neural Networks (CNNs) are promising SD techniques capable of handling large amounts of data and extracting relevant predictor information for each particular site. These characteristics of the CNN represent a major advantage over traditional SD methods, which typically rely on human-guided predictor selection. Notwithstanding, an adequate tuning of the CNN is key for optimising their potential.In southern South America (SSA), CNNs has proven to be skilful in representing daily extreme temperatures and extrapolating into future scenarios. Although the selection of the activation function introduces a source of uncertainty in the future projections. In this context, this study aims to explore the use of CNNs as a statistical downscaling tool to simulate the wet bulb temperature (Tw) over SSA, a multivariate heat-stress index estimated from temperature and humidity. Tw has been widely used as a heat-stress proxy in different parts of the world, however, its characterisation and modelling in SSA remain as a pending task. To this end, four different CNN architectures regarding the activation function (ReLU or linear), domain size and configuration of the CNN layers were tested. All CNN models were trained during summer days using a cross-validation (CV) scheme in the period 1991-2020 and then evaluated in four unseen summers between 2021 and 2024. For comparison purposes, CNN models were similarly trained and validated to simulate maximum temperature (Tx). Overall, CNN models well represented all the features evaluated, including the heat-waves that took place in the summers evaluated independently. In particular, CNN models presents a better performance in simulating Tw than Tx with smaller errors in terms of mean and extremes aspects. Regarding the domain size, for both temperatures, the configuration with the smaller domain yields the best results. Also in this latter case, the reduction of the number of filter size in the last layer slightly improves the representation of Tx. When considering the large domain, the differences between the CNNs based on different activation functions increase, and CNN models with linear configuration outperform the ones with ReLu. The findings of this work reinforces the potential of CNNs for climate downscaling in SSA, especially for its use to simulate multivariate impact indices.

Correlation tracker has made a huge success in visual object tracking. However, it is mainly because that the tracker cannot catch the occurrence of appearance changes, tracking based on correlation filters often drifts due to the unexpected appearance changes caused by occlusion, deformation and background clutter. In this paper, we propose a new method to detect the case when the tracker encountered the unexpected appearance changes. This method uses the following points: 1) Filter response curve would decreases dramatically when target suffers heavy appearance changes. 2) Features extracted from deeper layers of convolutional neural networks (CNNs) have more semantics information and features extracted from shadower layers have more spatial information. Extensive experimental results on several public benchmark datasets show that the proposed method can deal with the appearance changes effectively.

With the rapid development of artificial intelligence, deep learning is developing rapidly. As the main framework of in-depth learning, convolution neural network achieved much in semantic segmentation and image classification. Based on the low speed engineering vehicle tracking system project, this paper studies the activation function, which is an important component of convolution neural network. Firstly, it summarizes the advantages and disadvantages of four traditional activation functions such as Sigmoid, and then introduces three new activation functions: Swish, FTS, and Relu-Softplus. Finally, using the TensorFlow framework, a single new activation function and a hybrid new activation function are applied to the convolution layer of the four neural networks constructed, and the image recognition comparison is performed in the data set CIFAR-10. It is concluded that the recognition precision of mixing the new activation functions is higher than that of using a new activation function alone, and the average time is shorter and the convergence speed is faster.

The convolutional neural network (CNN) has been widely used in image recognition field due to its good performance. This paper proposes a facial expression recognition method based on the CNN model. Regarding the complexity of the hierarchic structure of the CNN model, the activation function is its core, because the nonlinear ability of the activation function really makes the deep neural network have authentic artificial intelligence. Among common activation functions, the ReLu function is one of the best of them, but it also has some shortcomings. Since the derivative of the ReLu function is always zero when the input value is negative, it is likely to appear as the phenomenon of neuronal necrosis. In order to solve the above problem, the influence of the activation function in the CNN model is studied in this paper. According to the design principle of the activation function in CNN model, a new piecewise activation function is proposed. Five common activation functions (i.e., sigmoid, tanh, ReLu, leaky ReLus and softplus–ReLu, plus the new activation function) have been analysed and compared in facial expression recognition tasks based on the Keras framework. The Experimental results on two public facial expression databases (i.e., JAFFE and FER2013) show that the convolutional neural network based on the improved activation function has a better performance than most-of-the-art activation functions.

Developing computational algorithms to model the biological vision system has challenged researchers in the computer vision field for several decades. As a result, state-of-the-art algorithms such as the Convolutional Neural Network (CNN) have emerged for image classification and recognition tasks with promising results. CNNs however remain view-specific, producing good results when the variation between test and train data is small. Making CNNs learn invariant features to effectively recognise objects that undergo appearance changes as a result of transformations such as scaling remains a technical challenge. Recent physiological studies of the visual system are suggesting new paradigms. Firstly, our visual system uses both local features and global features in its recognition function. Secondly, cells tuned to global features respond quickly to visual stimuli for recognising objects. Thirdly, information from modalities that handle local features, global features and color are integrated in the brain for performing recognition tasks. While CNNs rely on aggregation of local features for recognition, these theories provide the potential for using global features to solve transformation invariance problems in CNNs. In this paper we realise these paradigms into a computational model, named as global features improved CNN (GCNN), and test it on classification of scaled images. We experiment combining Histogram of Gradients (HOG) global features, CNN local features and color information and test our technique on benchmark data sets. Our results show GCNN outperforms traditional CNN on classification of scaled images indicating potential effectiveness of our model towards improving scale-invariance in CNN based networks.

Convolutional neural networks are known for their excellent performance in computer vision, achieving results in the state of the art. Moreover, recent research has shown that these networks can also provide promising results for natural language processing. In this case, the basic idea is to concatenate the vector representations of words into a single block and use it as an image. However, despite the good results, the problem of using convolution networks is the large numbers of design decisions that need to be made a priori. These models require the definition of many hyper-parameters, including the type of word embeddings, which consists of the data vectorized representation, the activation function that prints the non-linearity characteristics to the model, the size of the filter that applies data convolution, the number of feature maps, which are responsible for identifying the attributes and the pooling method used for data reduction. In addition, one must also predefine the regularization constant and the dropout rate, which are responsible for avoiding any network over-fitting. In existing research works, convolutional neural network architectures capable of overcoming the performance of traditional machine learning models are presented. Even though these can compete with more complex models, the problem of how the different setting of the hyper-parameters may affect the performance of this type of network has not yet been explored. In this paper, we propose an efficient sentiment analysis classifier using convolutional neural networks by analyzing the impact of the hyper-parameters on the model performance. The main interest in analyzing sentiment comes from the advent of social media and the technological advances that flood the Internet with opinions. Nonetheless, mining the Internet for opinion and sentiment analysis is not an easy task and thus needs outstanding models with the best hyper-parameters setting to be able to get pertinent answers. The results achieved are obtained with the use of GPU and show that the different configurations exceed the reference models in the most of the cases with gains of up to 18% and have similar performance to the models of the state of the art with gains of up to 2% in some cases.

Detection of brain tumor is very challenging task in today’s medical world. Nowadays, many doctors prefer ready-made methods to diagnose the tumor from given magnetic resonance imaging (MRI) images. Accurate and less time-consuming methods are used frequently to identify it. This paper introduces new techniques for finding and differentiating brain tumors. The new methods include support vector machine (SVM) and convolution neural network (CNN) with Softmax classifier. The SVM uses histogram of oriented gradients (HOGs) as a feature to label the given dataset. Further, the same feature of the test image is verified against labeled sets and classified. The SVM gives accuracy of 87.33%. In CNN, multilayer structure is used. It contains a convolution layer followed by the pooling layer and finally fully connected layer. In CNN, specifically, Alexnet is preferred for classification. In Alexnet, rectified linear unit (ReLU) is used as an activation function. At the output of it, Softmax is used as activation to classify the images. The CNN with Softmax classifier gives improved accuracy of 98.68%. In addition, the performance measures are also found out such as specificity, sensitivity, and precision. The sensitivity with SVM is 93.18%, and with CNN, it increases up to 100%. Comparing both methods with the obtained results, the CNN with Softmax classifier gives better results. Proposed method with CNN produces only one misclassified image as an advantage over the existing method, while SVM gives nine misclassified images. The methods are tested on 253 MRI images which are detected and classified correctly with Matlab.KeywordsSVMCNNHOGMRI

Artificial intelligence (AI) has become a cornerstone of modern technology, revolutionizing industries from healthcare to finance. Convolutional neural networks (CNNs) are a subset of AI that have emerged as a powerful tool for various tasks including image recognition, speech recognition, natural language processing (NLP), and even in the field of genomics, where they have been utilized to classify DNA sequences. This paper provides a comprehensive overview of CNNs and their applications in image recognition tasks. It first introduces the fundamentals of CNNs, including the layers of CNNs, convolution operation (Conv_Op), Feat_Maps, activation functions (Activ_Func), and training methods. It then discusses several popular CNN architectures such as LeNet, AlexNet, VGG, ResNet, and InceptionNet, and compares their performance. It also examines when to use CNNs, their advantages and limitations, and provides recommendations for developers and data scientists, including preprocessing the data, choosing appropriate hyperparameters (Hyper_Param), and evaluating model performance. It further explores the existing platforms and libraries for CNNs such as TensorFlow, Keras, PyTorch, Caffe, and MXNet, and compares their features and functionalities. Moreover, it estimates the cost of using CNNs and discusses potential cost-saving strategies. Finally, it reviews recent developments in CNNs, including attention mechanisms, capsule networks, transfer learning, adversarial training, quantization and compression, and enhancing the reliability and efficiency of CNNs through formal methods. The paper is concluded by summarizing the key takeaways and discussing the future directions of CNN research and development.

Towards an effective model for lung disease classification: Using Dense Capsule Nets for early classification of lung diseases

SummaryThe present existing weather prediction systems deployed by weather forecasting agencies predict the anomalous events using numerical simulation models developed based on geophysics and oceanographic theories. The major disadvantage with existing models is that are explicit simulation models. So, we try to build a real time prediction system that will automatically predict the intensity of any marine natural disaster events and alert the authorities to take necessary actions using a satellite and observational real time data set including moored met ocean data. We propose a novel hybrid model which is a combination of Convolutional Neural Network (CNN) and Long Short‐Term Memory (LSTM). The paper analyzes various other models such as Gated Recurrent Unit (GRU), Bidirectional LSTM (Bi‐LSTM) Recurrent Neural Network (RNN), and Extreme Learning Machine. In addition the other standard CNN architectures are also analyzed. Since the residual CNN network is the most effective architecture, the paper proposes a modification of the residual CNN network by combining the residual CNN network with LSTM/GRU RNN network along with a proposed new activation function which was experimentally proved to be more effective than the deep learning community standard activation function ReLU. The proposed new activation function overcomes the dead neuron propagation problem in ReLU activation function and hence the model's accuracy is increased by more than 1%. The proposed CNN + LSTM/GRU model is benchmarked with UCR time series benchmark dataset and it is experimentally seen that the model performs better than all existing algorithms.

Deep convolutional neural network (DCNN) is a powerful model for learning significant data at multiple levels of abstraction form an input image. However, training DCNN is often complicated because of parameter initialization, overfitting and convergence problems. Hence this work has been targeted to overcome the challenges of training DCNN with an optimized model. This chapter describes a deep learning framework for image classification with cifar-10 dataset. The model contains a set of convolutional layers with rectified linear unit activation function, max-pooling layers, and a fully-connected layer with softmax activation function. This model learns the features automatically and classifies the image without using the hand-crafted image based features. In this investigation, various optimizers have been applied in gradient descent technique for minimizing the loss function. Model with Adam optimizer constantly minimizes the objective function compared with other standard optimizers such as momentum, Rmsprop, and Adadelta. Dropout and batch normalization techniques are adapted to improve the model performance further by avoiding overfitting. Dropout function deactivates the insignificant node form the model after every epoch. The initialization of a large number of parameters in DCNN is regularized by batch normalization. Results obtained from the proposed model shows that batch normalization with dropout significantly improves the accuracy of the model with the tradeoff of computational complexity.

The application of deep learning technology has increased rapidly in recent years. Technologies in deep learning increasingly emulate natural human abilities, such as knowledge learning, problem-solving, and decision-making. In general, deep learning can carry out self-training without repetitive programming by humans. Convolutional neural networks (CNNs) are deep learning algorithms commonly used in wide applications. CNN is often used for image classification, segmentation, object detection, video processing, natural language processing, and speech recognition. CNN has four layers: convolution layer, pooling layer, fully connected layer, and non-linear layer. The convolutional layer uses kernel filters to calculate the convolution of the input image by extracting the fundamental features. The pooling layer combines two successive convolutional layers. The third layer is the fully connected layer, commonly called the convolutional output layer. The activation function defines the output of a neural network, such as 'yes' or 'no'. The most common and popular CNN activation functions are Sigmoid, Tanh, ReLU, Leaky ReLU, Noisy ReLU, and Parametric Linear Units. The organization and function of the visual cortex greatly influence CNN architecture because it is designed to resemble the neuronal connections in the human brain. Some of the popular CNN architectures are LeNet, AlexNet and VGGNet.

Sentiment Analysis was a process for identifying whether a source of text contains certain opinions, emotions, and polarity. Twitter Sentiment Analysis was a process for identifying sentiment and polarity on tweet. Twitter Sentiment Analysis provided a way for did survey about public sentiment to product, or particular even through collections of tweet. Main problem in sentiment identifying was how to determine classification model that gave high accuracy to classifying sentiment of tweet. One of the method for classifying sentiment of tweet was Deep Learning. Convolutional Neural Network (CNN) was special type of architecture from Deep Learning that its architecture had convolution layer. Convolution layer was important for extract relevant feature from text for classifying sentiment. The objective of this research was for found out the best CNN model for classifying sentiment of tweet. By using a dataset of tweets about public opinion on the Smartfren 4G network service, we searched the best CNN model using 6 combination parameters, that is the computational eficiency method, window size, and dimension of word embedding for parameters in Word2Vec Skip-gram model, then activation function in convolution layer, dropout rate, and pool size in pooling layer for parameters in CNN. The test is done using 10-fold cross validation for each search for the best parameter value and produced the best CNN model with an accuracy value of 88,21%.