Fully Convolutional Siamese Fusion Networks for Object Tracking

Recently, the Siamese network based visual tracking methods have shown great potentials in balancing the tracking accuracy and computational efficiency. These methods use two-branch convolutional neural networks (CNNs) to generate a response map between the target exemplar and each of candidate patches in the search region. However, since these methods have not fully exploit the target-specific information contained in the CNN features during the computation of the response map, they are less effective to cope with target appearance variations and background clutters. In this paper, we propose a Target-Specific Response Attention (TSRA) module to enhance the discriminability of these methods. In TSRA, a channel-wise cross-correlation operation is used to produce a multi-channel response map, where different channels correspond to different semantic information. Then, TSRA uses an attention network to dynamically re-weight the multi-channel response map at every frame. Moreover, we introduce a shortcut connection strategy to generate a residual multi-channel response map for more discriminative tracking. Finally, we integrate the proposed TSRA into the classical Siamese based tracker (i.e., SiamFC) to propose a new tracker (called TSRA-Siam). Experimental results on three popular benchmark datasets show that the proposed TSRA-Siam outperforms the baseline tracker (i.e., SiamFC) by a large margin and obtains competitive performance compared with several state-of-the-art trackers.

The application of deep learning technology has increased rapidly in recent years. Technologies in deep learning increasingly emulate natural human abilities, such as knowledge learning, problem-solving, and decision-making. In general, deep learning can carry out self-training without repetitive programming by humans. Convolutional neural networks (CNNs) are deep learning algorithms commonly used in wide applications. CNN is often used for image classification, segmentation, object detection, video processing, natural language processing, and speech recognition. CNN has four layers: convolution layer, pooling layer, fully connected layer, and non-linear layer. The convolutional layer uses kernel filters to calculate the convolution of the input image by extracting the fundamental features. The pooling layer combines two successive convolutional layers. The third layer is the fully connected layer, commonly called the convolutional output layer. The activation function defines the output of a neural network, such as 'yes' or 'no'. The most common and popular CNN activation functions are Sigmoid, Tanh, ReLU, Leaky ReLU, Noisy ReLU, and Parametric Linear Units. The organization and function of the visual cortex greatly influence CNN architecture because it is designed to resemble the neuronal connections in the human brain. Some of the popular CNN architectures are LeNet, AlexNet and VGGNet.

Progressive kernel pruning with saliency mapping of input-output channels

Deep learning (DL) has present great diagnostic results in fault diagnosis field. However, the poor interpretability and noise robustness of DL-based methods are still the main factors limiting their wide application in industry. To address these issues, an interpretable wavelet packet kernel-constrained convolutional network (WPConvNet) is proposed for noise-robust fault diagnosis, which combines the feature extraction ability of wavelet bases and the learning ability of convolutional kernels together. First, the wavelet packet convolutional (WPConv) layer is proposed, and constraints are imposed to convolutional kernels, so that each convolution layer is a learnable discrete wavelet transform. Second, a soft threshold activation is proposed to reduce the noise component in feature maps, whose threshold is adaptively learned by estimating the standard deviation of noise. Third, we link the cascaded convolutional structure of convolutional neutral network (CNN) with wavelet packet decomposition and reconstruction using Mallat algorithm, which is interpretable in model architecture. Extensive experiments are carried out on two bearing fault datasets, and the results show that the proposed architecture outperforms other diagnosis models in terms of interpretability and noise robustness.

The convolutional neural networks (CNNs) have proven to be powerful classification tools in tasks that range from check reading to medical diagnosis, reaching close to human perception, and in some cases surpassing it. However, the problems to solve are becoming larger and more complex, which translates to larger CNNs, leading to longer training times that not even the adoption of Graphics Processing Units (GPUs) could keep up to. This problem is partially solved by using more processing units and distributed training methods that are offered by several frameworks dedicated to neural network training, such as Caffe, Torch, or TensorFlow. However, these techniques do not take full advantage of the possible parallelization offered by CNNs and the cooperative use of heterogeneous devices with different processing capabilities, clock speeds, memory size, among others. This paper presents a new method for the parallel training of CNNs where only the convolutional layer is distributed. The paper analyzes the influence of network size, bandwidth, batch size, number of devices, including their processing capabilities, and other parameters. Results show that this technique is capable of diminishing the training time without affecting the classification performance for both CPUs and GPUs. For the CIFAR-10 dataset, using a CNN with two convolutional layers, and and kernels, respectively, best speedups achieve using four CPUs and with three GPUs. Larger datasets will certainly require more than -% of processing time calculating convolutions, and speedups will tend to increase accordingly.

Scene classification is an important research branch of image comprehension, which gains information from images and interprets them using computer system by imitating the biological systems of human beings. AlexNet model is limited in image classification because of the large convolution kernel and stride in the first convolutional layer leading to over rapid decline of feature maps resolution and excessive compression of spatial information. This paper proposed an improved AlexNet model according to the design principle of convolutional neural networks (CNNs). The large convolution kernel is decomposed into a structure cascaded by two small convolution kernels with reduced stride. Another convolutional layer is added after the first one to enhance the integration process of the low-level features or the spatial information. The asymmetric convolution kernel is applied in the last three convolutional layers. The experiments on two datasets show that the classification accuracy of the improved AlexNet model is higher than those of AlexNet model and ZFNet model for 23 categories of scene classification.

In this study, deep convolutional neural networks (CNNs) are applied in computer-aided diagnosis of three types of disc herniation disease based on lumbar Axial MR Images. AlexNet architecture is explored and evaluated in order to classify images into four groups: Normal, Bulge, Protrusion and Extrusion. First, a large-scale dataset of 2329 scanned MRI images is gathered from local medical centers, then it is extended by using Visual Rotation and Mirror Reversal techniques to 9316 images. The CNNs are used for feature extraction and feature reduction in convolution layers; in addition, classification is performed by the CNN SoftMax layer. Also, to improve MR images resolution, CLAHE (Contrast-Limited Adaptive Histogram Equalization) and USM (UnSharp Masking) filters, which are of critical importance to the increase of the final accuracy, are employed. A region of interest (ROI) is then selected to reduce the size of the input images and eliminate additional features causing overhead on network. A number of changes are made to original AlexNet model including adding a Batch Normalization layer, as well as initial Xavier initialization, with remarkable effectiveness displayed in results. Also, instead of fully-connected layers, 1×1 convolution layer is applied to test accuracy. Finally, the proposed method is compared with the results of three state-of-the-art methods. Experimental results prove that this deep CNN results enjoy an improvement compared to former proposed methods; the accuracy, sensitivity and specificity for random sub sampling method are 87.75%, 86.5% and 94.75%, respectively, achieved by AlexNet architecture of CNN.

Convolutional neural network (CNN) consists of a few pairs of both convolution layer and subsampling layer. Thus it has more hidden layers than multi-layer perceptron. With the increased layers, the size of convolution mask ultimately determines the total number of weights in CNN because the mask is shared among input images. It also is an important learning factor which makes or breaks CNNs learning. Therefore, this paper proposes the best method to choose the convolution size and the number of layers for learning CNN successfully. Through our face recognition with vast learning examples, we found that the best size of convolution mask is 5 by 5 and 7 by 7, regardless of the number of layers. In addition, the CNN with two pairs of both convolution and subsampling layer is found to make the best performance as if the multi-layer perceptron having two hidden layers does.

Convolutional neural network (CNN) is the primary technique that has greatly promoted the development of computer vision technologies. However, there is little research on how to allocate parameters in different convolution layers when designing CNNs. We research mainly on revealing the relationship between CNN parameter distribution, i.e., the allocation of parameters in convolution layers, and the discriminative performance of CNN. Unlike previous works, we do not append more elements into the network, such as more convolution layers or denser short connections. We focus on enhancing the discriminative performance of CNN through varying its parameter distribution under strict size constraint. We propose an energy function to represent the CNN parameter distribution, which establishes the connection between the allocation of parameters and the discriminative performance of CNN. Extensive experiments with shallow CNNs on three public image classification data sets demonstrate that the CNN parameter distribution with a higher energy value will promote the model to obtain better performance. According to the motivated observation, the problem of finding the optimal parameter distribution can be transformed into an optimization problem of finding the biggest energy value. We present a simple yet effective guideline that uses balanced parameter distribution to design CNNs. Extensive experiments on ImageNet with three popular backbones, i.e., AlexNet, ResNet34, and ResNet101, demonstrate that the proposed guideline can make consistent improvements upon different baselines under strict size constraint.

Several economists and social scientists have held a longstanding fascination with the practice of stock market prediction. As the stock market is essentially uncontrollable chaos, many experts believe that trying to predict it is futile. Due to the complexity of the numerous factors, accurate stock price predictions are notoriously difficult to achieve. While the market behaves more like a scale than a voting machine over the long run, its behavior may be predicted with some certainty. Information from Twitter is used into the algorithm. In this proposed method, a convolutional extreme learning machine model with kernel support was introduced (CKELM). To improve feature extraction and data classification, the CKELM model builds on the KELM's hidden layer by adding convolutional and subsampling layers. The convolutional layer and the subsampling layer do not employ the gradient technique to fine-tune their parameters because some designs worked well with random weights. When compared to popular models like CNN and KELM, The proposed model fares quite well, with an accuracy of around 98.3 percent.

We propose a learning method to generate synthetic CT (sCT) image for MRI-only radiation treatment planning. The proposed method integrated a dense-block concept into a cycle-generative adversarial network (cycle-GAN) framework, which is named as dense-cycle-GAN in this study. Compared with GAN, the cycle-GAN includes an inverse transformation between CT (ground truth) and sCT, which could further constrain the learning model. A 2.5D fully convolution neural network (FCN) with dense-block was introduced in generator to enable end-to-end transformation. A FCN is used in discriminator to urge the generator’s sCT to be similar with the ground-truth CT images. The well-trained model was used to generate the sCT of a new MRI. This proposed algorithm was evaluated using 14 patients’ data with both MRI and CT images. The mean absolute error (MAE), peak signal-to-noise ratio (PSNR) and normalized cross correlation (NCC) indexes were used to quantify the correction accuracy of the prediction algorithm. Overall, the MAE, PSNR and NCC were 60.9−11.7 HU, 24.6±0.9 dB, and 0.96±0.01. We have developed a novel deep learning-based method to generate sCT with a high accuracy. The proposed method makes the sCT comparable to that of the planning CT. With further evaluation and clinical implementation, this method could be a useful tool for MRI-based radiation treatment planning and attenuation correction in a PET/MRI scanner.

Convolutional neural networks (CNNs) are widely adopted in various applications. State-of-the-art CNN models deliver excellent classification performance, but they require a large amount of computation and data exchange because they typically employ many processing layers. Among these processing layers, convolution layers, which carry out many multiplications and additions, account for a major portion of computation and memory access. Therefore, reducing the amount of computation and memory access is the key for high-performance CNNs. In this study, we propose a cost-effective neural network accelerator, named CENNA, whose hardware cost is reduced by employing a cost-centric matrix multiplication that employs both Strassen’s multiplication and a naïve multiplication. Furthermore, the convolution method using the proposed matrix multiplication can minimize data movement by reusing both the feature map and the convolution kernel without any additional control logic. In terms of throughput, power consumption, and silicon area, the efficiency of CENNA is up to 88 times higher than that of conventional designs for the CNN inference.

Sound-event recognition often utilizes time-frequency analysis to produce an image-like spectrogram that provides a rich visual representation of original signal in time and frequency. Convolutional Neural Networks (CNN) with the ability of learning discriminative spectrogram patterns are suitable for sound-event recognition. However, there is relatively little effort that CNN makes full use of the important temporal information. In this paper, we propose MCRNN, a Convolutional Recurrent Neural Networks (CRNN) architecture for sound-event recognition, the letter “M” in the name “MCRNN” of our model denotes the multi-sized convolution filters. Richer features are extracted by using several different convolution filter sizes at the last convolution layer. In addition, cochleagram images are used as the input layer of the network, instead of the traditional spectrogram image of a sound signal. Experiments on the RWCP dataset shows that the recognition rate of the proposed method achieved 98.4% in clean conditions, and it robustly outperforms the existing methods, the recognition rate increased by 0.9%, 1.9% and 10.3% in 20 dB, 10 dB and 0 dB signal-to-noise ratios (SNR), respectively.

Introduction: Recent studies have acknowledged the potential of convolutional neural networks (CNNs) in distinguishing healthy and morbid samples by using heart sound analyses. Unfortunately the performance of CNNs is highly dependent on the filtering procedure which is applied to signal in their convolutional layer. The present study aimed to address this problem by applying filter bank learning concept in CNNs. Material and Methods: In proposed method, the filter bank of CNN is updated based on a cross-entropy minimization rule to extract higher-level features from spectral characteristics of the heart sound signal. The deeper level of the extracted features in parallel with their spectral-based nature leads to better discrimination between healthy and morbid heart sounds. The proposed method was applied to three different heart sound datasets of PASCAL-A, PASCAL-B, and Kaggle, including normal and abnormal categories. Results: The proposed method obtained a true positive rate (TPR) between minimally 86% and maximally 96% (if FPR=0%) among all the examined datasets. In addition, the false-positive rate (FPR) was obtained as 7-8% (if TPR=100%) among the mentioned datasets. Finally, the accuracy was achieved in the range of 93-98% when the FPR was 0% and within the range of 96-96.5% when the TRP was 100%. Conclusion: Increased TPR in the proposed method (96% for the proposed method vs. 87% for CNN) in parallel with a decrease in its FPR (7% for the proposed method vs. 10% for CNN) showed the proposed method's superiority against its well-known alternative in automated self-assessment of the heart.

Recently, Template matching approach has been widely used to locate faces with various pose, illumination and clutter background. Normalized Cross-correlation (NCC) is an effective and simple measurement method to compute the similarity matching between the stored faces templates and the rectangular blocks of the input image to locate the face position. However, localization error occurs very often due to some rectangular blocks which have more face than correct blocks because of the effect of matrices values of these blocks. In this paper we proposed a simple preprocessing method before the use of NCC. This is to reduce the effects of such problems by increasing the values of the input image pixels. The result showed a significant improvement in localization accuracy compared with the use of NCC alone which is only up to 11%. Yale University database was used to evaluate our proposed method.