Disentangled convolution for optimizing receptive field

Deep convolutional neural networks (CNNs) have shown their great success on image classification. CNNs mainly consist of convolutional and pooling layers, both of which are performed on local image areas without considering the dependence among different image regions. However, such dependence is very important for generating explicit image representation. In contrast, recurrent neural networks (RNNs) are well known for their ability of encoding contextual information in sequential data, and they only require a limited number of network parameters. Thus, we proposed the hierarchical RNNs (HRNNs) to encode the contextual dependence in image representation. In HRNNs, each RNN layer focuses on modeling spatial dependence among image regions from the same scale but different locations. While the cross RNN scale connections target on modeling scale dependencies among regions from the same location but different scales. Specifically, we propose two RNN models: 1) hierarchical simple recurrent network (HSRN), which is fast and has low computational cost and 2) hierarchical long-short term memory recurrent network, which performs better than HSRN with the price of higher computational cost. In this paper, we integrate CNNs with HRNNs, and develop end-to-end convolutional hierarchical RNNs (C-HRNNs) for image classification. C-HRNNs not only utilize the discriminative representation power of CNNs, but also utilize the contextual dependence learning ability of our HRNNs. On four of the most challenging object/scene image classification benchmarks, our C-HRNNs achieve the state-of-the-art results on Places 205, SUN 397, and MIT indoor, and the competitive results on ILSVRC 2012.

Deep (machine) learning in recent years has significantly increased the predictive modeling strengths for applications in the areas of computer vision, speech recognition, and natural language processing. Being heavily data-driven, conventional general-purpose deep learning methods mainly rely on training data to learn task-specific data representations. In this thesis, we focus on leveraging novel learning algorithms in deep learning to tackle visual recognition challenges at the three key levels -- namely pixel, object, and image levels. Throughout the thesis, we demonstrate how each and every of our proposed methods can address a specific visual recognition problem. Firstly, we apply deep learning to improve the saliency detection performance of deep networks. A coarse saliency map of the input image's salient object is initially predicted by a convolutional-deconvolutional network (or equivalently a fully convolutional network). Following that, the coarse saliency map is sequentially refined by a separate network called Recurrent Attentional Convolutional-Deconvolutional Network (RACDNN) that shares its network weights across its sequential steps. At every sequential (temporal) step, the same RACDNN is applied to a newly selected/attended image sub-region and it performs saliency map refinement for that particular sub-region. Despite having many learnable network parameters and no access to additional training data, RADCNN can be trained well and it outperforms the "single-step" (coarse saliency detection) baseline and state-of-the-art saliency detection methods by large margins, on several saliency detection benchmark datasets. Secondly, we introduce auto-encoder weight transfer networks (AE-WTN+), a deep model for scaling up object detection. Conventional WTN transfers object class knowledge from the classification weights of a pretrained large-scale image classification network, to the classification weights of an object detection network. However, they suffer from under-fitting and are not able to preserve the rich object class information of novel classes. In this work, we propose a new WTN method -- AE-WTN+ which makes use of normalization techniques to ease network optimization. Inputs and intermediate features are normalized to improve trainability for reduced under-fitting. More importantly, a novel auto-encoding (reconstruction) loss is proposed to encourage information preservation of all classes in the weight predictions of AE-WTN+. This enables the predicted weights of AE-WTN+ to maintain the class discriminability and neighbourhood relationships contained in the pre-trained large-scale classification weights. Experiments on large-scale detection datasets validate the effectiveness of AE-WTN+ on both seen and novel classes. Thirdly, we improve the efficiency of cross-layer network connections for very deep convolutional network in the context of image classification. This gives rise to a new convolutional network architecture called Deluge Networks (DelugeNets) which pass the output (feature activations) of any preceding composite network layer as direct inputs to all succeeding composite layers. For the incoming feature activations of preceding composite layers, the succeeding composite layers first apply rather lightweight cross-layer depthwise convolutional weights on them for feature aggregation. The heavy lifting is carried out by the last convolutional weight layer in each preceding composite layer that performs cross-channel convolutions. The proposed technique enables unimpeded information flow among many layers in very deep DelugeNets with great efficiency. Experiments on small-scale and large-scale datasets show that DelugeNets perform competitively to state-of-the-art deep convolutional networks at lower computational and parameter costs. Fourthly, we propose Stochastic Downsampling (SDPoint), a method to overcome a major limitation suffered by conventional deep convolutional networks -- their computational costs during inference are fixed and cannot be changed. SDPoint downsamples the feature maps at a randomly selected point (layer index) in the network hierarchy during training. The many downsampling configurations known as SDPoint instances each entail a unique downsampling point and downsampling ratio. The same convolutional network weights are shared across the different SDPoint instances that downsample feature maps differently. During inference, the computational cost of a network trained with SDPoint can be instantaneously adjusted to fit a given computational budget, by manually selecting an appropriate SDPoint instance. Additionally, sharing weights between different feature map scales provides significant regularization benefits, making the networks trained with SDPoint less sensitive to scales and more competent. In image classification experiments, SDPoint demonstrates significant cost-accuracy and computational advantages, over independently-trained (non-weight-shared) baselines with various inference costs.

In this study, a method of predicting the seismic responses of building structures based on a convolutional neural network (CNN) is proposed. In the method, the time histories of acceleration responses previously measured in a building during earthquakes are used in the CNN input layer, with the corresponding time histories of the displacement responses being used in the CNN output layer. The correlations between the features automatically extracted from the acceleration responses by the convolution and pooling operations in the various CNN layers and the displacement responses in the CNN output layer are established through CNN training. The trained CNN predicts the displacement responses for a future earthquake event using only the measured acceleration responses. To verify the validity of the proposed method, a numerical study on the ASCE benchmark model and an experimental study on a reinforced concrete frame structure are conducted. In the numerical study, the structural responses of the ASCE model subject to artificial earthquakes are utilized for CNN training, and the performance of the trained CNN is verified through seismic response prediction. In the experimental study, shaking table tests are performed for various earthquakes to measure the seismic responses of the reinforced concrete test structure. The seismic response prediction performance of the proposed method is examined using CNN trained with the measured seismic responses. Furthermore, the performances of CNNs trained with different numbers of datasets generated by the proposed data generation method based on data overlapping with the same data pool are discussed with related to the number of CNN training iteration number.

As one of the indispensable components in convolutional neural network (CNN), loss function assists in updating parameters of CNN models during the training phase. Generally, different loss functions can assist convolutional neural network (CNN) to learn different feature representations, and different feature representations can be treated as different knowledge learned from objects. In this paper we introduce a novel training framework, namely Loss Transferring (LT), to improve the generalization ability of CNN. LT contains multiple training phases, and each training phase uses a different loss function. Under this framework, CNN models can combine different knowledge of objects by transferring the knowledge learned via one loss function to another. LT contains two components, i.e., loss function set and training strategy. In order to build appropriate loss function set, we establish two basic guides. And according to these basic guides, we design a new loss function in the last layer of CNN models (layer before softmax operation), namely Near Classifier Hyper-Plane (N-CHP) loss, which makes the learned object features belonging to the same category have the minimum intra-class distance and be near the classifier hyper-plane. Based on the two loss function set $\{\text {MSE, softmax}\}$ and $\{\text {N-CHP, softmax}\}$ , we setup two specific training methods, $LT_{\text {MSE, softmax}}$ and $LT_{\text {N-CHP, softmax}}$ , which can be universally applied to different CNN models with low additional computation cost. Meanwhile, two training strategies, multi-phase strategy 1 and multi-phase strategy 2, are further proposed to improve the training efficiency of LT. Extensive experimental results on shallow, moderate and deep models with four benchmark datasets, including MNIST, SVHN, CIFAR-10 and CIFAR-100, demonstrate that CNN models can bring obvious performance improvements when working with $LT_{\text {MSE, softmax}}$ and $LT_{\text {N-CHP, softmax}}$ , which verifies the effectiveness of LT and the proposed two basic guides.

Convolutional Neural Networks (CNNs) have recently been utilized in a wide range of applications, including image and pattern identification, voice recognition, biometric embedded vision, food recognition, and video analysis for surveillance, industrial robots, and autonomous vehicles. Convolutional neural networks (CNNs) are gaining popularity for a variety of reasons. Feature extractors are created by hand in classic image recognition models. The weights of the convolutional layer utilized for feature extraction, as well as the fully connected layer, are used for classification in CNNs, and these weights are set throughout the training phase. The goal of this study is to evaluate a few convolutional neural network (CNN) learning machine approaches for image recognition. Furthermore, machine learning algorithms are used extensively in contemporary approaches to picture recognition. This article focuses on the development trend of convolution neural network (CNNs) model owing to various learning methods in image recognition during the 2000s, which is largely introduced from the angles of capturing, verification, and clustering, based on twenty five journals that have been reviewed. As a result, deep convolutional neural networks (DCNNs) have been very effective in a variety of machine learning and computer vision problems because they provide a large quality gain at a low computational cost. This training approach also enables models with many processing layers to learn different degrees of abstraction for data representation.

Computational machine learning has attracted a great deal of attention for its ability to analyze large-scale data. In particular, convolutional neural networks (CNNs) have been proposed in the fields of image recognition and object detection in efforts to develop models with improved accuracy as well as more lightweight models that require a smaller number of parameters and a lower computational cost. Octave Convolution (OctConv) is a method used to reduce the memory and computational cost of a model while also improving its accuracy by replacing the conventional convolutional layer with an OctConv layer. However, the number of parameters used in OctConv is almost the same as that in the case of conventional convolutional processing. In this paper, we propose the Pointwise Octave Convolution (Pointwise OctConv) method, which combines the Pointwise Convolution (Pointwise Conv) method with OctConv to reduce the number of parameters used in OctConv and thus create a lighter model. In the proposed method, the number of parameters is reduced by performing Pointwise Conv before and after the convolution process for each path in the OctConv layer. In an evaluation using ResNet-56, the proposed method reduces the number of parameters by about 63.8\(\%\) with a loss of classification accuracy of 3.04\(\%\) when \(\alpha = 0.75\).

Recently, deep learning (DL) became one of the essential tools in bioinformatics. A modified convolutional neural network (CNN) is employed in this paper for building an integrated model for deoxyribonucleic acid (DNA) classification. In any CNN model, convolutional layers are used to extract features followed by max-pooling layers to reduce the dimensionality of features. A novel method based on downsampling and CNNs is introduced for feature reduction. The downsampling is an improved form of the existing pooling layer to obtain better classification accuracy. The two-dimensional discrete transform (2D DT) and two-dimensional random projection (2D RP) methods are applied for downsampling. They convert the high-dimensional data to low-dimensional data and transform the data to the most significant feature vectors. However, there are parameters which directly affect how a CNN model is trained. In this paper, some issues concerned with the training of CNNs have been handled. The CNNs are examined by changing some hyperparameters such as the learning rate, size of minibatch, and the number of epochs. Training and assessment of the performance of CNNs are carried out on 16S rRNA bacterial sequences. Simulation results indicate that the utilization of a CNN based on wavelet subsampling yields the best trade-off between processing time and accuracy with a learning rate equal to 0.0001, a size of minibatch equal to 64, and a number of epochs equal to 20.

In this paper, we study the performance of variants of well-known Convolutional Neural Network (CNN) architectures on different audio tasks. We show that tuning the Receptive Field (RF) of CNNs is crucial to their generalization. An insufficient RF limits the CNN's ability to fit the training data. In contrast, CNNs with an excessive RF tend to over-fit the training data and fail to generalize to unseen testing data. As state-of-the-art CNN architectures - in computer vision and other domains - tend to go deeper in terms of number of layers, their RF size increases and therefore they degrade in performance in several audio classification and tagging tasks. We study well-known CNN architectures and how their building blocks affect their receptive field. We propose several systematic approaches to control the RF of CNNs and systematically test the resulting architectures on different audio classification and tagging tasks and datasets. The experiments show that regularizing the RF of CNNs using our proposed approaches can drastically improve the generalization of models, out-performing complex architectures and pre-trained models on larger datasets. The proposed CNNs achieve state-of-the-art results in multiple tasks, from acoustic scene classification to emotion and theme detection in music to instrument recognition, as demonstrated by top ranks in several pertinent challenges (DCASE, MediaEval).

Recent research on fault diagnosis mainly focuses on how to improve diagnostic accuracy. For a given accuracy level, a variety of convolutional neural network architectures have been developed and available to achieve the specific accuracy level. With equivalent fault diagnosis accuracy, smaller convolutional neural network structure offers at least three advantages: (1) it can be deployed on hardware with limited memory, such as FPGAs; (2) the training of smaller convolutional neural networks can be faster under the same processor performance; (3) smaller network structures require less communication across severs during distributed training. So, in this paper, an improved convolutional neural network is constructed, and the following strategies are used to reduce network parameters and further make the convolutional neural networks with smaller structures: (1) equivalent replacement of large-scale convolution layers by multiple small-sized convolution layers; (2) avoid using a fully connected layer, and replace with a global pooling layer. Meanwhile, to ensure the model’s fault diagnosis accuracy and training effectiveness, inspired by the network in network, a 1*1 convolutional layer is inserted into a traditional convolutional layer to improve feature expression, and the batch-normalization layer is used to increase the training effectiveness.

Aim:This study investigates the topology of convolutional neural networks and proposes an information technology for the early detection of pneumonia in X-rays.Background:For the past decade, pneumonia has been one of the most widespread respiratory diseases. Every year, a significant part of the world's population suffers from pneumonia, which leads to millions of deaths worldwide. Inflammation occurs rapidly and usually proceeds in severe forms. Thus, early detection of the disease plays a critical role in its successful treatment.Objective:The most operating means of diagnosing pneumonia is the chest X-ray, which produces radiographs. Automated diagnostics using computing devices and computer vision techniques have become beneficial in X-ray image analysis, serving as an ancillary decision-making system. Nonetheless, such systems require continuous improvement for individual patient adjustment to ensure a successful, timely diagnosis.Methods:Nowadays, artificial neural networks serve as a promising solution for identifying pneumonia in radiographs. Despite the high level of recognition accuracy, neural networks have been perceived as black boxes because of the unclear interpretation of their performance results. Altogether, an insufficient explanation for the early diagnosis can be perceived as a severe negative feature of automated decision-making systems, as the lack of interpretation results may negatively affect the final clinical decision. To address this issue, we propose an approach to the automated diagnosis of early pneumonia, based on the classification of radiographs with weakly expressed disease features.Results:An effective spatial convolution operation with several dilated rates, combining various receptive feature fields, was used in convolutional layers to detect and analyze visual deviations in the X-ray image. Due to applying the dilated convolution operation, the network avoids significant losses of objects' spatial information providing relatively low computational costs. We also used transfer training to overcome the lack of data in the early diagnosis of pneumonia. An image analysis strategy based on class activation maps was used to interpret the classification results, critical for clinical decision making.Conclusion:According to the computational results, the proposed convolutional architecture may be an excellent solution for instant diagnosis in case of the first suspicion of early pneumonia.

Convolutional neural networks (CNNs) have been successfully applied to many tasks such as digit and object recognition. In this paper we study the size of the receptive field of deep convolutional neural networks, in particular, we check the idea of a redundant receptive field. We run a set of experiments on two common CNN models — VGG16 and ResNet18 — in order to explore the influence of receptive field size on CNN's training time, accuracy, and performance. We run experiments using the MakiFlow framework on the CALTECH256 dataset. The experiments' results show that the optimization of neural networks (NNs) by reducing the size of the receptive field allows to reduce the NN’s training time by 5-7% while maintaining the accuracy of the network.

Deep convolutional neural networks (CNNs) are a popular choice for many image classification tasks due to their good performance. In general, there is a correlation between the generalization performance of a deep learning model, the number of training samples available, and the number of free parameters in the model. If there are too many free parameters in the model relative to the amount of training data, the model overfits the training data. Reducing the number of trainable parameters in CNNs while maintaining performance levels is an active area of research. This is important, especially in resource-constrained environments where either the size of the training data is insufficient or the memory of the hardware where the model is to be deployed is limited, such as in mobile vision applications. In this paper, kernel transposition is proposed as a method for the reduction of free parameters in the model’s convolutional layers. This allows kernel reuse, in which a model learns a given kernel only once but uses it twice. The learned kernels and their transposes can either be used in sequence or in parallel to one another. These form series and parallel convolutional modules respectively. The modules are used as replacements of the traditional convolutional layers in a CNN model. The use of the modules reduces the number of free parameters and computational costs associated with convolution operations in the model without necessarily compromising its generalization performance. The proposed method is generic and can be adapted to existing state-of-the-art network architectures as demonstrated in the experiments using established architectures. The proposed method was validated on the CIFAR-10 and CIFAR-100 standard datasets using a series of experiments with five model architectures: a basic 5-convolutional layer CNN model, the ResNet-56, small MobileNetV3, EfficientNetB0 and ConvNext model architectures; all trained from scratch. Models based on standard convolutional layers (for basic CNN and ResNet-56 architectures) and depth-wise convolutional layers (for MobileNetV3, EfficientNetB0 and ConvNext architectures) were compared against those based on the proposed convolutional modules. The models based on the standard and depth-wise convolutional layers were used as baselines. The comparisons were in terms of the number of parameters, computational efficiencies (Floating Point Operations / FLOPs) and classification accuracies of the models. For a kernel size k, the experiments showed that, despite improvements on parameter and computational efficiencies, models based on the proposed modules for the basic CNN (k = 3) architecture had lower accuracy levels compared to their baseline model. However, the huge saving on the number of parameters (> 88%) and FLOPs (> 84%) is significant compared to the drop in accuracy (< 4%). For the MobileNetV3 (k = 3 or k = 5) and EfficientNetB0 (k = 3 or k = 5), only the model based on the series module had lower accuracy than the baseline models. The models based on the parallel module were generally at par (within a margin of $$\pm$$ 1.5%) in terms of accuracy with their respective baselines. In relation to the ConvNext (k = 7) and the ResNet-56 (k = 3 or k = 11, or k = 19) architectures, other than the improved parameter and computational efficiencies, the classification accuracies of all the models based on the proposed modules were either at par or better than those of their baseline models. In all the above architectures, the use of the proposed modules reduced the number of convolutional parameters in the networks by at least 78% with fewer FLOPs relative to the baseline models

Fingerprint recognition has widespread security applications owing to its uniqueness, permanence, and simplicity in capture. However, conventional fingerprint authentication systems face high false acceptance and rejection rates and susceptibility to spoofing attacks. To address these issues, this study proposes a deep learning-based fingerprint authentication system using a Convolutional Neural Network (CNN) with five convolution layers to derive robust spatial features. The model was trained and cross-validated on the SOCOFing dataset with regularization and data augmentation to enhance generalization and spoof resistance. Experimental results show that the proposed CNN achieved a training accuracy of 99.10% with a loss of 0.0223 and a validation accuracy of 98.89% with a loss of 0.0114. Moreover, the model maintained a low false acceptance rate of 0.33% and false rejection rate of 0.25%, demonstrating its efficacy and credibility for secure and real-time biometric authentication. A rigorous comparison with conventional CNN models and DCCN architectures confirmed that the proposed model provides higher accuracy, lower computational cost, and stronger resistance to spoofing attacks. These findings indicate that the proposed system successfully addresses existing limitations and offers a practical, scalable, and reliable solution for fingerprint verification using deep CNN architectures.

The Accurate and the early detection of the neurodegenerative brain disorders such as the Alzheimer’s disease(AD) is crucial today for an effective patient care and prevention of disease progression. Computer Assisted Diagnosis (CAD) based on the neuroimaging data is an active research that was enhanced recent years by the deep learning methods. Brain modalities hold the key to achieve a comprehensive understanding of the brain changes and disorders. Magnetic Resonance Imaging(MRI) is a powerful brain modality and biomarker to capture the brain changes. It has been widely adopted for the AD diagnosis. Most of the recent studies are based on the Convolution Neural Network (CNN)using the MRI modality for the AD detection. However the CNN requires a huge amount of data. Furthermore, current CNN models cannot provide a good feature representation due to the lack of an adaptive channel weighting. The main drawback of the CNN network is its inability to learns the importance of the image features and the main correlations among them. Current methods that are based on the CNN does not ensure a valuable information extraction about which brain regions are related to the AD detection. To deal with these challenges, we proposed 2 contributions. Firstly, we used a Deep Convolution Generative Adversarial neural network(DCGAN) for data augmentation to enhance the training and to tackle the lack of data. We proposed also a new method for AD detection based on the integration of the Efficient Channel Attention(ECA-Net) into the Residual network. The ECA module improve the residual network by an attention mechanism to extract the most relevant features and ensure an adaptive weighting channel mechanism. Our method provide an efficient feature representation and extraction with low computational cost. The proposed method was validated using the ADNI (Alzheimer’s disease neuroimaging initiative) and the Open Access Series of Imaging Studies (OASIS). The experimental results outlines that our proposed method achieved the highest classification accuracy comparing with different methods by reaching an accuracy rate of 95%KeywordsAlzheimer’s diseaseData augmentationAttention mechanismDCGANECA-ResNet101

Acoustic scene classification and related tasks have been dominated by Convolutional Neural Networks (CNNs). Top-performing CNNs use mainly audio spectograms as input and borrow their architectural design primarily from computer vision. A recent study has shown that restricting the receptive field (RF) of CNNs in appropriate ways is crucial for their performance, robustness and generalization in audio tasks. One side effect of restricting the RF of CNNs is that more frequency information is lost. In this paper, we perform a systematic investigation of different RF configuration for various CNN architectures on the DCASE 2019 Task 1.A dataset. Second, we introduce Frequency Aware CNNs to compensate for the lack of frequency information caused by the restricted RF, and experimentally determine if and in what RF ranges they yield additional improvement. The result of these investigations are several well-performing submissions to different tasks in the DCASE 2019 Challenge.