A deep learning convolution architecture for simple embedded applications

The demand on devices and systems empowered by biometric verification and identification mechanisms has been increasing in recent years as they have become a significant part of our lives. Palm vein biometric is an emerging technology that has drawn considerable attention from researchers and scientists over the last decade. In the present work, a novel feature extraction methodology named GPWLD combing the Gabor features with positional Weber's local descriptor (PWLD) features is proposed. WLD is a highly representative micro-pattern descriptor that performs well against noise and illumination changes in images. However, it lacks the ability to capture the vein pattern features at different orientations. Moreover, its descriptor packs the local information content into a single histogram that does not take the spatial locality of micro-patterns into consideration. To solve these two issues, firstly, the palm vein image is passed through Gabor filter with different orientations to capture the salient rotational features found in the output feature maps. Secondly, the spatiality is achieved by uniformly dividing each feature map into several blocks. Next, Weber's law feature descriptor (WLD) histogram is computed for each block in every feature map. Finally, these histograms are concatenated to compose the final feature vector. Due to the high dimensionality of the final feature vector, Principal component analysis (PCA) algorithm is utilized for feature size reduction. In the classification stage, a deep neural network (DNN) comprising an optimized stacked autoencoder (SAE) with Bayesian optimization and a softmax layer is used. Optimization of the SAE is carried out by using Bayesian optimization to find the optimal SAE structure and the options of the training algorithm. The Experimental results verify the discriminative power of the extracted features and the accuracy of the proposed DNN. For both Identification and verification experiments, the proposed method attains higher identification rate and lower EER than state-of-the-art methods.

High-level feature maps of Convolutional Neural Networks are computed by reusing their corresponding low-level feature maps, which brings into full play feature reuse to improve the computational efficiency. This form of feature reuse is referred to as feature reuse between convolutional layers. The second type of feature reuse is referred to as feature reuse within the convolution, where the channels of the output feature maps of the convolution are computed by reusing the same channels of the input feature maps, which results in an approximation of the channels of the output feature maps. To compute them accurately, we need specialized input feature maps for every channel of the output feature maps. In this paper, we first discuss the approximation problem introduced by full feature reuse within the convolution and then propose a new feature reuse scheme called Reducing Approximation of channels by Reducing Feature reuse (REAF). The paper also shows that group convolution is a special case of our REAF scheme and we analyze the advantage of REAF compared to such group convolution. Moreover, we develop the REAF+ scheme and integrate it with group convolution-based models. Compared with baselines, experiments on image classification demonstrate the effectiveness of our REAF and REAF+ schemes. Under the given computational complexity budget, the Top-1 accuracy of REAF-ResNet50 and REAF+-MobileNetV2 on ImageNet will increase by 0.37% and 0.69% respectively. The code and pre-trained models will be publicly available.

Convolutional Neural Networks (CNNs) are frequently used algorithms because of their propensity to learn relevant and hierarchical features through their feature extraction technique. However, the availability of enormous volumes of data in various variations is crucial for their performance. Capsule networks (CapsNets) perform well on a small amount of data but perform poorly on complex images. To address this, we proposed a new Capsule Network architecture called Tri Texton-Dense CapsNet (TTDCapsNet) for better complex and medical image classification. The TTDCapsNet is made up of three hierarchic blocks of Texton-Dense CapsNet (TDCapsNet) models. A single TDCapsNet is a CapsNet architecture composed of a texton detection layer to extract essential features, which are passed onto an eight-layered block of dense convolution that further extracts features, and then the output feature map is given as input to a Primary Capsule (PC), and then to a Class Capsule (CC) layer for classification. The resulting feature map from the first PC serves as input into the second-level TDCapsNet, and that from the second PC serves as input into the third-level TDCapsNet. The routing algorithm receives feature maps from each PC for the various CCs. Routing the concatenation of the three PCs creates an additional CC layer. All these four feature maps combined, help to achieve better classification. On fashion-MNIST, CIFAR-10, Breast Cancer, and Brain Tumor datasets, the proposed model is evaluated and achieved validation accuracies of 94.90%, 89.09%, 95.01%, and 97.71% respectively. Findings from this work indicate that TTDCapsNet outperforms the baseline and performs comparatively well with the state-of-the-art CapsNet models using different performance metrics. This work clarifies the viability of using Capsule Network on complex tasks in the real world. Thus, the proposed model can be used as an intelligent system, to help oncologists in diagnosing cancerous diseases and administering treatment required.

Semantic segmentation of high-resolution optical remote sensing images is an important but challenging task. To solve the problem that many semantic segmentation networks fail to efficiently utilize global and local context information to improve the segmentation performance, this paper proposes a semantic segmentation network based on sparse self-attention (SDANet) to model the global context dependencies. Specifically, the feature maps are first divided into four regions in spatial and channel dimensions, respectively, and the divided feature maps are rearranged to form new regions. Second, the position and channel self-attention operations are performed on the rearranged regions. Third, the feature maps are restored to the original combination and the position together with channel self-attention operations are performed again to obtain the output feature maps. Finally, semantic segmentation is completed based on the output feature maps. Extensive experiments conducted on the ISPRS Vaihingen dataset demonstrate that the proposed method is superior to self-attention-based DANet, CCNet, and other general semantic segmentation networks, such as FCN, Deeplabv3+, HRNet, etc.

The study of remote sensing image object detection has excellent research value in environmental protection and public safety. However, the performance of the detectors is unsatisfactory due to the large variability of object size and complex background noise in remote sensing images. Therefore, it is essential to improve the detection performance of the detectors. Inspired by the idea of "divide and conquer", we proposed a Multiple Receptive Field Attention(MRFA) to solve these problems and which is a plug-and-play attention method. First, we use the method of multiple receptive field feature map generation to convert the input feature map into four feature maps with different receptive fields. In this way, the small, medium, large, and immense objects in the input feature maps are "seen" in these feature maps, respectively. Then, we used the multiple attention map fusion method to focus objects of different sizes separately, which can effectively suppress noise in the background of remote sensing images. Experiments on remote sensing object detection datasets DIOR and HRRSD demonstrate that the performance of our method is better than other state-of-the-art attention modules. At the same time, the experiments on remote sensing image semantic segmentation dataset WHDLD and classification dataset AID prove the generalization and superiority of our method.

In deep convolutional neural networks (CNNs), local pooling operation is a key building block to effectively downsize feature maps for reducing computation cost as well as increasing robustness against input variation. There are several types of pooling operation, such as average/max-pooling, from which one has to be manually selected for building CNNs. The optimal pooling type would be dependent on characteristics of features in CNNs and classification tasks, making it hard to find out the proper pooling module in advance. In this paper, we propose a flexible pooling method which adaptively tunes the pooling functionality based on input features without manually fixing it beforehand. In the proposed method, the parameterized pooling form is derived from a probabilistic perspective to flexibly represent various types of pooling and then the parameters are estimated by means of global statistics in the input feature map. Thus, the proposed local pooling guided by global features effectively works in the CNNs trained in an end-to-end manner. The experimental results on image classification tasks demonstrate the effectiveness of the proposed pooling method in various deep CNNs.

The goal of multi-modal learning is to use complementary information on the relevant task provided by the multiple modalities to achieve reliable and robust performance. Recently, deep learning has led significant improvement in multi-modal learning by allowing for fusing high level features obtained at intermediate layers of the deep neural network. This paper addresses a problem of designing robust deep multi-modal learning architecture in the presence of the modalities degraded in quality. We introduce deep fusion architecture for object detection which processes each modality using the separate convolutional neural network (CNN) and constructs the joint feature maps by combining the intermediate features obtained by the CNNs. In order to facilitate the robustness to the degraded modalities, we employ the gated information fusion (GIF) network which weights the contribution from each modality according to the input feature maps to be fused. The combining weights are determined by applying the convolutional layers followed by the sigmoid function to the concatenated intermediate feature maps. The whole network including the CNN backbone and GIF is trained in an end-to-end fashion. Our experiments show that the proposed GIF network offers the additional architectural flexibility to achieve the robust performance in handling some degraded modalities.

In this paper, we propose a simple but effective framework for lane boundary detection, called SpinNet. Considering that cars or pedestrians often occlude lane boundaries and that the local features of lane boundaries are not distinctive, therefore, analyzing and collecting global context information is crucial for lane boundary detection. To this end, we design a novel spinning convolution layer and a brand-new lane parameterization branch in our network to detect lane boundaries from a global perspective. To extract features in narrow strip-shaped fields, we adopt strip-shaped convolutions with kernels which have 1 × n or n × 1 shape in the spinning convolution layer. To tackle the problem of that straight strip-shaped convolutions are only able to extract features in vertical or horizontal directions, we introduce the concept of feature map rotation to allow the convolutions to be applied in multiple directions so that more information can be collected concerning a whole lane boundary. Moreover, unlike most existing lane boundary detectors, which extract lane boundaries from segmentation masks, our lane boundary parameterization branch predicts a curve expression for the lane boundary for each pixel in the output feature map. And the network utilizes this information to predict the weights of the curve, to better form the final lane boundaries. Our framework is easy to implement and end-to-end trainable. Experiments show that our proposed SpinNet outperforms state-of-the-art methods.

Bearings, extensively utilized in the industrial sector, play a pivotal role in the defect detection of industrial components. This paper proposes a defect detection algorithm based on an enhanced YOLOv5s to improve the accuracy and speed of bearing ring end face defect detection. The algorithm boosts minimal target detection by incorporating a small object detection head and combining multi-scale representation learning with anchor box calculations. It introduces a formula to calculate the receptive field sizes of convolutional layers. By calculating the receptive field sizes corresponding to the feature map pixels and integrating this data with anchor analysis, three anchors optimal for the small object detection head are determined. Additionally, an attention mechanism refines the receptive fields of the neural network’s output feature maps, enhancing the model’s performance. Extensive experiments on a dataset of bearing ring end face defects from industrial sites reveal that the improved YOLOv5s algorithm achieves a detection accuracy (mAP) of 96.14%, a detection speed of up to 44 FPS, and a model size of 20.08M. Compared to other mainstream detection models, this algorithm not only meets but exceeds the real-time detection requirements of industrial production in terms of accuracy and model complexity. With its high precision, compact model size, and rapid detection speed, this algorithm provides a robust foundation for quality control in bearing production.

Knowledge-driven building extraction method exhibits a restricted adaptability scope and is vulnerable to external factors that affect its extraction accuracy. On the other hand, data-driven building extraction method lacks interpretability, heavily relies on extensive training data, and may result in extraction outcomes with building boundary blur issues. The integration of pre-existing knowledge with data-driven learning is essential for the intelligent identification and extraction of buildings from high-resolution aerial images. To overcome the limitations of current deep learning building extraction networks in effectively leveraging prior knowledge of aerial images, a geometric significance-aware deep mutual learning network (GSDMLNet) is proposed. Firstly, the GeoSay algorithm is utilized to derive building geometric significance feature maps as prior knowledge and integrate them into the deep learning network to enhance the targeted extraction of building features. Secondly, a bi-directional guidance attention module (BGAM) is developed to facilitate deep mutual learning between the building feature map and the building geometric significance feature map within the dual-branch network. Furthermore, the deployment of an enhanced flow alignment module (FAM++) is utilized to produce high-resolution, robust semantic feature maps with strong interpretability. Ultimately, a multi-objective loss function is crafted to refine the network’s performance. Experimental results demonstrate that the GSDMLNet excels in building extraction tasks within densely populated and diverse urban areas, reducing misidentification of shadow-obscured regions and color-similar terrains lacking building structural features. This approach effectively ensures the precise acquisition of urban building information in aerial images.

A deep learning approach to glioma segmentation is presented. An encoder and decoder pair deep learning network is designed which takes T1, T2, T1-CE (contrast enhanced) and T2-Flair (fluid attenuation inversion recovery) images as input and outputs the segmented labels. The encoder is a 49 layer deep residual learning architecture that encodes the \(240\,\times \,240\,\times \,4\) input images into \(8\,\times \,8\,\times \,2048\) feature maps. The decoder network takes these feature maps and extract the segmented labels. The decoder network is fully convolutional network consisting of convolutional and upsampling layers. Additionally, the input images are downsampled using bilinear interpolation and are inserted into the decoder network through concatenation. This concatenation step provides spatial information of the tumor to the decoder, which was lost due to pooling/downlsampling during encoding. The network is trained on the BRATS-17 training dataset and validated on the validation dataset. The dice score, sensitivity and specificity of the segmented whole tumor, core tumor and enhancing tumor is computed on validation dataset. The mean dice score for whole tumor, core tumor and enhancing tumor for validation dataset were 0.824, 0.627 and 0.575, respectively.

Recent studies on semantic segmentation are exploiting contextual information to address the problem of inconsistent parsing prediction in big objects and ignorance in small objects. However, they utilize multilevel contextual information equally across pixels, overlooking those different pixels may demand different levels of context. Motivated by the above-mentioned intuition, we propose a novel global-guided selective context network (GSCNet) to adaptively select contextual information for improving scene parsing. Specifically, we introduce two global-guided modules, called global-guided global module (GGM) and global-guided local module (GLM), to, respectively, select global context (GC) and local context (LC) for pixels. When given an input feature map, GGM jointly employs the input feature map and its globally pooled feature to learn its global contextual demand based on which per-pixel GC is selected. While GLM adopts low-level feature from the adjacent stage as LC and synthetically models the input feature map, its globally pooled feature and LC to generate local contextual demand, based on which per-pixel LC is selected. Furthermore, we combine these two modules as a selective context block and import such SCBs in different levels of the network to propagate contextual information in a coarse-to-fine manner. Finally, we conduct extensive experiments to verify the effectiveness of our proposed model and achieve state-of-the-art performance on four challenging scene parsing data sets, i.e., Cityscapes, ADE20K, PASCAL Context, and COCO Stuff. Especially, GSCNet-101 obtains 82.6% on Cityscapes test set without using coarse data and 56.22% on ADE20K test set.

OENet: An overexposure correction network fused with residual block and transformer

Deep learning has been successfully deployed to a broad range of applications with its outstanding performance. Supporting an efficient hardware architecture is critical to making effective use of a deep learning approach with proven algorithm performance. One challenge in implementation of deep learning algorithm is to reduce memory bandwidth because a single memory access normally consumes 100* more energy than an arithmetic operation. To reduce the memory bandwidth, deep learning data could be compressed and decompressed before memory write/read operations. Especially, feature maps, which account for a significant portion of the convolutional neural network (CNN), could be compressed further by reducing the correlations between feature map planes. This paper proposes a compression method for feature maps in CNN that adaptively exploits the varying correlation between feature map planes. For every feature map plane, the proposed method searches the most similar plane among nearby planes in the same layer, and compresses the residual of the two planes instead of the plane itself. Experimental results show that the average bit length to store feature maps is reduced by 14.2% compared to the compression without correlation reduction, and the CNN accuracy does not change and additional training is also not required because the proposed method applies lossless compression.

Instance segmentation based on deep learning is one of the popular research direction in the field of artificial intelligence. It is widely used in fields such as intelligent manufacturing and autonomous driving. At present, most of the instance segmentation are based on the object detection. The defect of this scheme is that the pixels used to predict the mask cannot fully contain all the information of the target object, resulting in the decline of segmentation accuracy. The accuracy of instance segmentation will affect the reliability of the application that applies computer vision programs. To solve the above problems, the paper proposes a global feature fusion module to improve the prediction accuracy of mask in instance segmentation. GFF module can extract the feature of the whole size feature map directly. GFF module can make up the target information of the pixel corresponding to the receptive field of the feature map that cannot be covered. The actual receptive field is smaller than the theoretical receptive field, which leads to some loss of information in the high-level feature map. The feature map produced by GFF module can also be used as supplementary information to add to the prediction of the mask to solve the problem of the limitation of theoretical receptive field and practical receptive field. At the same time, GFF module only uses one hyperparameter, which reduces the difficulty of network hyperparameter adjustment. It was added to YOLACT to improve the mask prediction accuracy of the mask. Although the structure of YOLACT becomes more complex after adding GFF module, the overall parameters of the network do not change much. The reasoning speed of YOLACT-GFF with resnet50 as the backbone is 20.79 FPS and mAP is 28.3 under the condition of GTX 2080ti.