Hint Pyramid Learning Via Salient Semantic Mining

Given the wide diffusion of deep neural network architectures for computer vision tasks, several new applications are nowadays more and more feasible. Among them, a particular attention has been recently given to instance segmentation, by exploiting the results achievable by two-stage networks (such as Mask R-CNN or Faster R-CNN), derived from R-CNN. In these complex architectures, a crucial role is played by the Region of Interest (RoI) extraction layer, devoted to extracting a coherent subset of features from a single Feature Pyramid Network (FPN) layer attached on top of a backbone. This paper is motivated by the need to overcome the limitations of existing RoI extractors which select only one (the best) layer from FPN. Our intuition is that all the layers of FPN retain useful information. Therefore, the proposed layer (called Generic RoI Extractor - GRoIE) introduces non-local building blocks and attention mechanisms to boost the performance. A comprehensive ablation study at component level is conducted to find the best set of algorithms and parameters for the GRoIE layer. Moreover, GRoIE can be integrated seamlessly with every two-stage architecture for both object detection and instance segmentation tasks. Therefore, the improvements brought about by the use of GRoIE in different state-of-the-art architectures are also evaluated. The proposed layer leads up to gain a 1.1% AP improvement on bounding box detection and 1.7% AP improvement on instance segmentation. The code is publicly available on GitHub repository at https://github.com/IMPLabUniPr/mmdetection/tree/groie_dev

HIGHLIGHTSWe propose a balanced feature pyramid network model to achieve automatic recognition of agricultural pests.We propose and improved feature pyramid module to address the problems of the large semantic gap before feature fusion in FPN (feature pyramid network), information loss during feature fusion, and how to utilize complementary information to strengthen features after feature fusion.Our method is evaluated on the pest dataset and achieves better recognition results, the mAP (mean Average Precision) reaches 90.04%, and the final mAP after combining with the data enhancement strategy reaches 92.56%.Abstract.In order to ensure the steady improvement of the quality and efficiency of agricultural production, the precise prevention and control of agriculture pests should be given top priority. Based on the Faster R-CNN model, we propose a feature-augmented recognition model (FARM) for the detection of agricultural pests to improve the professional level of pest control. Firstly, we select VoVNet as the backbone network to extract rich visual information. Secondly, in view of the existing problems of FPN (Feature Pyramid Network), we propose a feature pyramid enhancement network to improve it. Finally, we use an adaptive anchor box network to train anchor boxes to avoid the problem of unbalanced positive and negative samples caused by artificial parameter adjustment. The experimental result shows that our method can achieve better recognition results; the mAP on the agricultural pest dataset reaches 90.04%, and the final mAP after combining with the data enhancement strategy reaches 92.56%. Keywords: Adaptive anchor box, Agricultural pests, Feature augmented, Feature pyramid.

FPN (Feature Pyramid Network) and transformer-based target detectors are commonly employed in target detection tasks. However, these approaches suffer from design flaws that restrict their performance. To overcome these limitations, we proposed TIG-DETR (Texturized Instance Guidance DETR), a novel target detection model. TIG-DETR comprises a backbone network, TE-FPN (Texture-Enhanced FPN), and an enhanced DETR detector. TE-FPN addresses the issue of texture information loss in FPN by utilizing a bottom-up architecture, Lightweight Feature-wise Attention, and Feature-wise Attention. These components effectively compensate for texture information loss, mitigate the confounding effect of cross-scale fusion, and enhance the final output features. Additionally, we introduced the Instance Based Advanced Guidance Module in the DETR-based detector to tackle the weak detection of larger objects caused by the limitations of window interactions in Shifted Window-based Self-Attention. By incorporating TE-FPN instead of FPN in Faster RCNN and employing ResNet-50 as the backbone network, we observed an improvement of 1.9 AP in average accuracy. By introducing the Instance-Based Advanced Guidance Module, the average accuracy of the DETR-based target detector has been improved by 0.4 AP. TIG-DETR achieves an impressive average accuracy of 44.1% with ResNet-50 as the backbone network.

Semantic segmentation of remote sensing images plays an important role in a\nwide range of applications including land resource management, biosphere\nmonitoring and urban planning. Although the accuracy of semantic segmentation\nin remote sensing images has been increased significantly by deep convolutional\nneural networks, several limitations exist in standard models. First, for\nencoder-decoder architectures such as U-Net, the utilization of multi-scale\nfeatures causes the underuse of information, where low-level features and\nhigh-level features are concatenated directly without any refinement. Second,\nlong-range dependencies of feature maps are insufficiently explored, resulting\nin sub-optimal feature representations associated with each semantic class.\nThird, even though the dot-product attention mechanism has been introduced and\nutilized in semantic segmentation to model long-range dependencies, the large\ntime and space demands of attention impede the actual usage of attention in\napplication scenarios with large-scale input. This paper proposed a\nMulti-Attention-Network (MANet) to address these issues by extracting\ncontextual dependencies through multiple efficient attention modules. A novel\nattention mechanism of kernel attention with linear complexity is proposed to\nalleviate the large computational demand in attention. Based on kernel\nattention and channel attention, we integrate local feature maps extracted by\nResNeXt-101 with their corresponding global dependencies and reweight\ninterdependent channel maps adaptively. Numerical experiments on three\nlarge-scale fine resolution remote sensing images captured by different\nsatellite sensors demonstrate the superior performance of the proposed MANet,\noutperforming the DeepLab V3+, PSPNet, FastFCN, DANet, OCRNet, and other\nbenchmark approaches.\n

Timely fault diagnosis and localization of transmission lines is crucial for ensuring the reliable operation of increasingly complex power systems. This study proposes an optimized R-Net algorithm based on a feature pyramid network (FPN) and densely connected convolutional network (D-Net) for transmission line fault diagnosis and localization. The R-Net network is enhanced by reshaping the anchor points using an improved K-means algorithm and incorporating an FPN for multi-scale feature extraction. The backbone network is further optimized using D-Net to strengthen inter-layer connections and improve feature reuse. Experimental results demonstrate that the optimized R-Net achieves an overall average accuracy of 0.64, outperforming the original network by 1.30%. The accuracy improvement is particularly significant for ground wire defects (2.40%). The D-Net-based R-Net, despite having fewer parameters, maintains high accuracy (0.6502). Compared to other object detection algorithms, such as YOLO-v3 and Faster R-CNN, the optimized R-Net exhibits superior performance in terms of mean average precision (15.58% and 2.45% higher, respectively) and parameter efficiency (17M vs. 38M and 81M). Considering both performance and speed, the optimized R-Net achieves a processing rate of 10.5 frames per second. This study provides an efficient and accurate tool for transmission line fault diagnosis and localization, with significant practical implications for power system operation and maintenance.

HighlightsA feature pyramid network with Scaling Feature Fusion and Enhanced Expansion Feature module is constructed to overcome the shortcomings of feature pyramids for small target detection.Noise addition, cropping, and other data augmentation methods for expanding data sets.Multi-channel Denoising Feature Fusion is constructed in the detection module to solve the inconsistencies inside the feature pyramid.Combine with YOLOv5's Backbone to complete the feature anchor box selection and detection can quickly and accurately complete the grape leaf disease detection.ABSTRACT.Grape leaf diseases all over the world cannot be ignored; it is one of the important causes of agricultural economic damage. In order to control the cost of manual detection of grape diseases and to overcome the impact on detection accuracy due to the large variation of target scales during automatic detection of grape leaves, the feature pyramid method is commonly used to solve this problem, but it may destroy the feature consistency of grape leaves at different scales. In addition, in practical applications, the commonly used methods are difficult to improve the detection accuracy of multi-scale grape leaves while ensuring detection. In this paper, we apply the improved YOLOv5 algorithm, which is used to detect the images of grapevine powdery mildew, grape mosaic virus disease, and grape downy mildew, as well as the images of healthy grape leaves. The proposed algorithm is based on the improved feature pyramid network SE-FPN, which added Scaling Feature Fusion (SFF) to enhance feature map information and Enhanced Expansion Feature (EEF) to improve multi-scale accuracy prediction. And replacing the original Path Aggregation Network (PANet) detection module with the Multi-channel Denoising Feature Fusion (MDFF) network improves the scale invariance of features and reduces the inference cost. By experimentally comparing the original YOLOv5 network and the YOLOv5 network with other improved methods applied, considering the following metrics: mean Average Precision (mAP), accuracy, recall, and mean target detection loss, mean classification loss, and runtime of the validation set, the results conclude that the improved model proposed in this paper performs the best, which increases the accuracy by 5.96% and the recall by 4.69% compared to the original YOLOv5 model. The algorithm provides good technical support for real-time detection and control of grapevine diseases. Keywords: Detection module, Disease identification, Feature pyramid network, Grape leaf disease, YOLO.

Scene text spotting has made tremendous progress with the in-depth research on deep convolutional neural networks (DCNN). Previous approaches mainly focus on the spotting of arbitrary-shaped scene text, on which it is difficult to achieve satisfactory results on dense scene text containing various instances of bending, occlusion, and lighting. To address this problem, we propose an approach called Rwin-FPN++, which incorporates the long-range dependency merit of the Rwin Transformer into the feature pyramid network (FPN) to effectively enhance the functionality and generalization of FPN. Specifically, we first propose the rotated windows-based Transformer (Rwin) to enhance the rotation-invariant performance of self-attention. Then, we attach the Rwin Transformer to each level on our feature pyramids to extract global self-attention contexts for each feature map produced by the FPN. Thirdly, we fuse these feature pyramids by upsampling to predict the score matrix and keypoints matrix of the text regions. Fourthly, a simple post-processing process is adopted to precisely merge the pixels in the score matrix and keypoints matrix and obtain the final segmentation results. Finally, we use the recurrent neural network to recognize each segmentation region and thus achieve the final spotting results. To evaluate the performance of our Rwin-FPN++ network, we construct a dense scene text dataset with various shapes and occlusion from the wiring of the terminal block of the substation panel cabinet. We train our Rwin-FPN++ network on public datasets and then evaluate the performance on our dense scene text dataset. Experiments demonstrate that our Rwin-FPN++ network can achieve an F-measure of 79% and outperform all other methods in F-measure by at least 2.8%. This is because our proposed method has better rotation invariance and long-range dependency merit.

The application of base on deep learning remote sensing image landslide detection has become a trend. In view of the, large difference in the scale of landslide objects and the very important characteristics of the shallow layer of the landslide object, we modified the original Faster RCNN. First compare the average accuracy values of different network models, and select the network ResNet50 with deeper layers, and then integrated into Feature Pyramid Network (FPN) on the basis of ResNet50, Fusion of FPN structure with RPN and Fast RCNN. Finally, it is verified on the public landslide dataset in BiJie City, Guizhou Province, China, and the average accuracy of the obtained model is 87.10%, which is 16.9% higher than the 70.20% of VGG16. Contrast experiments with different anchor ratios and different hyperparameters in the FPN part are carried out, and the generated results are analyzed and compared. The experimental results show that the proposed method has good robustness and high accuracy for the detection of landslide objects, in remote sensing images, which can provide a technical reference for further research on landslide detection in remote sensing images.

Attention mechanisms have been widely used to address the common multi-scale challenges in remote sensing imagery for tasks such as object detection and instance segmentation. However, they often require the design of task-specific network architectures tailored to different data targets. To better accommodate objects of varying scales and downstream task requirements, we propose a dynamic search framework based on a Mixture-of-Experts model. In this framework, each module is treated as an expert, and their combinations are flexibly adjusted to ensure that the pyramid structure adapts to the needs of different-scale tasks. Specifically, we design a Scalable Bi-Directional Feature Pyramid Network (SBFPN), which incorporates various hybrid attention mechanisms to dynamically enhance feature fusion across different layers. This approach not only captures long-range dependencies within the image but also suppresses noise and interference from complex backgrounds. The scalable feature pyramid structure we construct utilizes sparse feature fusion within the same layers, while also computing self-attention weights between nodes and retaining or trimming skip connections across layers. We conduct experiments on object detection, instance segmentation, and panoptic segmentation tasks. This structure can be transferred to YOLO-based and R-CNN-based networks, achieving superior detection and segmentation performance. On the Airbus Ship dataset, the mAP for detection and segmentation increased from 71.3% and 62.4% to 82.7% and 71.1%, respectively, demonstrating the effectiveness of our proposed method. Code is available at https://github.com/chaibosong/SAFPN.

The accurate and rapid acquisition of crop and weed information is an important prerequisite for automated weeding operations. This paper proposes the application of a network model based on Faster R-CNN for weed identification in images of cropping areas. The feature pyramid network (FPN) algorithm is integrated into the Faster R-CNN network to improve recognition accuracy. The Faster R-CNN deep learning network model is used to share convolution features, and the ResNeXt network is fused with FPN for feature extractions. Tests using >3000 images for training and >1000 images for testing demonstrate a recognition accuracy of >95%. The proposed method can effectively detect weeds in images with complex backgrounds taken in the field, thereby facilitating accurate automated weed control systems.

Enhanced Detection of Glass Insulator Defects Using Improved Generative Modeling and Faster RCNN

Intelligent traffic surveillance (ITS) in nighttime has increasingly become important in advanced-driving-assisted-systems (ADAS). ITS relies on efficient detection of objects and adaptability to changing environments in the nighttime. However, detecting the visual features of the objects in nighttime poses challenge due to insufficient illumination conditions. With the development of deep learning object detectors, higher accuracies have been achieved in object detection from nighttime images. In this paper, we employ convolutional neural network (CNN) object detectors such as Faster R-CNN (with various feature extractors), Feature Pyramid Network, and Single Shot Detector to analyse their performance in terms of speed and accuracy using our datasets for nighttime object detection. We carry out the image quality assessment of our datasets and perform an automatic parameter tuning for re-training the object detectors. Experimental results show that the Faster R-CNN with inception has higher detection performance compared with other CNN based detectors.

In this work, we propose to detect the iris and periocular regions simultaneously using coarse annotations and two well-known object detectors: YOLOv2 and Faster R-CNN. We believe coarse annotations can be used in recognition systems based on the iris and periocular regions, given the much smaller engineering effort required to manually annotate the training images. We manually made coarse annotations of the iris and periocular regions (122K images from the visible (VIS) spectrum and 38K images from the near-infrared (NIR) spectrum). The iris annotations in the NIR databases were generated semi-automatically by first applying an iris segmentation CNN and then performing a manual inspection. These annotations were made for 11 well-known public databases (3 NIR and 8 VIS) designed for the iris-based recognition problem and are publicly available to the research community. Experimenting our proposal on these databases, we highlight two results. First, the Faster R-CNN + Feature Pyramid Network (FPN) model reported an Intersection over Union (IoU) higher than YOLOv2 (91.86% vs 85.30%). Second, the detection of the iris and periocular regions being performed simultaneously is as accurate as performed separately, but with a lower computational cost, i.e., two tasks were carried out at the cost of one.

To identify and locate industrial textile defects accurately, this study proposes a textile detection model based on a convolution neural network (CNN) known as Faster R-CNN. First, a textile defect feature map was extracted by ResNet-101 deep convolution network. Faster R-CNN only extracts features from the last layer of the feature map, which leads to a loss of low-level location information. The proposed method adds the feature pyramid network (FPN) to the network architecture to make an independent prediction for each level in the feature extraction stage. The extracted feature map is input into the regional proposal network, among which the overlapping regional proposals are suppressed. The proposed improved Faster R-CNN model with Region Proposal Network (RPN), Soft Non-Maximum Suppression (NMS), and Region of Interest (ROI) Align can achieve a detection accuracy of 98% and an mean of Average Precision (mAP) of 85%, which is more competitive than the state-of-the-art deep learning-based object detection algorithms.

Aiming at the problems of high computational cost based on deep backbones (e.g., ResNet-50, ResNet-101, DenseNet-169) in the state-of-the-art method about object detector, this paper improves the capability of feature representations by using New Feature Pyramid module on the basis of fast lightweight backbone network (vgg-16), and finally establishes a fast and accurate detector. The architecture of our model is named New FPN (New Feature Pyramid Network). Based on the structure of Feature Pyramid Network, we design a novel New Feature Pyramid Network, which consists of a combination of top-down and bottom-up connections to fuse features across scales, and achieves high-level semantic feature map at all scales. The experimental results show that New FPN achieves state-of-the-art detection accuracy (i.e. 79.2%mAP) on PASCAL VOC 2007 with high efficiency (i.e. 73FPS).