Anchor-Free Arbitrary-Oriented Object Detector Using Box Boundary-Aware Vectors

We propose Multi-scale convolutional recurrent neural networks (MCRNN) and data augmentation methods to detect polyphonic sound event with few training data. MCRNN consists of Multi-scale convolutional neural networks (MCNN) and recurrent neural networks (RNN). MCNN concatenates the higher level features extracted using multiple convolution kernels with different scales from the time domain and frequency domain at the same time. RNN is able to capture the longer term temporal context characteristics. A novel background spectrum random replacement (BSRR) data augmentation method is applied to expand training data, which uses standard normal distribution data with randomly selected position and length instead of the original time-domain, frequency-domain or time-frequency domain background spectrum features. Our method is tested on the datasets of DCASE 2019 Task3 (T3). The experimental results showed that the MCRNN and BSRR data augmentation method are efficient. We achieved better results than the first place and the single advanced on the T3 by applying BSRR and SpecAugment data augmentation method simultaneously. On the evaluation dataset (T3-eval), our best result shows 0.05 and 0.975 of error rate (ER) and F1 respectively. Our method got the best performance and relatively improved 17% and 1% than the corresponding values of the single advanced.

The development of technological devices, such as satellites and drones, has made it easier to collect images and videos from above. With rich and diverse data sources, the problem of detecting objects in aerial images is formed to serve real-life situations: rescue missions, social scenarios, security needs, and more. This issue attracts a lot of attention from researchers in computer vision because of its high applicability in real life. However, object detection in images collected from drones faces many significant challenges due to the complex factors of real-life environments. To overcome these difficulties, many studies focus on developing effective models for detecting and tracking objects from UAV systems. In this paper, our primary emphasis is comprehensive survey of methodologies pertaining to horizontal and oriented object detection. We analyzed and compared existing approaches such as CNN-based vs. Transformer-based, Anchor-based vs. Anchor-free. Moreover, we also learned about ways to represent objects using bounding boxes in oriented object detection. After researching many practical and innovative techniques in different approaches, we have carefully selected 15 methods for horizontal object detection and 18 for oriented object detection that were recently researched and developed to conduct classifications, surveys and assessments. We also provide a taxonomy mind map to assist readers in gaining an overview of the methods we surveyed. Additional, we examine 17 aerial image datasets collected from platforms and in various locations worldwide. These are typical datasets used to serve two primary tasks: Horizontal Object Detection and Oriented Object Detection. Then, we compared the experimental resultsof the previously introduced methods on four typical datasets: VisDrone, UAVDT, DOTAv1.0, and HRSC2016. The above datasets present diversity in the number of classes and the complexity of real-life conditions. Furthermore, they have been tested with various methods and shown remarkable results. For the reasons mentioned, we have enough basis to confirm that the comparison results are objective so that other research groups can make accurate comments and assessments on experimental methods on each dataset. We believe that updating and synthesizing the latest research will provide complete and comprehensive information, helping researchers to have more survey material for the development of the field of object detection in the future.

This paper addresses the challenge of 3D object detection from a single panoramic image under severe deformation. The advent of the two-stage approach has impelled significant progress in 3D object detection. However, most available methods only can localize region proposals by a single-scale architecture network, which are sensitive to deformation and distortion. To address this issue, we propose a multi-scale convolutional neural network (MSCNN) to estimate the 3D pose of an object. To be specific, the proposed MSCNN consists of three steps for effectively detecting the distorted object on the panoramic images. The MSCNN contains the CycleGAN network that converts rectilinear images into panoramas, a fused framework that improves both accuracy and speed for object detection, and an adversarial spatial transformer network (ASTN) that extracts the deformation features of the object on panoramic images. Additionally, we recover the 3D pose of the object using a coordinate projection and a 3D bounding box. Extensive experiments demonstrate that the proposed method can achieve a 3D detection accuracy of 38.7% in high-resolution panoramic images, which is higher than the current state-of-the-art algorithm of 5.2%. Moreover, the speed of detection is only about 0.6 seconds per image, which is six times faster than Faster R-CNN (COCO). The code will be available at https://github.com/Yanhui-He.

To address the issues of low diagnostic accuracy in traditional rolling bearing fault diagnosis models and the ineffective extraction of spatial and temporal features from vibration signals, this paper proposes a rolling bearing fault diagnosis method based on the improved sparrow search algorithm optimized VMD and multi-scale convolutional neural networks. First, the improved sparrow search algorithm is employed to adaptively optimize the penalty parameter and mode count in variational modal decomposition (VMD). This achieves finer frequency band segmentation and effectively suppresses energy leakage, thereby yielding high quality frequency domain representations. Second, a multi-scale convolutional neural networks (MSCNN) is constructed, with feature level fusion implemented. Subsequently, a bidirectional long short-term memory networks (BiLSTM) is introduced to model the temporal dependencies of the fused features, enabling fault mode learning. A softmax layer is employed to achieve multi-class classification. Finally experimental results and comparisons based on the CWRU bearing dataset demonstrate the effectiveness of the proposed method in the rolling bearing fault classification task, providing significant application value for achieving efficient and reliable bearing fault detection.

Due to the complex structure of the traditional SSD network model and the large amount of parameters, it is not suitable for porting to mobile or embedded platforms, so the LW-SSD vehicle detection algorithm is proposed, this paper redesigns the basic feature extraction network of SSD, using MobileNet feature extraction network instead of VGG16, for the original MobileNet to improve, remove the original MobileNet network average pooling layer and Softmax layer, while reducing the MobileNet network from the original 28 layers to 13 layers. Then, on the basis of the subsequent network, feature maps of different scales are scaled, residual mapping and deep feature fusion operations, and the low-level position information and high-level semantic information are effectively combined to obtain new feature maps of different scales. The designed network model is trained using the Tensorflow framework, and the test effect of the network model is evaluated. The experimental results show that the accuracy of the network model designed in this paper is 78.76%, which is 9.27% lower than the original SSD network model. However, the amount of parameters is only 1/4 of the SSD network model. At the expense of partial accuracy, the design of a lightweight network model is exchanged, which lays the foundation for subsequent development on mobile or embedded platforms.

Learning multiscale and deep representations for classifying remotely sensed imagery

Mobile devices usually mount a depth sensor to resolve ill-posed problems, like salient object detection on cluttered background. The main barrier of exploring RGBD data is to handle the information from two different modalities. To cope with this problem, in this paper, we propose a boundary-aware cross-modal fusion network for RGBD salient object detection. In particular, to enhance the fusion of color and depth features, we present a cross-modal feature sampling module to balance the contribution of the RGB and depth features based on the statistics of their channel values. In addition, in our multi-scale dense fusion network architecture, we not only incorporate edge-sensitive losses to preserve the boundary of the detected salient region, but also refine its structure by merging the estimated saliency maps of different scales. We accomplish the multi-scale saliency map merging using two alternative methods which produce refined saliency maps via per-pixel weighted combination and an encoder-decoder network. Extensive experimental evaluations demonstrate that our proposed framework can achieve the state-of-the-art performance on several public RGBD-based datasets.

Compared with general object detection with horizontal bounding boxes in natural images, oriented object detection in remote sensing images is an active and challenging research topic as objects are usually displayed in arbitrary orientations. To model the variant orientations of oriented objects, general CNN-based methods usually adopt more parameters or well-designed modules, which are often complex and inefficient. To address this issue, the detector requires two key components to deal with: (i) generating oriented proposals in a light-weight network to achieve effective representation of arbitrarily oriented objects; (ii) extracting the rotation-invariant feature map in both spatial and orientation dimensions. In this paper, we propose a novel, lightweight rotated region proposal network to produce arbitrary-oriented proposals by sliding two vertexes only on adjacent sides and adopt a simple yet effective representation to describe oriented objects. This may decrease the complexity of modeling orientation information. Meanwhile, we adopt the rotation-equivariant backbone to generate the feature map with explicit orientation channel information and utilize the spatial and orientation modules to obtain completely rotation-invariant features in both dimensions. Without tricks, extensive experiments performed on three challenging datasets DOTA-v1.0, DOTA-v1.5 and HRSC2016 demonstrate that our proposed method can reach state-of-the-art accuracy while reducing the model size by 40% in comparison with the previous best method.

Single Shot MultiBox Detector (SSD) method using multi-scale feature maps for object detection, showing outstanding performance in object detection task. However, as a one-stage detection method, it’s difficult for SSD methods to quickly notice significant areas of objects in the image. In the SSD network structure, feature maps of different scales are used to independently predict object, and there is a lack of interaction between low-level feature maps and high-level feature maps. In this paper we propose an enhanced SSD method using interactive multi-scale attention features (MA-SSD). Our method uses the attention mechanism to generate attention features of multiple scales and adds it to the original detection branch of the SSD method, which effectively enhances the feature representation ability and improves the detection accuracy. At the same time, the feature of different detection scales interacts with each other, and all the detection branches in our method have a parallel structure, which ensures the detection efficiency. Our proposed method achieves competitive performance on the public dataset PascalVOC.

Small object detection has important application value in the fields of autonomous driving and drone scene analysis. As one of the most advanced object detection algorithms, YOLOv3 suffers some challenges when detecting small objects, such as the problem of detection failure of small objects and occluded objects. To solve these problems, an improved YOLOv3 algorithm for small object detection is proposed. In the proposed method, the dilated convolutions mish (DCM) module is introduced into the backbone network of YOLOv3 to improve the feature expression ability by fusing the feature maps of different receptive fields. In the neck network of YOLOv3, the convolutional block attention module (CBAM) and multi-scale fusion module are introduced to select the important information for small object detection in the shallow network, suppress the uncritical information, and use the fusion module to fuse the feature maps of different scales, so as to improve the detection accuracy of the algorithm. In addition, the Soft-NMS and Complete-IOU (ClOU) strategies are applied to candidate frame screening, which improves the accuracy of the algorithm for the detection of occluded objects. The experimental results on MS COCO2017, VOC2007, VOC2012 datasets and the ablation experiments on MS COCO2017 datasets demonstrate the effectiveness of the proposed method.The experimental results show that the proposed method achieves better accuracy in small object detection than the original YOLOv3 model.

Cartographic generalization of the rivers is a very complex process in cartography. Cartographic generalization involves reduction of water flows, their length, width and the density of river basin with the reducing of the map scale. In this study we analyzed the ratio of the density of river basin and the size of the degree of generalization of the maps of different scale. The example is the Velika Morava river basin. Based on the results given here we get to the basic regularities of proportion of the degree of generalization of the density of river basin.

Background: New drugs are costly, time-consuming, and often accompanied by safety concerns. With the development of deep learning, computer-aided drug design has become more mainstream, and convolutional neural networks and graph neural networks have been widely used for drug–target affinity (DTA) prediction. Objective: The paper proposes a method of predicting DTA using graph convolutional networks and multiscale convolutional neural networks. Methods: We construct drug molecules into graph representation vectors and learn feature expressions through graph attention networks and graph convolutional networks. A three-branch convolutional neural network learns the local and global features of protein sequences, and the two feature representations are merged into a regression module to predict the DTA. Results: We present a novel model to predict DTA, with a 2.5% improvement in the consistency index and a 21% accuracy improvement in terms of the mean squared error on the Davis dataset compared to DeepDTA. Morever, our method outperformed other mainstream DTA prediction models namely, GANsDTA, WideDTA, GraphDTA and DeepAffinity. Conclusion: The results showed that the use of multiscale convolutional neural networks was better than a single-branched convolutional neural network at capturing protein signatures and the use of graphs to express drug molecules yielded better results.

In this paper, an object detection model based on MFF-RPN and Multi-scale CNN is proposed. Firstly, the region proposal network based on multi-level feature fusion (MFF-RPN) is presented to extract the candidate proposals. Secondly, a convolutional neural network (CNN) with different scale convolution kernels is conducted to extract features adaptively. Finally, multi-task loss is employed to establish a complex mapping between image object features and object detection mode. The experimental results prove that the proposed algorithm gets better classification performance and higher detection accuracy.

Side-channel attacks based on attention mechanism and multi-scale convolutional neural network

Automatic distinction between COVID-19 and common pneumonia using multi-scale convolutional neural network on chest CT scans