PSTNet: object detection in remote sensing images with point supervision and object templates

High tracking frame rates have been achieved based on traditional tracking methods which however would fail due to drifts of the object template or model, especially when the object disappears from the camera’s field of view. To deal with it, tracking-and-detection-combination has become more and more popular for long-term unknown object tracking, whose detector almost does not drift and can regain the disappeared object when it comes back. However, for online machine learning and multiscale object detection, expensive computing resources and time are required. So it is not a good idea to combine tracking and detection sequentially like Tracking-Learning-Detection algorithm. Inspired from parallel tracking and mapping, this article proposes a framework of parallel tracking and detection for unknown object tracking. The object tracking algorithm is split into two separate tasks—tracking and detection which can be processed in two different threads, respectively. One thread is used to deal with the tracking between consecutive frames with a high processing speed. The other thread runs online learning algorithms to construct a discriminative model for object detection. Using our proposed framework, high tracking frame rates and the ability of correcting and recovering the failed tracker can be combined effectively. Furthermore, our framework provides open interfaces to integrate state-of-the-art object tracking and detection algorithms. We carry out an evaluation of several popular tracking and detection algorithms using the proposed framework. The experimental results show that different tracking and detection algorithms can be integrated and compared effectively by our proposed framework, and robust and fast long-term object tracking can be realized.

Point cloud-based three-dimensional (3D) object detection is a critical task in autonomous driving, robotics, and augmented reality, where accurate localization and classification of objects are essential under diverse and challenging scenarios. This work introduces a point cloud-based multi-target 3D object detection framework using Light Detection and Ranging (LiDAR) sensors. The key contribution lies in improving the encoding technique to preserve spatial information and adapt to varying point densities, enabling efficient processing of raw LiDAR data for robust 3D detection. Specifically, we design an efficient network architecture that incorporates a single unified classification head to jointly handle positive and negative samples, simplifying the design and improving training stability. Our model detects all classes (car, pedestrian, cyclist) simultaneously within a single network, enhancing computational efficiency while avoiding separate networks for each class. Furthermore, the proposed feature extraction pipeline combines a voxel-based encoder with a sparsely embedded convolutional backbone and a feature pyramid network, facilitating multi-scale feature representation and effective detection of objects at different scales. This encoding method facilitates the efficient processing of raw LiDAR data, enabling accurate object detection across diverse scenarios. As a result, our model achieves state-of-the-art performance on the KITTI dataset, surpassing baseline methods such as PointPillars and VoxelNet by delivering superior Average Precision (AP) across all difficulty levels for the car class in both bird’s-eye view (BEV) and 3D object detection tasks. Specifically, it achieves AP40 scores of 92.84%, 88.63%, and 85.68% in BEV detection and 87.84%, 76.65%, and 73.60% in 3D detection for Easy, Moderate, and Hard levels, respectively. These results highlight the effectiveness of our encoding technique in enhancing model efficiency and detection accuracy across diverse scenarios.

A background-limited optical system is to view an object through a turbulent medium. The threshold systems for detecting and resolving specific details of the object are derived and their probabilities of correct detection and resolution evaluated. The variance of a maximum-likelihood estimate of the position of the object is calculated for large signal-to-noise ratio. Specific numerical results are given for turbulence that produces a gaussian mutual-coherence function in light from a point object. A model of such turbulence, depicting it as concentrated in one or more thin, homogeneous phase screens, is proposed.

Detection of point objects on a sequence of video images is an important step in optical monitoring systems for UAVs, since they allow them to be detected on the maximum range. This task is quite complicated because the intensity of the object in the image is low due to the low signal / noise ratio (SNR). Decisive statistics of marks are used to increase the effectiveness of object tracking. The paper presents a method for detecting point objects on a video images sequence according to Wald test using distributions of decisive statistics in likelihood ratio. Results of statistical modeling show that the synthesized algorithm reduces the detection time and reduce computational complexity compared with known algorithms detect a moving object IMM-PDAF-AI, which uses the decisive statistics of marks and IMM-PDAF, which does not use decisive statistics of marks.

Laser scanning system provides an efficient solution to rapidly acquire 3D information of large-scale scenes. Point clouds collected by laser scanning systems contain numerous objects with significant disparities in size, complicated and incomplete structures, holes, varied point densities, and huge data volumes, raising great challenges for automated point clouds registration, segmentation, and object detection. The dissertation presents a hierarchical merging based multi-platform point clouds registration algorithm to align MLS point clouds and unordered TLS point clouds from various scenes and validates its performance on nine challenging datasets. The algorithm improves the efficiency and accuracy of point cloud registration and enhances the registration ability of the algorithm for low-overlap and high-symmetry point clouds.

We propose octree-based transformers, named OctFormer, for 3D point cloud learning. OctFormer can not only serve as a general and effective backbone for 3D point cloud segmentation and object detection but also have linear complexity and is scalable for large-scale point clouds. The key challenge in applying transformers to point clouds is reducing the quadratic, thus overwhelming, computation complexity of attentions. To combat this issue, several works divide point clouds into non-overlapping windows and constrain attentions in each local window. However, the point number in each window varies greatly, impeding the efficient execution on GPU. Observing that attentions are robust to the shapes of local windows, we propose a novel octree attention, which leverages sorted shuffled keys of octrees to partition point clouds into local windows containing a fixed number of points while permitting shapes of windows to change freely. And we also introduce dilated octree attention to expand the receptive field further. Our octree attention can be implemented in 10 lines of code with open-sourced libraries and runs 17 times faster than other point cloud attentions when the point number exceeds 200 k. Built upon the octree attention, OctFormer can be easily scaled up and achieves state-of-the-art performances on a series of 3D semantic segmentation and 3D object detection benchmarks, surpassing previous sparse-voxel-based CNNs and point cloud transformers in terms of both efficiency and effectiveness. Notably, on the challenging ScanNet200 dataset, OctFormer outperforms sparse-voxel-based CNNs by 7.3 in mIoU. Our code and trained models are available at https://wang-ps.github.io/octformer.

The results of developing an intelligent model of radar object images for surveillance radars are presented. The relevance of this work deals with the development of algorithm for automatic processing images of radar objects that provide effective detection of weak true signals due to the accumulation of signal and logical information in the analyzed cell and in its surroundings under interferences. The improvement of air safety tools and the automation of air traffic management processes require effective procedures to process signal information. The issues of more complete use and qualitative improvement of the information-processing capabilities of control systems are also topical, especially in difficult conditions of interfering signals. The basis of this study is the idea of using an intellectual model of radar object images for automatic decision-making on detection and recognition of radar objects, built on the space of semantic features. The main result is optical object recognition, similar to how an expert can easily recognize aerial objects and their types when viewing radar object images. Based on semantic features intelligent model of radar object images has been developed, which makes it possible to effectively detect and classify aerial objects. It is worth noting that the characteristic description of intelligent model of radar object images for point, extended, moving and stationary radar objects is the mathematical description of procedures and relationships at perception and analysis of signals in the form of distinguishing features or properties. As a result, various virtual images of radar object are generated in the form of spatial-semantic and spectral-semantic models. The main features and structural elements of the model are given. It is shown that the advantages of this model are related to the possibility of characteristic description of the radar object images using the algebra of finite predicates.

For camera movement causes moving objects detecting and tracking problems under complex background, we propose an adaptive object detection scope algorithm based on SIFT features. Firstly, let camera stationary and obtain three images to detect the moving object by using three-frame-difference method, then extract the object SIFT features. Secondly, according to the location and displacement of the object in the dynamic background, we determine the detection scope which matches the object well and obtain the minimum rectangle which can surround the right matching points in the detection scope, and then update the object template. The algorithm avoids the analysis of the complex relative motion between the object and the background, and reduces mismatch points and the calculation amount. This algorithm can quickly and accurately track the object without occlusion, and performs robust in small occlusion case.

针对当前激光雷达（LiDAR）三维目标检测算法存在对小目标物体检测效果较差的问题，提出一种融合多尺度特征和分组卷积的三维目标检测算法。该算法以PV-RCNN++为基准网络，首先使用三维多尺度特征网络融合不同空间分辨率的特征，得到具有不同空间语义信息的三维特征体，以增强模型对不同尺度物体的特征提取能力和优化检测效果；然后，在二维特征图上，对不同通道进行分组，每组使用不同尺寸的卷积核进行卷积，得到具有不同尺寸感受野信息的二维特征；最后，使用SimAM注意力机制提高二阶段前景点的特征权重，使前景点在候选框细化阶段发挥更大的作用。在KITTI数据集上的实验结果表明，所提算法优于当前主流的目标检测算法，与基准网络PV-RCNN++对比，行人在简单、中等、困难情形下的检测平均精度分别提升了3.12百分点、3.95百分点、3.70百分点，骑行者在3种情形下的检测平均精度分别提升了4.45百分点、3.52百分点、3.08百分点。

Due to illumination variation under different lighting conditions, texture-less objects have posed significant challenges to visual object localization algorithms fur robot grasping. We propose a method to determine the GD pose of both textured and texture-less objects from a single RGB-D image with a Kinect. First, we apply hierarchical clustering strategy to pre-process the point cloud of a scene. Then, we achieve the 3D object detection by comparing the diameter between clustering point cloud and object model. Last, the rough pose of object is estimated through Hough voting and the estimation result is refined by ICP (Iterative Closest Point). Experimental results show that the accumulation error between the model and the corresponding point in the scene is less than Gmm and the attitude error is less than 1.5°. The average detection accuracy rate of the proposed method reaches 97% which can satisfy the grasping requirements of the manipulator. We also demonstrate that our approach has good performance in dynamic lighting conditions.

While salient object detection (SOD) on 2D images has been extensively studied, there is very little SOD work on 3D measurement surfaces. We propose an effective point transformer-based SOD network for 3D measurement point clouds, termed PSOD-Net. PSOD-Net is an encoder-decoder network that takes full advantage of transformers to model the contextual information in both multi-scale point- and scene-wise manners. In the encoder, we develop a Point Context Transformer (PCT) module to capture region contextual features at the point level; PCT contains two different transformers to excavate the relationship among points. In the decoder, we develop a Scene Context Transformer (SCT) module to learn context representations at the scene level; SCT contains both Upsampling-and-Transformer blocks and Multi-context Aggregation units to integrate the global semantic and multi-level features from the encoder into the global scene context. Experiments show clear improvements of PSOD-Net over its competitors and validate that PSOD-Net is more robust to challenging cases such as small objects, multiple objects, and objects with complex structures. Code is available at: https://github.com/ZeyongWei/PSOD-Net.

In this paper, we propose a data-driven foreground object detection technique which can detect foreground objects from a moving camera. We propose to build a data-driven consensus foreground object template (CFOT) and then detect the foreground object region in each frame. The proposed foreground object detection technique is equipped with the following functions: (1) the ability to detect the foreground object captured by a fast moving camera ; (2) the ability to detect a low contrast (spatially/temporally) foreground object; and (3) the ability to detect a foreground object from a dynamic background. There are three contributions of our method: (1) a newly proposed data-driven foreground region decision process for generating the CFOT has been shown robust and efficient; (2) a foreground object probability is proposed for properly dealing with the imperfect initial foreground region estimations; and (3) a CFOT is generated for precise foreground object detection.

Target detection and tracking in real-time videos are very important and yet difficult for many applications. Numerous detection and tracking techniques have been proposed, typically by imposing some constraints on the motion and image to simplify the problem depending on the application and environment. This paper focuses on target detection and tracking in airborne videos, in which not much simplification can be made. We have recently proposed a combined/switching detection and tracking method which is based on the combination of a spatio-temporal segmentation and statistical snake model. This paper improves the statistical snake model by incorporating both edge and region information and enhancing the snake contour deformation. A more complex motion model is used to improve the accuracy of object detection and size classification. Mean-shift is integrated into the proposed combined method to track small point objects and deal with the problem of object disappearance-reappearance. Testing results using real UAV videos are provided.