Optimal Partition Assignment for Universal Object Detection

Object detection has received significant interest in the research field of computer vision and is widely used in human-centric applications. The occlusion problem is a frequent obstacle that degrades detection quality. In this paper, we propose a novel object detection framework targeting robust object detection in occlusion. The proposed deep learning-based network consists mainly of two parts: 1) object detection framework, which classifies the object categories and localizes the object location and 2) plug-in bounding-box (BB) estimator, which estimates the object and occlusion region from the feature map of the backbone network and the corresponding critic network for evaluating the predicted BB map. The BB estimator and the critic network are the plug-in modules added to the object detection framework and learned competitively with adversarial manner. As the plug-in BB estimator is learned to estimate the BB map containing the object and occlusion pattern information, the backbone network can embed this information to enable robust detection under occlusion in the test phase. The comprehensive experimental results on the PASCAL VOC, MS COCO, and KITTI dataset showed that the performance is improved with the plug-in BB-Critic network by predicting and criticizing object and occlusion in general generic object detection framework.

In this paper, we propose SparseDet for end-to-end 3D object detection from point cloud. Existing works on 3D object detection rely on dense object candidates over all locations in a 3D or 2D grid following the mainstream methods for object detection in 2D images. However, this dense paradigm requires expertise in data to fulfill the gap between label and detection. As a new detection paradigm, SparseDet maintains a fixed set of learnable proposals to represent latent candidates and directly perform classification and localization for 3D objects through stacked transformers. It demonstrates that effective 3D object detection can be achieved with none of post-processing such as redundant removal and non-maximum suppression. With a properly designed network, SparseDet achieves highly competitive detection accuracy while running with a more efficient speed of 34.5 FPS. We believe this end-to-end paradigm of SparseDet will inspire new thinking on the sparsity of 3D object detection.

Accurate object detection is a fundamental requirement for autonomous systems to operate in dynamic urban environments. Considering the intricacy of the environment and the occlusion of objects, various point cloud based three-dimensional (3D) object detection methods have been proposed, such as point-based or voxel-based methods. In this paper, a feature selection mechanism is proposed in a voxel-based method to generate bird’s eye view (BEV) feature maps from the original point cloud. For the 3D Backbone, the U-shaped structure and single scale feature selection module are combined. After combining high-level semantics and low-level fine-grained features, the optimized features after the BEV feature maps are applied to region of interest (RoI) refinement, so that the voxel features can better serve the subsequent 3D object detection. The experimental results on KITTI dataset show higher 3D object detection accuracy compared to the state-of-the-art 3D detection methods, which reflects the effectiveness of the proposed architecture.

With the rapid development of computer vision technology and the maturity of 2D object detection technology, 3D object detection has attracted more and more attention of scholars. As an extension of 2D object detection, 3D object detection can not only detect the object category and position in the image, but also output the spatial position and pose angle of the object in a certain 3D coordinate system. It has a considerable application prospects in automatic driving, robot operation and augmented reality. However, 3D object detection is much more difficult than 2D object detection, and it is very difficult to make and label the data set, which has high hardware requirements. At present, there is no dominant algorithm, and scholars use a variety of methods. In order to solve the above problems, this paper proposes a method of making and synthesising 3D object detection data sets. Based on the improved 2D object detection algorithm, 3D object detection of single RGB image is realised.

Non-maximum suppression (NMS) is an indispensable post-processing step in object detection. After a picture is detected by the target, a large number of redundant anchors will be obtained, so we need to go through NMS to process these repeated anchors. Its essence is to extract the target detection frame with high confidence and suppress the false detection frame with low confidence. The biggest problem in the NMS algorithm is that it completely removes adjacent low-confidence detection frames. In some images with dense targets, this is very likely to cause missed detection and false detection. Therefore, we proposed the Simple-NMS algorithm, which adds two new thresholds to the original NMS, and sets a constraint condition different from the original based on the new threshold, and does not change the complexity of the traditional NMS algorithm, which can be very good Improve the effectiveness of NMS missed targets. The new NMS algorithm is improved on the standard data sets PASAL VOC2007 and MS COCO. In addition, the algorithm is simple and efficient, and can be easily integrated into any other object detection process.

This study evaluates the predictive performance of advanced machine learning models, including DeepBoost, XGBoost, CatBoost, RF, and MLP, in estimating the Ω2, Ω4, and Ω6 parameters based on a comprehensive set of input variables. Among the models, DeepBoost consistently demonstrated the best performance across the training and testing phases. For the Ω2 prediction, DeepBoost achieved an R2 of 0.974 and accuracy of 99.895% in the training phase, with corresponding values of 0.971 and 99.902% in the testing phase. In comparison, XGBoost ranked second with an R2 of 0.929 and accuracy of 99.870% during testing. For Ω4, DeepBoost achieved a training phase R2 of 0.955 and accuracy of 99.846%, while the testing phase results included an R2 of 0.945 and accuracy of 99.951%. Similar trends were observed for Ω6, where DeepBoost obtained near-perfect training phase results (R2 = 0.997, accuracy = 99.968%) and testing phase performance (R2 = 0.994, accuracy = 99.946%). These findings are further supported by violin plots and correlation analyses, underscoring DeepBoost’s superior predictive reliability and generalization capabilities. This work highlights the importance of model selection in predictive tasks and demonstrates the potential of machine learning for capturing complex relationships in data.

Point cloud-based three-dimensional (3D) object detection is a critical task in autonomous driving, robotics, and augmented reality, where accurate localization and classification of objects are essential under diverse and challenging scenarios. This work introduces a point cloud-based multi-target 3D object detection framework using Light Detection and Ranging (LiDAR) sensors. The key contribution lies in improving the encoding technique to preserve spatial information and adapt to varying point densities, enabling efficient processing of raw LiDAR data for robust 3D detection. Specifically, we design an efficient network architecture that incorporates a single unified classification head to jointly handle positive and negative samples, simplifying the design and improving training stability. Our model detects all classes (car, pedestrian, cyclist) simultaneously within a single network, enhancing computational efficiency while avoiding separate networks for each class. Furthermore, the proposed feature extraction pipeline combines a voxel-based encoder with a sparsely embedded convolutional backbone and a feature pyramid network, facilitating multi-scale feature representation and effective detection of objects at different scales. This encoding method facilitates the efficient processing of raw LiDAR data, enabling accurate object detection across diverse scenarios. As a result, our model achieves state-of-the-art performance on the KITTI dataset, surpassing baseline methods such as PointPillars and VoxelNet by delivering superior Average Precision (AP) across all difficulty levels for the car class in both bird’s-eye view (BEV) and 3D object detection tasks. Specifically, it achieves AP40 scores of 92.84%, 88.63%, and 85.68% in BEV detection and 87.84%, 76.65%, and 73.60% in 3D detection for Easy, Moderate, and Hard levels, respectively. These results highlight the effectiveness of our encoding technique in enhancing model efficiency and detection accuracy across diverse scenarios.

Segmenting or detecting objects in sparse Lidar point clouds are two important tasks in autonomous driving to allow a vehicle to act safely in its 3D environment. The best performing methods in 3D semantic segmentation or object detection rely on a large amount of annotated data. Yet annotating 3D Lidar data for these tasks is tedious and costly. In this context, we propose a self-supervised pretraining method for 3D perception models that is tailored to autonomous driving data. Specifically, we leverage the availability of synchronized and calibrated image and Lidar sensors in autonomous driving setups for distilling self-supervised pre-trained image representations into 3D models. Hence, our method does not require any point cloud nor image annotations. The keyingredient of our method is the use of superpixels which are used to pool 3D point features and 2D pixel features in visually similar regions. We then train a 3D network on the self-supervised task of matching these pooled point features with the corresponding pooled image pixel features. The advantages of contrasting regions obtained by superpixels are that: (1) grouping together pixels and points of visually coherent regions leads to a more meaningful contrastive task that produces features well adapted to 3D semantic segmentation and 3D object detection; (2) all the different regions have the same weight in the contrastive loss regardless of the number of 3D points sampled in these regions; (3) it mitigates the noise produced by incorrect matching of points and pixels due to occlusions between the different sensors. Extensive experiments on autonomous driving datasets demonstrate the ability of our image-to-Lidar distillation strategy to produce 3D representations that transfer well on semantic segmentation and object detection tasks.

Confluence is a novel non-Intersection over Union (IoU) alternative to Non-Maxima Suppression (NMS) in bounding box post-processing in object detection. It overcomes the inherent limitations of IoU-based NMS variants to provide a more stable, consistent predictor of bounding box clustering by using a normalized Manhattan Distance inspired proximity metric to represent bounding box clustering. Unlike Greedy and Soft NMS, it does not rely solely on classification confidence scores to select optimal bounding boxes, instead selecting the box which is closest to every other box within a given cluster and removing highly confluent neighboring boxes. Confluence is experimentally validated on the MS COCO and CrowdHuman benchmarks, improving Average Precision by 0.2--2.7% and 1--3.8% respectively and Average Recall by 1.3--9.3 and 2.4--7.3% when compared against Greedy and Soft-NMS variants. Quantitative results are supported by extensive qualitative analysis and threshold sensitivity analysis experiments support the conclusion that Confluence is more robust than NMS variants. Confluence represents a paradigm shift in bounding box processing, with potential to replace IoU in bounding box regression processes.

Towards accurate dense pedestrian detection via occlusion-prediction aware label assignment and hierarchical-NMS

In this paper, we study object detection using a large pool of unlabeled images and only a few labeled images per category, named "few-example object detection". The key challenge consists in generating trustworthy training samples as many as possible from the pool. Using few training examples as seeds, our method iterates between model training and high-confidence sample selection. In training, easy samples are generated first and, then the poorly initialized model undergoes improvement. As the model becomes more discriminative, challenging but reliable samples are selected. After that, another round of model improvement takes place. To further improve the precision and recall of the generated training samples, we embed multiple detection models in our framework, which has proven to outperform the single model baseline and the model ensemble method. Experiments on PASCAL VOC'07, MS COCO'14, and ILSVRC'13 indicate that by using as few as three or four samples selected for each category, our method produces very competitive results when compared to the state-of-the-art weakly-supervised approaches using a large number of image-level labels.

The real-time object detection has recently achieved a series of exciting effect on several dataset like Pascal VOC and MS COCO. These highly-developed algorithms enable real-time image processing from a camera sensing system to be implemented. This paper proposes an object detection model, which is named Adaptive Cascade Single-Shot detector (ACSD) based on earlier Single-Shot Multi box Detector (SSD). The model makes use of adaptive anchor boxes via training dataset statistic experience and add multi-scale cascade layer to reduce over-fitting due to exponentially vanishing positive samples, besides, we proposed a way using spectral clustering to make detection dataset. This model aims to improve the detection accuracy and the processing speed in real-time detection tasks. Experiments include several detection tasks on different datasets while the results demonstrate the improvement of ACSD in terms of effectiveness and efficiency. Working performance of the ACSD is verified by comparing it to several state-of-the-art approaches to guarantee it a promising method for the development of the camera sensing system.

In recent years, 3D object detection has attracted considerable interest from researchers due to its crucial importance in scene understanding tasks for mobile robots. The task of 3D object detection is to determine the position, orientation, scale, and category of objects in a scene, with multimodal data inputs. These inputs consist of two distinct types: images and point clouds. Images provide colour information about the scene, while point clouds deliver geometric information. The fusion of these two diverse data types can enhance the detection of 3D objects by allowing them to complement each other. In this paper, a review of real-time multimodal 3D object detection from the perspective of multimodal sensor fusion is presented. It is important to note that previous reviews of multimodal 3D object detection have been limited to autonomous driving scenarios. In contrast, this paper reviews multimodal 3D object detection approaches in both indoor and outdoor scenarios. In addition, the popular datasets and assessment metrics employed for 3D object detection are described, as well as the issues and future prospects of 3D object detection.

Multi-camera 3D object detection aims to detect and localize objects in 3D space using multiple cameras, which has attracted more attention due to its cost-effectiveness trade-off. However, these methods often struggle with the lack of accurate depth estimation caused by the natural weakness of the camera in ranging. Recently, multi-modal fusion and knowledge distillation methods for 3D object detection have been proposed to solve this problem, which are time-consuming during the training phase and not friendly to memory cost. In light of this, we propose PromptDet, a lightweight yet effective 3D object detection framework motivated by the success of prompt learning in 2D foundation model. Our proposed framework, PromptDet, comprises two integral components: a general camera-based detection module, exemplified by models like BEVDet and BEVDepth, and a LiDAR-assisted prompter. The LiDAR-assisted prompter leverages the LiDAR points as a complementary signal, enriched with a minimal set of additional trainable parameters. Notably, our framework is flexible due to our prompt-like design, which can not only be used as a lightweight multi-modal fusion method but also as a camera-only method for 3D object detection during the inference phase. Extensive experiments on nuScenes validate the effectiveness of the proposed PromptDet. As a multi-modal detector, PromptDet improves the mAP and NDS by at most 22.8% and 21.1% with fewer than 2% extra parameters compared with the camera-only baseline. Without LiDAR points, PromptDet still achieves an improvement of at most 2.4% mAP and 4.0% NDS with almost no impact on camera detection inference time. We will release our code.

The development of object detection networks has reached a high point, and there have been significant improvements in accuracy and detection speed. Object detection is widely used in intelligent robots, self‐driving cars, and other edge‐intelligent terminals. Unfortunately, when a detector is allowed to learn new objects in an unfamiliar environment, it can catastrophically forget the objects it has already learned. In particular, reliable and stable knowledge cannot be extracted from old models. Based on this, a new multinetwork mean distillation loss function for open‐world domain incremental object detection is presented. To better extract reliable and stable knowledge from old models, we enhanced the distillation output of the detector with a ResNet50 backbone and an output RoI head. The distillation output of the intermediate RPN is softened by adaptive distillation. To obtain more stable results, the ResNet50 backbone and RPN on the channel are zero‐averaged. Various incremental steps and stability experiments are performed on two benchmark datasets, PASCAL VOC and MS COCO. The experimental results show the excellent performance of our method in different experimental scenarios, and it is superior to the most advanced methods. For example, in the setting of the batch task, incremental object detection on the PASCAL VOC and MS COCO datasets is improved by 3.4% and 2.1%, respectively.