Infrared Small Target Detection Based on Singularity Analysis and Constrained Random Walker

Detecting multiple salient objects in complex scenes is a challenging task. In this paper, we present a novel method to detect salient objects in images. The proposed method is based on the general ‘center-surround’ visual attention mechanism and the spatial frequency response of the human visual system (HVS). The saliency computation is performed in a statistical way. This method is modeled following three biologically inspired principles and compute saliency by two ‘scatter matrices’ which are used to measure the variability within and between two classes, i.e., the center and surrounding regions, respectively. In order to detect multiple salient objects of different sizes in a scene, the saliency of a pixel is estimated via its saliency support region which is defined as the most salient region centered at the pixel. Compliance with human perceptual characteristics enables the proposed method to detect salient objects in complex scenes and predict human fixations. Experimental results on three eye tracking datasets verify the effectiveness of the method and show that the proposed method outperforms the state-of-the-art methods on the visual saliency detection task.

A pooling-based feature pyramid network for salient object detection

Perceptual localization and focus refinement network for RGB-D salient object detection

Six degrees-of-freedom (6D) object pose estimation plays an important role in pattern recognition of fields such as robotics and augmented reality. However, there are issues with low accuracy and real-time performance of 6D object pose estimation in complex scenes. To address these challenges, in this article, RFF-PoseNet (a 6D object pose estimation network based on robust feature fusion) is proposed for complex scenes. Firstly, a more lightweight Ghost module is used to replace the convolutional blocks in the feature extraction network. Then, a pyramid pooling module is added to the semantic label branch of PoseCNN to fuse the features of different pooling layers and enhance the network’s ability to capture information about objects in complex scenes and the correlations between contextual information. Finally, a pose regression and optimization module is utilized to further improve object pose estimation in complex scenes. Simulation experiments conducted on the YCB-Video and Occlusion LineMOD datasets show that the RFF-PoseNet algorithm can strengthen the correlation of features between different levels and the recognition ability of unclear targets, thereby achieving excellent accuracy and real-time performance, as well as strong robustness.

Research on Infrared Dim and Small Target Detection Algorithm Based on Low-Rank Tensor Recovery

In the field of multimodal robotics, achieving comprehensive and accurate perception of the surrounding environment is a highly sought-after objective. However, current methods still have limitations in motion keypoint detection, especially in scenarios involving small target detection and complex scenes. To address these challenges, we propose an innovative approach known as YOLOv8-PoseBoost. This method introduces the Channel Attention Module (CBAM) to enhance the network’s focus on small targets, thereby increasing sensitivity to small target individuals. Additionally, we employ multiple scale detection heads, enabling the algorithm to comprehensively detect individuals of varying sizes in images. The incorporation of cross-level connectivity channels further enhances the fusion of features between shallow and deep networks, reducing the rate of missed detections for small target individuals. We also introduce a Scale Invariant Intersection over Union (SIoU) redefined bounding box regression localization loss function, which accelerates model training convergence and improves detection accuracy. Through a series of experiments, we validate YOLOv8-PoseBoost’s outstanding performance in motion keypoint detection for small targets and complex scenes. This innovative approach provides an effective solution for enhancing the perception and execution capabilities of multimodal robots. It has the potential to drive the development of multimodal robots across various application domains, holding both theoretical and practical significance.

This paper introduces a novel technique for the automatic identification of real world objects in complex scenes. The identification problem requires the comparison of assemblies of image regions with a previously stored view of a known prototype. Shape context representation and matching are employed for recovering point correspondences between the image and the prototype object. This work is inspired from the problem that, a number of ambient conditions such as partial object occlusion and contour distortion, may affect the performance of the matching process and consequently the identification result. In our approach, subtractive clustering is applied in a novel fashion to enable the identification of regions of interest on the complex scene, based on a density metric. In order to increase the robustness of the identifier to mismatches and reduce the computational cost of the process, a selection of the initial suspicious regions is performed. The performance of the identifier has been examined in a great range of image and prototype selections.

Salient object detection in complex scenes and environments is a challenging research topic. Most works focus on RGB-based salient object detection, which limits its performance of real-life applications when confronted with adverse conditions such as dark environments and complex backgrounds. Taking advantage of RGB and thermal infrared(RGBT) images becomes a new research direction for detecting salient objects in complex scenes, since the thermal infrared spectrum provides the complementary information and has been used in many computer vision tasks. However, current research for RGBT salient object detection is limited by the lack of a large-scale dataset and comprehensive benchmark. This work contributes such a RGBT image dataset named VT5000, including 5000 spatially aligned RGBT image pairs with ground truth annotations. VT5000 has 11 challenges collected in different scenes and environments for exploring the robustness of algorithms. With this dataset, we propose a powerful baseline approach, which extracts multilevel features of each modality and aggregates these features of all modalities with the attention mechanism for accurate RGBT salient object detection. To further solve the problem of blur boundaries of salient objects, we also use an edge loss to refine the boundaries. Extensive experiments show that the proposed baseline approach outperforms the state-of-the-art methods on VT5000 dataset and other two public datasets. In addition, we carry out a comprehensive analysis of different algorithms of RGBT salient object detection on VT5000 dataset, and then make several valuable conclusions and provide some potential research directions for RGBT salient object detection. Our new VT5000 dataset is made publicly available at https://github.com/lz118/RGBT-Salient-Object-Detection.

In order to overcome the problems of the small proportion of the object, the variable shooting angle, and height in the UAV aerial image, an anchor-free small object detection algorithm based on the YOLOX network is proposed in this paper. Firstly, aiming at the problem of small object size, a Tri-Head module is proposed to change the size of the detection head to obtain more original information on small objects. Secondly, in order to accurately detect dense continuous small objects in complex scenes, a CBAM-CSP module incorporating the CBAM attention mechanism is proposed to increase the weights for detecting dense continuous regions of interest in images. Then, to alleviate the problems caused by background clutter and pixel blur, BiNet, a jumping multi-scale feature enhancement module, is proposed. This module fully integrates shallow feature information and deep feature information so that detection heads of different scales can obtain enough semantic image information and spatial information. Finally, the experimental results on the VisDrone dataset show that, comparing with the YOLOX network, the mAP is increased by 7.56%±0.06% when the model parameters are reduced by about 5MB, which proves that the proposed algorithm can effectively improve the detection accuracy of small targets.

Compared with RGB images, light field images contain more abundant visual information, which is helpful to accurately detect salient objects in complex scenes. However, most of the existing light field saliency detection methods use single light field data or do not fully consider the differences and complementarities between different light field data, resulting in insufficient multi-modal fusion. To address these issues, a multi-modal feature fusion network is proposed, which makes full use of the rich visual information in the light field images to realize the accurate saliency object detection. The proposed network consists of two parallel subnets, which are used to process the micro-lens image array and all-foucs image respectively. Then the light field refinement module is used to refine the feature map extracted from the micro-lens array stream, and finally the multi-modal feature fusion is realized by the light field attention module to predict saliency objects more accurately. In order to verify the effectiveness of proposed method, extensive comparison with several existing light field saliency detection algorithms is carried on both Lytro-Illum and LFSD datasets. Experimental results show that the proposed method is superior to others in all evaluation metrics on Lytro-Illum dataset, and has desired generalization abilities on LFSD dataset.

Open-vocabulary object detection is an innovative computer vision task capable of widely recognizing and localizing various objects in images. Unlike traditional methods, it can handle diverse object categories and is suitable for real-time applications in dynamic environments. Existing methods typically achieve zero-shot detection capabilities by fusing images and text. However, when semantic discrepancies exist between text and images, biased prediction problems arise, diminishing the effectiveness of semantic guidance. To address these issues, we propose a universal open-vocabulary object detection method that leverages foundational models to provide fine-grained semantic guidance for the detection process. We design a multi-level adaptive scene perception algorithm that captures subtle features of target objects in complex scenes, enabling precise separation of background and foreground. Additionally, we introduce the Text-KAN (T-KAN) model, which integrates textual descriptions with image features. By employing learnable activation functions, it resolves dependencies on linear matrices, enhances text interpretability, corrects semantic biases, and achieves precise alignment between images and text at a fine-grained level. We comprehensively evaluate the performance of our proposed method on existing open-vocabulary benchmarks, conducting experiments on the COCO and LVIS datasets. The results demonstrate significant performance gains in detecting novel categories, highlighting the method’s strong generalization capabilities. This work provides valuable insights and references for advancing the field of open-vocabulary object detection.

We are developing a helper robot that carries out tasks ordered by users through speech. The robot needs a vision system to recognize objects appearing in the orders. However, conventional vision systems cannot recognize objects in complex scenes. They may find many objects and cannot determine which is the target. This paper proposes a method of using a conversation with the user to solve this problem. The robot asks a question to which the user can easily answer and whose answer can efficiently reduce the number of candidate objects. It considers the characteristics of features used for object recognition such as the easiness for humans to specify them by word, generating a user-friendly and efficient sequence of questions. Experimental results show that the robot can detect target objects by asking the questions generated by the method.

Robust detection of moving objects in complex scenes is one of the most challenging issues in computer vision. In this paper, we present a novel texture-wise approach to segment moving objects with codebook and local binary patterns (LBP). In many moving segmentation algorithms, the information from limited frames before current image is used. Our approach models background over long time with small memory. Firstly, we construct codebook model which represents a compressed form of background model for long image sequences. A single Gaussian model of per-pixel is built to deal with illumination changes. By using the correlation and texture of spatially proximal pixels, local binary patterns background model is constructed. Finally current image is segmented into two parts, foreground and background, by comparing current image with background model. Experiments show that the proposed approach achieves promising results robustly in real videos.

Hyperspectral unmixing (HU) aims to obtain subpixel-level material composition information, which is crucial for the precise advancement of hyperspectral image processing techniques. In recent years, deep learning (DL) has been widely applied to HU due to its strong ability to capture complex and nonlinear feature relationships in the data. However, relying solely on hyperspectral images for unmixing often fails to effectively distinguish objects in complex scenes, particularly when different endmembers exhibit similar spectral characteristics. To address this limitation, we propose a Multimodal Fusion Network (MMFNet) that incorporates the elevation information inherent in light detection and ranging (LiDAR) data. MMFNet is capable of simultaneously extracting spectral features from hyperspectral images and spatial features from LiDAR data. Furthermore, most existing DL-based HU methods operate only in the original spectral domain, which makes them susceptible to spectral variability, noise, and limited discriminative capacity. To overcome these challenges, we integrate Learnable Wavelet Transform (LWT) into MMFNet to adaptively decompose hyperspectral signals into multiple frequency subdomains, thereby mitigating spectral variability and noise while preserving spatial consistency. In addition, a Multi-Scale Convolution Fusion Module (MSCFM) is designed to capture semantic information at different receptive fields and enhance the fine-grained fusion of spectral and spatial features. Through these designs, MMFNet produces more robust and discriminative feature representations, enabling better separation of spectrally similar endmembers. Extensive experiments demonstrate that the proposed MMFNet outperforms several state-of-the-art unmixing methods.

One of the functionalities a service robot should have, to work in a smart environment, is object recognition and handling of objects. Many researchers have attempted to make service robots recognize objects in natural environments but no conventional vision systems can recognize target objects in complex scenes. We built a prototype smart environment in our research building at Korea Institute of Industrial Technology (KITECH) to demonstrate the feasibility that a service robot with few sensors can provide reliable services by interacting with an environment full of smart devices through wireless sensor network communications. In this paper, we address the issues of how to develop an object recognition system for our RoboMaidHome project. Our object recognition system can not only recognize objects with no need of a previous training stage but make the service robot light-weight with minimum hardware and software requirements while working well in a smart home environment. In addition, the matching process for object recognition is simple since only a few image processing techniques are employed. Our object recognition scheme on the basis of MPEG-7 visual descriptors is under development and includes color, texture, and shape. It will also be incorporated into our mobile service robotics system.