GMMSP on GPU

Superpixel segmentation, a preprocessing component, is widely used in computer vision tasks. Superpixel algorithm is supposed to generate superpixels with required boundary adherence, compactness as well as less computational complexity. However, most of the existing superpixel algorithms do not perform satisfactorily when it comes to compactness and boundary adherence, as they are a paradox in nature. In this paper, we propose a new superpixel segmentation method based on density peak (DP) clustering and modification of the original DP algorithm. The experimental results have been compared with state-of-the-art methods, both quantitatively and qualitatively, revealing the efficient outcomes that adhere to the object boundaries better than others.

Point set registration is one of the challenging tasks in computer vision. One critical step is to find the corresponding relationship between the model point set and the scene point set. Existing registration algorithms primarily utilize the information of global and local shape, yet neglect the credibility of corresponding relation, therefore they may lead to the insufficient estimation of spatial transformation. To tackle this problem, we firstly adopt a relative polar coordinate system, it performs spatial pooling operation and further divides the feature extraction region into sub-areas with different scales. Then, based on the Relative Average Distance (RAD) and the Relative Average Offset Angle (RAOA), we propose multi-granular MRGS descriptor to extract visual structures of the point set. The similarity between the model point set and scene point set is then represented by the Gaussian Mixture Model, where the weights can be dynamically adjusted during the process of registration. Finally, we apply the robust mixture re-weighting to reduce the impact of false corresponding pairs and reinforce the weight of correct matching points. Experimental results on synthetic data and medical image data not only show that our method outperform state-of-the-art methods, but also demonstrate the robustness of our method when the non-grid transformation of point sets suffers from deformations, noises and outliers.

Feature-guided dictionary learning for patch-and-group sparse representations in single image deraining

Foreground detection is a basic but challenging task in computer vision. In this paper, a novel hybrid local texture based method is presented to model the background for complex scenarios and an image segmentation based denoising processing is applied to reduce noise. We combine the uniform pattern of eXtended Center-Symmetric Local Binary Pattern (XCS-LBP) and Center- Symmetric Local Derivative Pattern (CS-LDP) to generate a discriminative feature with shorter histogram. Retaining the strengths of the two textures, it appears to be robust to dynamic scenes, illumination changes and noise. Based on the hybrid feature, we employ an overlapping block based Gaussian Mixture Model (GMM) framework which makes classifying decision in pixel level. Experimental results on two changeling datasets (Wallflower and I2R dataset) clearly justify the performance of proposed method. Besides, we take the foreground masks obtained by proposed method as input to a tracking system showing notable results.

Object detection is a fundamental task in computer vision and remote sensing, aimed at recognizing and categorizing different types of objects within images. Unmanned aerial vehicle - based thermal infrared remote sensing provides crucial multi-scenario images and videos, serving as key data sources in public applications. However, object detection in these images remains challenging due to complex scene information, lower resolution compared to visible-spectrum videos, and a shortage of publicly available labeled datasets and trained models. This article introduces a Unmanned aerial vehicle - based thermal infrared object detection framework for analyzing images and videos in public applications and evaluates the performance of YOLOv8n/v8s, YOLOv11n/v11s, and YOLOv12n/v12s models in extracting features from ground-based thermal infrared images and videos captured by Forward-Looking Infrared cameras, as well as from unmanned aerial vehicle - recorded thermal infrared videos taken from various angles. The YOLOv8n/v8s, YOLOv11n/v11s, and the latest YOLOv12n/v12s models were deployed on a Raspberry Pi 5 using the OpenVINO framework. The successful deployment of these models, including the most recent version, demonstrates their feasibility for unmanned aerial vehicle-based thermal infrared object detection. The results show that YOLOv8 and YOLOv11 achieved high accuracy and recall rates of 93% and 92%, respectively, while the YOLOv12 model demonstrated good precision but comparatively lower performance in accuracy and recall, suggesting the possibility for further improvement.

Recent research has shown the initial success of sparse coding (Sc) in solving many computer vision tasks. Motivated by the fact that kernel trick can capture the nonlinear similarity of features, which helps in finding a sparse representation of nonlinear features, we propose kernel sparse representation (KSR). Essentially, KSR is a sparse coding technique in a high dimensional feature space mapped by an implicit mapping function. We apply KSR to feature coding in image classification, face recognition, and kernel matrix approximation. More specifically, by incorporating KSR into spatial pyramid matching (SPM), we develop KSRSPM, which achieves a good performance for image classification. Moreover, KSR-based feature coding can be shown as a generalization of efficient match kernel and an extension of Sc-based SPM. We further show that our proposed KSR using a histogram intersection kernel (HIK) can be considered a soft assignment extension of HIK-based feature quantization in the feature coding process. Besides feature coding, comparing with sparse coding, KSR can learn more discriminative sparse codes and achieve higher accuracy for face recognition. Moreover, KSR can also be applied to kernel matrix approximation in large scale learning tasks, and it demonstrates its robustness to kernel matrix approximation, especially when a small fraction of the data is used. Extensive experimental results demonstrate promising results of KSR in image classification, face recognition, and kernel matrix approximation. All these applications prove the effectiveness of KSR in computer vision and machine learning tasks.

Weakly supervised video anomaly detection aims to identify abnormal events in video sequences without requiring frame-level supervision, which is a challenging task in computer vision. Traditional methods typically rely on low-level visual features with weak supervision from a single backbone branch, which often struggles to capture the distinctive characteristics of different categories. This limitation reduces their adaptability to real-world scenarios. In real-world situations, the boundary between normal and abnormal events is often unclear and context-dependent. For example, running on a track may be considered normal, but running on a busy road could be deemed abnormal. To address these challenges, RelVid is introduced as a novel framework that improves anomaly detection by expanding the relative feature gap between classes extracted from a single backbone branch. The key innovation of RelVid lies in the integration of auxiliary tasks, which guide the model to learn more discriminative features, significantly boosting the model’s performance. These auxiliary tasks—including text-based anomaly detection and feature reconstruction learning—act as additional supervision, helping the model capture subtle differences and anomalies that are often difficult to detect in weakly supervised settings. In addition, RelVid incorporates two other components, which include class activation feature learning for improved feature discrimination and a temporal attention module for capturing sequential dependencies. This approach enhances the model’s robustness and accuracy, enabling it to better handle complex and ambiguous scenarios. Evaluations on two widely used benchmark datasets, UCF-Crime and XD-Violence, demonstrate the effectiveness of RelVid. Compared to state-of-the-art methods, RelVid achieves superior performance in both detection accuracy and robustness.

Convolutional neural networks (CNNs) have been widely deployed in computer vision tasks. However, the computation and resource intensive characteristics of CNN bring obstacles to its application on embedded systems. This article proposes an efficient inference accelerator on Field Programmable Gate Array (FPGA) for CNNs with depthwise separable convolutions. To improve the accelerator efficiency, we make four contributions: (1) an efficient convolution engine with multiple strategies for exploiting parallelism and a configurable adder tree are designed to support three types of convolution operations; (2) a dedicated architecture combined with input buffers is designed for the bottleneck network structure to reduce data transmission time; (3) a hardware padding scheme to eliminate invalid padding operations is proposed; and (4) a hardware-assisted pruning method is developed to support online tradeoff between model accuracy and power consumption. Experimental results show that for MobileNetV2 the accelerator achieves 10× and 6× energy efficiency improvement over the CPU and GPU implementation, and 302.3 frames per second and 181.8 GOPS performance that is the best among several existing single-engine accelerators on FPGAs. The proposed hardware-assisted pruning method can effectively reduce 59.7% power consumption at the accuracy loss within 5%.

Review and classification of AI-enabled COVID-19 CT imaging models based on computer vision tasks

Recent work in machine learning has significantly benefited semantic extraction tasks in computer vision, particularly for object recognition and image retrieval. We argue that the computer vision techniques that have been successfully applied in those settings can effectively be translated to other domains, such as audio. This claim is supported by recent results in music vs. speech classification, structure from sound, robust music identification and sound object recognition. This paper focuses on two such audio applications and demonstrates how ideas from computer vision map naturally to these problems.

Non-rigid object tracking is an important task in computer vision, while its natural contour extraction is one of the most difficult problems during the process. Most tracking-by-detection methods are based on rectangular bounding-boxes, this will lead errors into subsequent detection. This paper present a novel superpixel-based detector for accurate natural contour extraction, there are three main contributions: 1) combining real-time superpixel segmentation with natural contour detection, 2) proposing an object-oriented natural contour extraction method for non-rigid objects, 3) proposing a non-rigid object detection method based on flexible scanning window. Compared with those bounding-box based detection methods, our detector can provide very accurate initial input of object model, then produce accurate natural contour output of the non-rigid object. Our detector broke the conventional detection method based on scanning rectangle, which greatly reduced the interference caused by background information. The experiments show that the proposed method outperforms the state-of-the-art algorithms not only on the contour accuracy but also on the computation cost. In addition, the initialization stage of our method overcomes the limitation of HT caused by the size of initial bounding-box.

Single Image Super-Resolution (SISR) aims to reconstruct high-resolution (HR) images from low-resolution (LR) inputs, serving as a core task in computer vision. Despite recent advances, existing methods often struggle to balance structural fidelity and computational efficiency. To address this, we propose PFRNet, a lightweight superpixel-aware model integrating superpixel segmentation, local attention aggregation, and global structure modeling. The framework comprises four key modules: GASS, SPDF, SPFA, and LAP, jointly enabling multi-scale and structure-consistent feature learning. Experiments on benchmark datasets (e.g., Set5, Urban100) show that PFRNet achieves superior performance with fewer parameters. Ablation studies further verify the effectiveness of each module.

Toward designing intelligent PDEs for computer vision: An optimal control approach

Foreground extraction is an important and challenging problem in many applications of computer vision. Most existing algorithms either require user intervention as hard constraints or demand special inputs for extra information. Thus, an automatic foreground extraction algorithm from a single image is proposed in this paper. A Gaussian image pyramid is constructed and the gradient vector flow (GVF) snake is adopted at the coarsest level to generate a rough contour of the object which is then upsampled and serves as the initial input to GVF snake in the next level. This process repeats until the estimated contour is propagated to the finest level. Based on the result in the finest level, a binary mask can be generated accordingly and becomes the initial constraint in the segmentation. The proposed segmentation step includes two novel schemes, which simulate the Latent Dirichlet Allocation (LDA) generative model and a probabilistic stochastic process called Chinese restaurant process. With these mechanisms, the Gaussian Mixture Models adaptively determine the number of components for foreground and background individually. As a result, the proposed method is expected to not only produce a satisfactory result of foreground extraction automatically with more robustness and adaptation but also serve as a good preprocessing to improve the performance and accuracy for tasks in computer vision.

In recent studies, superpixel segmentation has been integrated into hyperspectral (HS) image classification methods. However, the existing superpixel-based classification methods usually suffer from two serious problems. First, the accuracy and efficiency of current superpixel segmentation approaches cannot meet the demands of practical applications for HS images; second, conventional superpixel-based classification methods generally consider each generated superpixel as a unit for the image classification, which may help to reduce the computing time but result in a significant decrease of the classification accuracy. To solve the problems, we propose a fast region growing based superpixel segmentation (FRGSS) algorithm and a novel texture-adaptive superpixel integration strategy (TASIS) for the HS image classification. Experimental results on real Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) HS images demonstrate that the proposed FRGSS outperforms the state-of-the-art superpixel algorithm. In addition, the superiority of the TASIS is verified compared to the pixel-wise and the conventional superpixel-based classification methods.