Edge Distraction-aware Salient Object Detection

Image retrieval is one of the most interesting and fastest growing research areas in all fields. It is an effective and efficient tool for managing large image databases. In most Content-Based Image Retrieval (CBIR) systems, an image is represented by a set of low-level visual features; hence a direct correlation with high-level semantic information will be absent. Therefore, a gap exists between high-level information and low-level features, which is the main reason that hinders the improvement of the image retrieval accuracy. In this work, main focus is on the semantic based image retrieval system using Gray Level Co-occurrence Matrix (GLCM) for texture feature extraction. Based on the texture features, semantic interpretations are given to the extracted textures. The images are retrieved according to user satisfaction and thereby reduce the semantic gap between low level features and high level features.

With the development of theories and technologies in medical imaging, most of the tumors can be detected in the early stage. However, the nature of ovarian cysts lacks accurate judgement, leading to that many patients with benign nodules still need Fine Needle Aspiration (FNA) biopsies or surgeries, increasing the physical pain and mental pressure of patients as well as unnecessary medical health care costs. Therefore, we present an image diagnosis system for classifying the ovarian cysts in color ultrasound images, which novelly applies the image features fused by both high-level features from deep learning network and low-level features from texture descriptor. Firstly, the ultrasound images are enhanced to improve the quality of training data set and the rotation invariant uniform local binary pattern (ULBP) features are extracted from each of the images as the low-level texture features. Then the high-level deep features extracted by the fine-tuned GoogLeNet neural network and the low-level ULBP features are normalized and cascaded as one fusion feature that can represent both the semantic context and the texture patterns distributed in the image. Finally, the fusion features are input to the Cost-sensitive Random Forest classifier to classify the images into "malignant" and "benign". The high-level features extracted by the deep neural network from the medical ultrasound image can reflect the visual features of the lesion region, while the low-level texture features can describe the edges, direction and distribution of intensities. Experimental results indicate that the combination of the two types of features can describe the differences between the lesion regions and other regions, and the differences between lesions regions of malignant and benign ovarian cysts.

Recurrent reverse attention guided residual learning for saliency object detection

In a Content Based Image Retrieval (CBIR) system, images are retrieved based on the content of the images such as color, shapes and objects. However, often end users experience a semantic gap between the notion of similarity used by the ranking model versus the users’ perception about image similarity. Explainable AI (XAI) is an emerging research field that attempts to provide transparency of “black box” models, to make AI systems trustworthy and gain user trust. This work aims at building an I mage R etrieval system with T extual EX planations such as “The (global) results are similar to the query by X% due to shape, Y% due to color”. Local explanations are generated using various methods such as comparing images with respect to overlap between low level features such as color, shape and regions (MPEG-7 features). Additionally, high level features such as background-foreground segmentation, deep learned features and major key-points, objects identified (SIFT features) are used to enrich the explanations. We evaluate the quality of rankings on benchmark data-sets such as PASCAL VOC. The XAI facet of user satisfaction and usefulness of the system is evaluated in a lab based user study. Our results show that the semantic gap is better bridged using high level features. While low level features might be better suited for re-ranking of the retrieved images.

Human scene categorization is characterized by its remarkable speed. While many visual and conceptual features have been linked to this ability, significant correlations exist between feature spaces, impeding our ability to determine their relative contributions to scene categorization. Here, we used a whitening transformation to decorrelate a variety of visual and conceptual features and assess the time course of their unique contributions to scene categorization. Participants (both sexes) viewed 2250 full-color scene images drawn from 30 different scene categories while having their brain activity measured through 256-channel EEG. We examined the variance explained at each electrode and time point of visual event-related potential (vERP) data from nine different whitened encoding models. These ranged from low-level features obtained from filter outputs to high-level conceptual features requiring human annotation. The amount of category information in the vERPs was assessed through multivariate decoding methods. Behavioral similarity measures were obtained in separate crowdsourced experiments. We found that all nine models together contributed 78% of the variance of human scene similarity assessments and were within the noise ceiling of the vERP data. Low-level models explained earlier vERP variability (88 ms after image onset), whereas high-level models explained later variance (169 ms). Critically, only high-level models shared vERP variability with behavior. Together, these results suggest that scene categorization is primarily a high-level process, but reliant on previously extracted low-level features.SIGNIFICANCE STATEMENT In a single fixation, we glean enough information to describe a general scene category. Many types of features are associated with scene categories, ranging from low-level properties, such as colors and contours, to high-level properties, such as objects and attributes. Because these properties are correlated, it is difficult to understand each property's unique contributions to scene categorization. This work uses a whitening transformation to remove the correlations between features and examines the extent to which each feature contributes to visual event-related potentials over time. We found that low-level visual features contributed first but were not correlated with categorization behavior. High-level features followed 80 ms later, providing key insights into how the brain makes sense of a complex visual world.

In this paper, we propose a text extraction method from camera-captured document style images and propose a text segmentation method based on a color clustering method. The proposed extraction method detects text regions from the images using two low-level image features and verifies the regions through a high-level text stroke feature. The two level features are combined hierarchically. The low-level features are intensity variation and color variance. And, we use text strokes as a high-level feature using multi-resolution wavelet transforms on local image areas. The stroke feature vector is an input to a SVM (support vector machine) for verification, when needed. The proposed text segmentation method uses color clustering to the extracted text regions. We improved K-means clustering method and it selects K and initial seed values automatically. We tested the proposed methods with various document style images captured by three different cameras. We confirmed that the extraction rates are good enough to be used in real-life applications.

The paper briefly reviews certain aspects of the biological visual system and presents a smart vision sensor for the detection of higher-level features. The visual system processes information in a hierarchical manner starting from the retina up to the visual cortex. It decomposes the image in simple features (edges, orientation, line stops, corners, etc) using spatial and temporal information. At the higher level it integrates these primitive features, resulting in the recognition of complex objects. The sensor described in the paper is loosely modeled after the visual system and incorporates pixel level, programmable elements which extract orientation, end stops, corners and junctions from a line drawing. The architecture resembles a CNN-UM that can be programmed with a 30-bit word. The 16/spl times/16 pixels array detects these higher-level features in about 54 /spl mu/s.

The existing semantic segmentation networks mostly focus on extracting and expressing deep image features. But none of them could adequately aggregate contextual information, or utilize features on different scales or layers. To improve prediction results, this paper proposes a multiscale attention fusion network for semantic segmentation (MAF-DeepLab), which highlights important features, and aggregates multi-scale features well. Firstly, the high-level semantic features and low-level texture features were captured by a lightweight feature extraction network. Secondly, cascaded spatial pyramidal pooling (CSPP) were employed to fuse feature extraction branches with different receptive fields, enhancing the correlation between multi-scale features. Finally, a bottom-up attention fusion module was adopted to guide the cascading aggregation of high-level and low-level features, producing detailed saliency maps. MAF-DeepLab achieved an excellent effect of semantic segmentation on two benchmark datasets: CamVid (74.8%) and Cityscapes (83.4%).

Effective scene representation is a fundamental part of high-resolution scene classification systems. In this letter, we present a holistic scene representation method, i.e., the two-level feature representation (TLFR) model. The TLFR is composed of low-level and high-level features. Low-level features are obtained by computing the residual error between a local descriptor and its corresponding visual word, and the high-level features are obtained using a proposed selection-constrained sparse coding method. In addition, low-level features in a cluster are integrated by summation pooling, whereas high-level features are fused by maximization pooling. The holistic scene representation is finally generated by incorporating these two levels of features into the bag-of-visual-words framework. Experimental results show that the TLFR model is robust to translation and rotation variations and demonstrates promising performance with the Land Use and Land Cover Database data set and a newly released Singapore data set.

In recent years, deep-learning-based methods have been widely applied in the field of industrial surface defect detection, including the precise segmentation of defects on printed circuit boards (PCBs). However, the high time and financial costs associated with obtaining trainable annotated datasets in practical production severely limit the performance of deep-learning-based segmentation networks. To address this issue, we propose a Siamese feature contrast network (SFCNet) that utilizes easily obtainable template images to enhance the performance of segmentation networks. The proposed SFCNet comprises two shared-weight encoders, a decoder, a feature contrast module (FCM), and an edge feature module (EFM). The shared-weight encoders are designed to extract multilevel high-level features from both template and input images. The FCM and EFM extract different pieces of information from the features and local edge features of the two images, respectively, and the fused features from these two modules are fed into the decoder to compute the defect prediction map. Unlike existing Siamese or contrastive learning-based methods that focus primarily on global feature similarity, SFCNet introduces specialized modules to explicitly capture both multiscale contrastive differences and fine-grained edge cues, improving defect localization and segmentation precision. Notably, we observe that using higher-level features in the FCM could degrade network performance, which we attribute to the abstract information in higher-level features not aiding the proposed FCM. In addition, we introduce a composite loss function for network training, which combines Lovász-Softmax Loss and Ohem Cross-Entropy Loss to calculate the loss between predictions and labels, employing weighted values to achieve the best combination. Extensive experimental results on real PCB surface defect datasets demonstrate that SFCNet achieves significant performance improvements over various baselines. For instance, when integrated with SFCNet, DANet++, and ANNet++ achieve intersection over union (IoU) score improvements of 8.61% and 6.05%. ANNet++ reached an IoU score of 64.12%, which is the highest among all evaluated advanced models.

The influence of architectural design on well-being is increasingly recognised, underscoring the need to understand how humans evaluate their surroundings. This study examines the impact of low-level visual features and high-level architectural elements in images of house facades generated by a generative adversarial network (GAN). We introduce a novel method to create controlled image datasets using the style mixing component of StyleGAN2-ADA, enabling the combination of specific architectural features with styles representing low-level image properties, such as brightness or colour. Our dataset consists of 900 images encompassing five high-level features (house size, garage door visibility, number of windows, entrance door visibility, and roof shape) and 16 low-level visual features. These GAN-generated images were rated in an online experiment by 303 participants on six dimensions: facelikeness, hominess, relaxation, invitingness, safety, and price. Linear mixed-effects models identified significant predictors, including the number of floors as a high-level feature and green pixel percentage, saturation (SD), brightness (SD), and contrast as low-level features. This indicated that houses with two floors and images with an overall higher amount of green pixel, as well as diversity of saturation and brightness and a lower contrast were rated as, e.g., more inviting and safer. The analysis showed that high- and low-level features explained up to 54% of the variance in house price ratings, with high-level features accounting for the larger share and highlighting their relevance for house facade evaluations. This work advances the use of GANs and offers a method to modify visual elements that shape architectural evaluations.

Content-based video retrieval system aims at assisting a user to retrieve targeted video sequence in a large database. Most of the search engines use textual annotations to retrieve videos. These types of engines offer a low-level abstraction while the user seeks high-level semantics. Bridging this type of semantic gap in video retrieval remains an important challenge. In this paper, colour, texture and shapes are considered to be low-level features and motion is a high-level feature. Colour histograms convert the RGB colour space into YcbCr and extract hue and saturation values from frames. After colour extraction, filter mask is applied and gradient value is computed. Gradient and threshold values are compared to draw the edge map. Edges are smoothed for sharpening to remove the unnecessary connected components. These diverse shapes are then extracted and stored in shape feature vectors. Finally, an SVM classifier is used for classification of low-level features. For high-level features, depth images are extracted for motion feature identification and classification is done via echo state neural networks (ESN). ESN are a supervised learning technique and follow the principle of recurrent neural networks. ESN are well known for time series classification and also proved their effective performance in gesture detection. By combining the existing algorithms, a high-performance multimedia event detection system is constructed. The effectiveness and efficiency of proposed event detection mechanism is validated using MSR 3D action pair dataset. Experimental results show that the detection accuracy of proposed combination is better than those of other algorithms

IntroductionFocal cortical dysplasia (FCD) is one of the common causes of refractory epilepsy. The subtle and indistinct edge of FCD lesions pose considerable challenges for accurate lesion localization. Therefore, we propose an edge guided segmentation network based on Laplacian pyramid to improve the localization performance of FCD lesions.MethodsThis is a retrospective study evaluated on two independent datasets. The proposed Laplacian Edge Mix UNet (LEM-UNet) builds upon the MedNeXt baseline and incorporates the Laplacian Edge Attention (LEA) block and the Multi-strategy Feature Fusion (MFF) block. LEA block captures lesion details and edge information during the encoding phase by integrating Laplacian pyramid feature maps with an attention mechanism, while MFF block fuses edge features with high level features during the decoding phase.ResultsThe model's performance was assessed through 5-fold cross-validation across both Open and Private Datasets, demonstrating superior performance. The average Dice Coefficient achieved 0.452 and 0.597 on the Open and Private Datasets, respectively, representing improvements of 2.40% and 2.90% compared to the baseline model.DiscussionThe results demonstrate the importance of focusing on lesion edge in the FCD segmentation task. The integration of the Laplacian pyramid enhances the mode's ability to capture lesions with blurred edge and subtle features. LEM-UNet exhibits significant advantages over current FCD segmentation algorithms. The source code and pre trained model weights are available at https://github.com/simplify403/LEM-UNet.

A pooling-based feature pyramid network for salient object detection

This study presents a new end-to-end change detection network, called difference-enhancement dense-attention convolutional neural network (DDCNN), that is designed for detection of changes in the bitemporal optical remote sensing images. To model the internal correlation between high-level and low-level features, a dense attention method consisting of several up-sampling attention units is proposed. Both the up-sampling spatial and up-sampling channel attention are adopted by the unit. The unit, which can use high-level features with rich category information to guide the selection of low-level features, can use the spatial context information to capture the changed features of ground objects. Furthermore, DDCNN also pays attention to the differentiating features of the bitemporal images. By introducing a DE unit, each pixel is weighted and the features are selectively aggregated. The combination of dense attention and the DE unit improves the effectiveness of the network and its accuracy in extracting the change features. The effectiveness of the proposed approach is demonstrated via five challenge data sets. The experimental results show that DDCNN achieves new state-of-the-art change detection performance on these five challenging data sets. For the seasonal change detection data set in particular, compared with the best existing change detection model, the proposed method increases the F1 score and IoU by 2.96% and 5.17%, respectively; compared with the baseline method, our method improved 3.75% and 6.50% on the F1 score and IoU, respectively.