Semantic Image Segmentation and Object Labeling

In this paper we present a framework for simultaneous image segmentation and region labeling leading to automatic image annotation. The proposed framework operates at semantic level using possible semantic labels to make decisions on handling image regions instead of visual features used traditionally. In order to stress its independence of a specific image segmentation approach we applied our idea on two region growing algorithms, i.e. watershed and recursive shortest spanning tree. Additionally we exploit the notion of visual context by employing fuzzy algebra and ontological taxonomic knowledge representation, incorporating in this way global information and improving region interpretation. In this process, semantic region growing labeling results are being re-adjusted appropriately, utilizing contextual knowledge in the form of domain-specific semantic concepts and relations. The performance of the overall methodology is demonstrated on a real-life still image dataset from the popular domains of beach holidays and motorsports.

Affective valences of stimuli (e.g. positive or negative) influence perceptual prioritisation, with emotionally charged stimuli often attended to over neutral ones. However, there are two critical issues in interpreting the previous findings on affective biases: (1) reliance on Reaction Time (RT) measures, which have limitations in capturing subtle cognitive biases at different phases of visual processing, and (2) potential confounds from low-level visual features that can carry affective valence. To address these issues, we used a Temporal Order Judgement (TOJ) paradigm with semantic stimuli and Chinese characters due to their minimised perceptual variations. In this task, two stimuli were presented nearly simultaneously, and participants indicated which appeared first. If an engagement bias is present, participants would consistently perceive the biased stimulus as first appearing even when it was veridically the second (Prior Entry Effect). We specifically examined the direct competition between positive and negative valences at the semantic level. Our results revealed a consistent positive bias over negative stimuli, although its magnitude was smaller than biases observed with facial stimuli in prior research. These findings suggest that affective biases might occur at both semantic and visual levels, offering a more nuanced understanding of emotional attention.

Affective analysis of images in social networks has drawn much attention, and the texts surrounding images are proven to provide valuable semantic meanings about image content, which can hardly be represented by low-level visual features. In this paper, we propose a novel approach for visual affective classification (VAC) task. This approach combines visual representations along with novel text features through a fusion scheme based on Dempster-Shafer (D-S) Evidence Theory. Specifically, we not only investigate different types of visual features and fusion methods for VAC, but also propose textual features to effectively capture emotional semantics from the short text associated to images based on word similarity. Experiments are conducted on three public available databases: the International Affective Picture System (IAPS), the Artistic Photos and the MirFlickr Affect set. The results demonstrate that the proposed approach combining visual and textual features provides promising results for VAC task.

The ever-increasing numbers of medical images in digital format are generated in clinical practice every day. These images include high-resolution and temporal data of various modalities and which constitute an important source of anatomical and functional information for diagnosis of diseases, research and education. So how to manage these large data and utilize them effectively and efficiently possess significant technical challenges. Thus, the technique of Content-based Medical Image Retrieval (CBMIR) emerges as the times require. However, current CBMIR is not sufficient to capture the semantic content of an image and difficult to provide good results according to the predefined categories in the medical domain for less using the medical knowledge. Accordingly, in this paper a mapping modelling of visual feature and knowledge representation is proposed to approach for medical image retrieval. Firstly, the low-level fusion visual features are extracted based on intensity, texture, and their extended versions. Secondly, a set of disjoint semantic tokens with appearance in lung CT images is selected to define a vocabulary based on medical knowledge representation. Finally, a mapping modeling is investigated to associate low-level visual image features with their high-level semantic. Experiments are conducted with a medical image database consisting of 220 lung medical images obtained from the Heilongjiang Province Hospital. And the comparison of the retrieval results shows that the approach proposed in this paper is effective.

Fuzzy sets on 2D spaces for fineness representation

To bridge the semantic gap between low-level visual features and high-level semantic concepts, this paper puts forward a novel semi-supervised learning framework of combining visual and keyword features. We assume all of images in the database have been annotated. In this framework, the visual space graph (VSG) and keyword space graph (KSG) are first constructed by means of manifold-based algorithm to explore the submanifold of Visual space and keyword space. Then, semantic space graph (SSG) is constructed by using neighbor information of KSG and VSF. Afterwards, the VSG and KSG are modified and unified by propagating semantic matrix. Finally, we propose a flexible ranking formula and introduce different relevance feedback methods of different query styles. Experimental results on COREL 5000 images show our method improves image retrieval performance from all aspects.

Region Labeling is to automatically assign semantic labels to the corresponding image regions. Most of the previous works focus on exploiting low level visual features, particularly visual spatial contextual information, to address the problem. However, very few work explore high level semantic information of the whole image to deal with the problem. In this paper, we propose a new region labeling approach by integrating both visual spatial and semantic contextual information into a unified model. In our method, region labeling is regarded as a multi-class classification problem. For each semantic concept, we train a Conditional Random Field CRF model respectively. It consists of both the region grid sub-graph and the co-occurred semantic label sub-graph. In our model, the integration of the two kinds of contextual information brings reinforcement effect on the improvement of region labeling. The experiments are conducted on two commonly used benchmark datasets and the experimental results show that our method achieves the best performance compared with the strong baselines and the state-of-the-art methods.

Previous research has shown that interacting with natural environments vs. more urban or built environments can have salubrious psychological effects, such as improvements in attention and memory. Even viewing pictures of nature vs. pictures of built environments can produce similar effects. A major question is: What is it about natural environments that produces these benefits? Problematically, there are many differing qualities between natural and urban environments, making it difficult to narrow down the dimensions of nature that may lead to these benefits. In this study, we set out to uncover visual features that related to individuals' perceptions of naturalness in images. We quantified naturalness in two ways: first, implicitly using a multidimensional scaling analysis and second, explicitly with direct naturalness ratings. Features that seemed most related to perceptions of naturalness were related to the density of contrast changes in the scene, the density of straight lines in the scene, the average color saturation in the scene and the average hue diversity in the scene. We then trained a machine-learning algorithm to predict whether a scene was perceived as being natural or not based on these low-level visual features and we could do so with 81% accuracy. As such we were able to reliably predict subjective perceptions of naturalness with objective low-level visual features. Our results can be used in future studies to determine if these features, which are related to naturalness, may also lead to the benefits attained from interacting with nature.

In this paper, we propose a novel and adaptive method for image search reranking. We firstly evaluate different visual features based on the results of image classification on object and scene separately. And visual features are chosen adaptively to rerank the initial image search result. For a given query, it can be classified into either object or scene using the trained classifier on text features. Then, low-level visual features are adaptively selected and fused for image search reranking. Experimental results on large scale image dataset of WebQueries demonstrate the efficacy of the proposed method.KeywordsImage search rerankingadaptive feature selection and fusionobject and scene

Semantic image annotation can be viewed as a mapping procedure from image features to semantic labels, by the steps of image feature extraction and image-semantic mapping. The features can be low-level visual features, such as color, texture, shape, etc., and the semantic labels can be related to the knowledge of human on the image understanding. However, these linear representations are insufficient to describe the complex natural scene. In this paper, we study currently existing visual models that are able to imitate the way the human visual system acts for the tasks of object recognition and scene interpretation. Therefore, it is expected to bring a better understanding to the image visual content in human cortex will. In the experiments, there are three state-of-the-art visual models are investigated for the application of automatic image annotation. The results demonstrate that with our proposed strategy, the annotation accuracy is improved comparing to the most used low-level linear representation features.

ABSTRACTA process of splitting the image into pixel bands is the image segmentation. As medical imaging contain uncertainties, there are difficulties in classification of images into homogeneous regions. There is a need for segmentation algorithm for removing the noise from the medical image segmentation. The very popular algorithm is Fuzzy C‐Means (FCM) algorithm used for image segmentation. Fuzzy sets, rough sets, and the combination of fuzzy and rough sets play a prominent role in formalizing uncertainty, vagueness, and incompleteness in diagnosis. But it will use intensity values only which will be highly sensitive to noise. In this article, an Intuitionistic FCM (IFCM) algorithm is presented for clustering. Intuitionistic fuzzy (IF) sets are generalized sets and their elements are characterized by a membership value as well as nonmembership value. This IFCM has an uncertainty parameter which is called hesitation degree and a new objective function is integrated in the standard FCM based on IF entropy. The IFCM will provide better performance than FCM for image segmentation.

Semantic image retrieval using text such keywords or captions at different semantic levels has attracted considerable research attention in recent years. Automatic image annotation (AIA) has been proved to be an effective and promising solution to automatically deduce the high-level semantics from low-level visual features. Due to the complexity of image-label mapping, it has become a challenge to systematically develop precise models with better performance. In this paper, we try to address the problem using Latent Semantic Indexing (LSI) together with mixed Bagging (Bootstrap Aggregating). Given a training set of annotated images, we first perform vector quantizing of visual space to construct a visual vocabulary consisting of visual terms based on K-means constrained by the semantic relationships derived from the associated annotations using Latent Semantic Indexing; then the statistical hidden correlation between the visual terms and keywords is modeled by a two-level mixed Bagging model. At the base level, each multi-class SVM is trained independently using the replicate training set via Bootstrap technique, and the decision of each multi-class SVM is integrated based on Average aggregating to generate a combined confidence label vector (CLV) associated with each visual term. All the CLVs are then propagated to the subsequent language model fusion level. At his level, WordNet is used for re-weighting the CLV for each visual term by incorporating the word correlations to obtain an appropriate and consistent image annotation. We carried out experiments on a medium-sized image collection with about 1000 images from Corel Stock Photo CDs. The experimental results demonstrated that the annotation performance of this method outperforms some traditional approaches, showing the feasibility and effectiveness of the proposed unified framework into which semantic image classification and language models are seamlessly integrated.

Infrared and Visible Image Fusion (IVIF) produces enhanced images by fusing complementary visual information. However, most existing methods generate fixed outputs and cannot flexibly adapt to user-specific requirements. Recent text-guided approaches offer partial control but are limited to global or semantic levels, lacking instance-level control. This limitation arises from two challenges: first, the lack of datasets that directly link textual instructions with corresponding spatial annotations, and second, the use of coarse cross-modal alignment methods that struggle to precisely match textual instructions with visual features. To overcome these challenges, we propose ControlFuse, a controllable IVIF framework enabling multi-granularity fusion across global, semantic, and instance levels, guided by user instructions. First, we construct an automated multi-granularity dataset that provides explicit textual-mask correspondences at these three levels. Second, inspired by manifold geometry, we design a Multimodal Feature Interaction Module (MFIM) comprising Feature Manifold Converter (FMC) and Curvature-Guided Interaction (CGI). FMC projects textual and visual features into a unified manifold space, while CGI leverages manifold curvature as a geometric cue to refine cross-modal alignment. Extensive experiments validate ControlFuse, outperforming state-of-the-art methods in robustness and flexibility.

In this paper, a learning approach coupling Support Vector Machines (SVMs) and a Genetic Algorithm (GA) is presented for knowledge-assisted semantic image analysis in specific domains. Explicitly defined domain knowledge under the proposed approach includes objects of the domain of interest and their spatial relations. SVMs are employed using low-level features to extract implicit information for each object of interest via training in order to provide an initial annotation of the image regions based solely on visual features. To account for the inherent visual information ambiguity spatial context is subsequently exploited. Specifically, fuzzy spatial relations along with the previously computed initial annotations are supplied to a genetic algorithm, which uses them to decide on the globally most plausible annotation. In this work, two different fitness functions for the GA are tested and evaluated. Experiments with outdoor photographs demonstrate the performance of the proposed approaches.

The goal of content based image retrieval is to retrieve the images that users want to search. Content based Image retrieval systems attempt to search through a database to find images that are perceptually similar to a query image. Set of low-Level visual features (Color, Shape and Texture) are used to represent an image in most modern content based image retrieval systems. Therefore, between high-level information and low-level features a gap exists, which are the main reason that down the improvement of the image retrieval accuracy.In this paper Hybrid support vector machine (SVM) method proposed to retrieve several features and shorten the semantic gap between low-level visual feature and high-level perception. Image data set is taken from coral image data set. MATLAB 2009a is used as a simulator to analysis the proposed work.