A Context-Based Region Labeling Approach for Semantic Image Segmentation

In this paper, we present a framework for simultaneous image segmentation and object labeling leading to automatic image annotation. Focusing on semantic analysis of images, it contributes to knowledge-assisted multimedia analysis and bridging the gap between semantics and low level visual features. The proposed framework operates at semantic level using possible semantic labels, formally represented as fuzzy sets, to make decisions on handling image regions instead of visual features used traditionally. In order to stress its independence of a specific image segmentation approach we have modified two well known region growing algorithms, i.e., watershed and recursive shortest spanning tree, and compared them to their traditional counterparts. Additionally, a visual context representation and analysis approach is presented, blending global knowledge in interpreting each object locally. Contextual information is based on a novel semantic processing methodology, employing fuzzy algebra and ontological taxonomic knowledge representation. In this process, utilization of contextual knowledge re-adjusts labeling results of semantic region growing, by means of fine-tuning membership degrees of detected concepts. The performance of the overall methodology is evaluated on a real-life still image dataset from two popular domains

Breast ultrasound (BUS) imaging is commonly used in the early detection of breast cancer as a portable, valuable, and widely available diagnosis tool. Automated BUS image classification and segmentation can assist radiologists in making accurate and fast decisions. Recent studies illustrate that tumor, peritumoral, and background regions of BUS images provide valuable information for BUS image segmentation or classification. However, few studies have investigated the influence of these three regions on multi-task learning. In this study, we propose an RMTL-Net to simultaneously segment tumor regions and classify tumors in BUS images into benign or malignant categories. To improve both segmentation and classification performance, we design a regional attention (RA) module that employs the predicted probability maps to automatically guide the classifier to learn important category-sensitive information in the tumor, peritumoral, and background regions and seamlessly fuse them to obtain a better feature representation. We conduct detailed ablation experiments of the proposed RA module and comparative experiments with four recent state-of-the-art peer multi-task learning methods, three single-task segmentation methods, and four single-task classification methods on two public BUS datasets. Experimental results show that the proposed RMTL-Net achieves the best overall segmentation and classification accuracy in terms of five segmentation metrics and six classification metrics.

Accurate blood vessel segmentation plays a crucial role in non-invasive blood flow velocity measurement based on complex-valued magnetic resonance images. We propose a specific snake active contour model-based blood vessel segmentation framework for complex-valued magnetic resonance images. The proposed framework combines both magnitude and phase information from a complex-valued image representation to obtain an optimum segmentation result. Magnitude information of the complex-valued image provides a structural localization of the target object, while phase information identifies the existence of flowing matters within the object. Snake active contour model, which models the segmentation procedure as a force-balancing physical system, is being adopted as a framework for this work due to its interactive, dynamic, and customizable characteristics. Two snake-based segmentation models are developed to produce a more accurate segmentation result, namely the Model-constrained Gradient Vector Flow-snake (MC GVF-snake) and Stochastic-snake. MC GVF-snake elaborates a prior knowledge on common physical structure of the target object to restrict and guide the segmentation mechanism, while Stochastic-snake implements the simulated annealing stochastic procedure to produce improved segmentation accuracy. The developed segmentation framework has been evaluated on actual complex-valued MRI images, both in noise-free and noisy simulated conditions. Evaluation results indicate that both of the developed algorithms give an improved segmentation performance as well as increased robustness, in comparison to the conventional snake algorithm.

LSRML: A latent space regularization based meta-learning framework for MR image segmentation

A general variational framework for image approximation and segmentation is introduced in which the boundary function has a simple explicit form in terms of the approximation function. At the same time, this variational framework is general enough to include the most commonly used objective functions. Since the optimal boundary function, that minimizes the associated objective functional for a given approximation function, can be found explicitly, the objective functional can be expressed in a reduced form that depends only on the approximating function. From this a partial differential equation descent method, aimed at minimizing the objective functional, is derived. The method is fast and produces excellent results as illustrated by a number of real and synthetic image problems.

Abstract—This research presents an integrated framework for satellite image retrieval, segmentation, and visualization, utilizing Python-based algorithms and geospatial data processing methods. It incorporates Google Maps-based image acquisition, a Tkinter-powered interactive GUI, and the U-Net segmentation model to efficiently process satellite imagery. Satellite image segmentation, powered by U-Net, plays a vital role in geospatial analysis, enabling automated classification and extraction of key features from high-resolution imagery. The framework facilitates seamless high-resolution map generation, segmentation, and an- alytical visualization through interactive tools like bar graphs and pie charts. By leveraging U-Net’s robust architecture, this implementation enhances segmentation accuracy and supports applications in urban planning, environmental monitoring, and disaster management. Through the automation of retrieval, processing, and visualization of satellite imagery, this research advances geospatial intelligence and provides a scalable solution for efficient image segmentation workflows. Keywords—Satellite Image Segmentation, Land Cover Classi- fication, Remote Sensing, Image Processing, Semantic Segmen- tation, Deep Learning Models, Convolutional Neural Networks (CNNs), U-Net Architecture, Geospatial Analysis, Land Use Analysis

Unsupervised segmentation of hyperspectral images is an important problem in the field of image processing with many applications, including scene labelling of remote-sensing images and object detection in medical images. However, the high-dimensionality of hyperspectral images poses considerable challenges in this problem. In previous studies, to deal with high-dimensionality, band selection has been performed as a separate step before image segmentation. In this approach, however, information loss can occur because band selection methods do not consider the segmentation task when they select the bands. In addition, in most segmentation tasks, the fact that the number of segments of the image needs to be given before the segmentation task is also an issue. To address these challenges, we propose a method for simultaneous segmentation of hyperspectral images and selection of important bands that are highly involved in the segmentation task. The proposed method, called a mixture of finite maximum margin mixtures, combines a mixture of finite mixtures, which allows the number of segments to be automatically inferred from the data, and a maximum margin classifier, which allows a flexible segmentation and tractable inference on the segmentation labels. Furthermore, we simultaneously select useful bands that help with the segmentation during the segmentation procedure by introducing a latent binary vector that represents discriminating or non-discriminating bands. We demonstrate the advantages of the proposed method using four real datasets.

The blood vessel density in a cancerous tissue sample may represent increased levels of tumor growth. However, identifying blood vessels in the histological (tissue) image is difficult and time-consuming and depends heavily on the observer's experience. To overcome this drawback, computer-aided image analysis frameworks have been investigated in order to boost object identification in histological images. We present a novel algorithm to automatically abstract the salient regions in blood vessel images. Experimental results show that the proposed framework is capable of deriving vessel boundaries that are comparable to those demarcated manually, even for vessel regions with weak contrast between the object boundaries and background clutter.

This paper proposes a novel framework for image segmentation through a unified model-based and pixel-driven integrated graphical model. Prior knowledge is expressed through the deformation of a discrete model that consists of decomposing the shape of interest into a set of higher order cliques (triplets). Such decomposition allows the introduction of region-driven image statistics as well as pose-invariant (i.e. translation, rotation and scale) constraints whose accumulation introduces global deformation constraints on the model. Regional triangles are associated with pixels labeling which aims to create consistency between the model and the image space. The proposed formulation is pose-invariant, can integrate regional statistics in a natural and efficient manner while being able to produce solutions unobserved during training. The challenging problem of tagged cardiac MR image segmentation is used to demonstrate the performance potentials of the method.

The purpose of this study was to evaluate a semi-supervised Mean Teacher (MT) framework for semantic segmentation of cataract surgical images, addressing the challenge of limited labeled data in real-world clinical applications. We adapted the MT framework for four-class segmentation of the iris, pupil, intraocular lens, and surgical instruments using a small labeled set and 40,000 unlabeled Cataract-1K frames. Performance was assessed using Dice Similarity Coefficient (DSC) and the 95th percentile Hausdorff Distance (HD95). MT was compared with a fully supervised UNet under varying labeled and unlabeled conditions, and additional baselines provided broader context. Ablation studies evaluated noise types and key hyperparameters, including consistency weight (λ) and exponential moving average (EMA) decay (α). Internal validation used Cataract-1K, and external testing was done on CaDIS and CatInstSeg. Our MT model consistently outperformed the supervised UNet baseline. With 100 labeled images and 40,000 unlabeled frames, MT achieved a DSC of 0.76 ± 0.12 compared to UNet with 0.59 ± 0.17 (t = 21.23, P < 0.05). On CaDIS and CatInstSeg, MT reached DSCs of 0.69 ± 0.17 and 0.71 ± 0.20, outperforming UNet at 0.62 ± 0.08 and 0.64 ± 0.23. Optimal performance was observed with λ = 0.1, α = 0.995, and Gaussian noise σ = 15. The MT framework provides an effective semi-supervised solution for surgical image segmentation with limited annotations. Full source code and utility scripts will be released upon acceptance at: https://github.com/mahtabfaraji1/Semi-supervised-segmentation-of-cataract-surgical-images. Accurate segmentation of ocular anatomy and instruments supports surgical guidance, intraoperative decision making, and training tools in data-limited clinical environments.

We study image segmentation on the basis of locally excitatory globally inhibitory oscillator networks (LEGION), whereby the phases of oscillators encode the binding of pixels. We introduce a potential for each oscillator so that only those oscillators with strong connections from their neighborhoods can develop high potentials. Based on this concept, a solution to remove noisy regions in an image is proposed for LEGION, so that it suppresses the oscillators corresponding to noisy regions, without affecting those corresponding to major regions. The network is applied to segmenting real gray-level images and produces reasonable results. LEGION may provide a neurally plausible and effective framework for image segmentation and figure-ground segregation.

We present a novel framework for image segmentation based on the maximum likelihood estimator. A common hypothesis for explaining the differences among image regions is that they are generated by sampling different likelihood functions called models. In this work, we construct on last hypothesis and, additionally, we assume that such samples come from independent and identically distributed random variables. Thus, the probability (likelihood) that a particular model generates the observed value (at a given pixel) is estimated by computing the likelihood of the sample composed with the surrounding pixels. This simple approach allows us to propose efficient segmentation methods able to deal with textured images. Our approach is naturally extended for combining different features. Experiments in interactive image segmentation, automatic stereo analysis and denoising of brain water diffusion multi-tensor fields demonstrate the capabilities of our approach.

Semantic segmentation is a fundamental task in image processing and computer vision domains. Therefore, it is a core component of most emerging industry applications such as medical imaging, autonomous driving, and agriculture. The last five years have witnessed a huge growth in the computer vision domain with the introduction of Generative Adversarial Networks (GANs) due to their ability to learn deep representations by a competitive process between two neural networks called the Generator the Discriminator. Neural Architecture Search (NAS) is undergoing a revolution aimed at automating the architectural engineering process. This paper discusses implementing a novel framework for semantic image segmentation called iSegmentor using NAS and GANs. The proposed approach is the first framework that utilizes NAS in GANs for semantic image segmentation to the best of our knowledge. The iSegmentor framework was evaluated using mean Intersection over Union (mIoU) and Mean Pixel Accuracy (MPA) metrics. The iSegmentor was mainly tested and evaluated for the autonomous driving domain. With the analysis of the past five-year literature, Cityscapes and CamVid datasets were chosen. The iSegmentor achieved 81.73 MPA and 71.90 mIoU for the Cityscapes dataset, whereas CamVid achieved 80.41 MPA and 70.63 mIoU. Further, we present future avenues that we identified with the conducted experiments to assist other researchers to extend the research area of NAS in GANs in semantic segmentation.

Deep learning models have obtained state-of-the-art results for medical image analysis. However, CNNs require a massive amount of labelled data to achieve a high performance. Moreover, many supervised learning approaches assume that the training/source dataset and test/target dataset follow the same probability distribution. Nevertheless, this assumption is hardly satisfied in real-world data and when the models are tested on an unseen domain there is a significant performance degradation. In this work, we present an unsupervised Cross-Modality Adversarial Domain Adaptation (C-MADA) framework for medical image segmentation. C-MADA implements an image-level and feature-level adaptation method in a two-step sequential manner. First, images from the source domain are translated to the target domain through an unpaired image-to-image adversarial translation with cycle-consistency loss. Then, a U-Net network is trained with the mapped source domain images and target domain images in an adversarial manner to learn domain-invariant feature representations and produce segmentations for the target domain. Furthermore, to improve the network’s segmentation performance, information about the shape, texture, and contour of the predicted segmentation is included during the adversarial training. C-MADA is tested on the task of brain MRI segmentation from the crossMoDa Grand Challenge and is ranked within the top 15 submissions of the challenge.

Neighbouring-slice Guided Multi-View Framework for brain image segmentation