An Interactive Image Segmentation Method Based on Multi-Level Semantic Fusion.

Interactive image segmentation is a challenging task and receives increasing attention recently; however, two major drawbacks exist in interactive segmentation approaches. First, the segmentation performance of region of interest (ROI)-based methods is sensitive to the initial ROI: different ROIs may produce results with great difference. Second, most seed-based methods need intense interactions, and are not applicable in many cases. In this paper, we generalize the neutro-connectedness (NC) to be independent of top-down priors of objects and to model image topology with indeterminacy measurement on image regions, propose a novel method for determining object and background regions, which is applied to exclude isolated background regions and enforce label consistency, and put forward a hybrid interactive segmentation method, NC Cut (NC-Cut), which can overcome the above two problems by utilizing both pixelwise appearance information and region-based NC properties. We evaluate the proposed NC-Cut by employing two image data sets (265 images), and demonstrate that the proposed approach outperforms the state-of-the-art interactive image segmentation methods (Grabcut, MILCut, One-Cut, MGCmaxsum, and pPBC).

We propose an interactive object segmentation method which learns feature-specific segmentation parameters based on a single image. The first step is to design discriminative features for each pixel, which integrate four kinds of cues, i.e, the color Gaussian mixture model (GMM), the graph learning-based attribute, the texture GMM, and the geodesic distance. Then we formulate the segmentation problem as a conditional random field model in terms of fusing multiple features. While an image-specific parameter setting is practical in interactive segmentation, the efficiency of learning process highly depends on the type of user interaction and the designed features. We propose a feature-specific parameter learning strategy to learn model parameters, in which an offline training stage is not required and parameters are computed according to some sparsely labeled pixels on the basis of a single image. Extensive experiments show that the proposed segmentation model performs well for segmenting images with a weak boundary, texture, or cluttered background. Comparative experiment results demonstrate that our method can achieve both qualitative and quantitative improvements over other state-of-the-art interactive segmentation methods.

In this paper, a novel interactive medical image segmentation method called SMOPL is proposed. This method only needs marking some pixels on foreground region for segmentation. To do this, SMOPL characterize the inherent correlations among foreground and background pixels as Hilbert-Schmidt independence. By maximizing the independence and minimizing the smoothness of labels on instance neighbor graph simultaneously, SMOPL gets the sufficiently smooth confidences of both positive and negative classes in absence of negative training examples. Then a image segmentation can be obtained by assigning each pixel to the label for which the greatest confidence is calculated. Experiments on real-world medical images show that SMOPL is robust to get a high-quality segmentation with only positive label examples.

Interactive segmentation methods utilize user-provided positive and negative clicks to guide the model in accurately segmenting target objects. Compared to fully automatic medical image segmentation, these methods can achieve higher segmentation accuracy with limited image data, demonstrating significant potential in clinical applications. Typically, for each new click provided by the user, conventional interactive segmentation methods reprocess the entire network by re-inputting the click into the segmentation model, which greatly increases the user’s interaction burden and deviates from the intended goal of interactive segmentation tasks. To address this issue, we propose an efficient segmentation network, ESM-Net, for interactive medical image segmentation. It obtains high-quality segmentation masks based on the user’s initial clicks, reducing the complexity of subsequent refinement steps. Recent studies have demonstrated the strong performance of the Mamba model in various vision tasks; however, its application in interactive segmentation remains unexplored. In our study, we incorporate the Mamba module into our framework for the first time and enhance its spatial representation capabilities by developing a Spatial Augmented Convolution (SAC) module. These components are combined as the fundamental building blocks of our network. Furthermore, we designed a novel and efficient segmentation head to fuse multi-scale features extracted from the encoder, optimizing the generation of the predicted segmentation masks. Through comprehensive experiments, our method achieved state-of-the-art performance on three medical image datasets. Specifically, we achieved 1.43 NoC@90 on the Kvasir-SEG dataset, 1.57 NoC@90 on the CVC-ClinicDB polyp segmentation dataset, and 1.03 NoC@90 on the ADAM retinal disk segmentation dataset. The assessments on these three medical image datasets highlight the effectiveness of our approach in interactive medical image segmentation.

One of the most important goals in image segmentation is the process of separating the object parts from the image background. Image segmentation is also a fundamental stage in the development of other image applications such as object recognition, target tracking, computer vision, and biomedical image processing. Interactive image segmentation methods with additional user interaction are still popular in research. Interactive image segmentation aims to provide additional information through simple interactions, especially in images with complex objects. Interactive image segmentation with region merging processes has drawbacks, one of which is suboptimal region splitting due to soft color shades, blurred contours, and uneven lighting, referred to in this study as ambiguous regions. However, in the fuzzy region initialization stage after obtaining values from the marker process, there is a possibility of missing or suboptimal determination of fuzzy regions. This is because it only takes the highest gray level value for the background marker and the lowest gray level value for the object marker. In this study, fuzzy region merging using hierarchical clustering is proposed to find optimal initialization for fuzzy regions in image segmentation. Based on the experimental results, the proposed method can achieve optimal segmentation with an average misclassification error value of 2.62% for Natural Images and 9.33% for Dental Images.

Image segmentation is a necessary but difficult task in many image processing applications. Unlike conventional auto-segmentation, semi-supervised image segmentation involves a moderate amount of user interaction. In this paper, a novel interactive image segmentation method based on diffusion maps is proposed, which can better account for the distribution of pixels in feature space. Our method replaces the pixel features of Euclidean distance with diffusion maps and performs feature optimization and dimensionality reduction in order to achieve a final feature vector, with the assistance of user interaction. On this basis, L0 gradient minimization is applied to achieve steady segmentation results. Finally, the K-means clustering algorithm is applied to perfect the segmentation. Experiments show that our method can obtain better segmentation results than several state-of-the-art interactive image segmentation methods.

Many medical image analysis systems require segmentation of the structures of interest as a first step. For scans with gross pathology, automatic segmentation methods may fail. The authors' aim is to develop a versatile, fast, and reliable interactive system to segment anatomical structures. In this study, this system was used for segmenting lungs in challenging thoracic computed tomography (CT) scans. In volumetric thoracic CT scans, the chest is segmented and divided into 3D volumes of interest (VOIs), containing voxels with similar densities. These VOIs are automatically labeled as either lung tissue or nonlung tissue. The automatic labeling results can be corrected using an interactive or a supervised interactive approach. When using the supervised interactive system, the user is shown the classification results per slice, whereupon he/she can adjust incorrect labels. The system is retrained continuously, taking the corrections and approvals of the user into account. In this way, the system learns to make a better distinction between lung tissue and nonlung tissue. When using the interactive framework without supervised learning, the user corrects all incorrectly labeled VOIs manually. Both interactive segmentation tools were tested on 32 volumetric CT scans of pigs, mice and humans, containing pulmonary abnormalities. On average, supervised interactive lung segmentation took under 9 min of user interaction. Algorithm computing time was 2 min on average, but can easily be reduced. On average, 2.0% of all VOIs in a scan had to be relabeled. Lung segmentation using the interactive segmentation method took on average 13 min and involved relabeling 3.0% of all VOIs on average. The resulting segmentations correspond well to manual delineations of eight axial slices per scan, with an average Dice similarity coefficient of 0.933. The authors have developed two fast and reliable methods for interactive lung segmentation in challenging chest CT images. Both systems do not require prior knowledge of the scans under consideration and work on a variety of scans.

The Pascal Visual Object Classes (VOC) challenge is a benchmark in visual object category recognition and detection, providing the vision and machine learning communities with a standard dataset of images and annotation, and standard evaluation procedures. Organised annually from 2005 to present, the challenge and its associated dataset has become accepted as the benchmark for object detection. This paper describes the dataset and evaluation procedure. We review the state-of-the-art in evaluated methods for both classification and detection, analyse whether the methods are statistically different, what they are learning from the images (e.g. the object or its context), and what the methods find easy or confuse. The paper concludes with lessons learnt in the three year history of the challenge, and proposes directions for future improvement and extension.

Image segmentation for object to extract data from ultrasound acquired is an essential preprocessing step for the effective diagnosis. Various image segmentation methods have been studied. In this study, interactive image segmentation method by graph cut algorithm is proposed to develop a variety of applications of vascular ultrasound imaging and diagnostics. General imaging and vascular ultrasound imaging segmentation by entering constrain condition such as foreground and background. In the future it will be able to develop new ultrasound diagnostics.

It is hard for current interactive stereo image segmentation methods to deal with large scale images with fast feedback after each interaction. In this paper, we present an interactive stereo image segmentation method. Different from current methods, our method introduces a multi-scale graph structure for fast graph cut optimization. Besides, we use GPU parallel computing to handle single instruction multiple data tasks involved in the segmentation. Compared with state-of-the-art methods, our approach significantly accelerates segmentation speed. In the meanwhile, our method obtains segmentation accuracy comparable with state-of-the-art methods.

The GrabCut, which uses the graph-cut iteratively, is popularly used as an interactive image segmentation method since it can produce the globally optimal result. However, since the initialization of the GrabCut is roughly performed by the manual interaction, the accuracy of the segmentation result is not guaranteed when the user defines an inaccurate guide. To solve this problem, in this paper, we present an improved GrabCut method which uses a visual saliency of the target region for the effective initialization. Experimental results demonstrate that the proposed method provides more accurate segmentation results compared with the conventional method.

It has been well established that there is a positive correlation between the dense appearance of breast stroma and parenchyma on a mammogram and the risk of breast cancer. Subjective assessment by radiologists indicated relative risks on the order of 4 to 6 for the group of women whose mammograms showed a density of over 75% or more of the projected area compared to those with an absence of density. In order to obtain a more quantitative, continuous and reproducible means of estimating breast density, which is sensitive to small changes, we have developed quantitative methods for the analysis of mammographic density, which can be applied to digitized mammograms. These techniques have been validated in a nested case-control study on 708 women aged 40-59 years (on entry) who participated in a national mammographic screening study. An interactive image segmentation method and two completely automated techniques based on image texture and grey scale histogram measures have been developed and evaluated. While our methods all show statistically significant risk factors for dense breasts, the interactive method currently provides the highest risk values (relative risk 4.0, 95% confidence interval (CI) = 2.12-7.56) compared to a measure based on the shape of the image histogram (relative risk 3.35, 95% CI = 1.57-7.12) or the fractal dimension of the mammogram (relative risk 2.54, 95% CI = 1.14-5.68). All methods were highly consistent between images of the left and right breast and between the two standard views (cranio-caudal and medio-lateral oblique) of each breast, so that studies can be done by sampling only one of the four views per examination. There is a large number of factors in addition to breast density which affect the appearance of the mammogram. In particular, the assessment of density is made difficult where the breast is not uniformly compressed, e.g. at the periphery. We have designed and are currently evaluating an image processing algorithm that effectively corrects for this problem and have considered methods for controlling some of the variables of image acquisition in prospective studies. Measurements of breast density may be helpful in assigning risk groups to women. Such measurements might guide the frequency of mammographic screening, aid the study of breast cancer aetiology, and be useful in monitoring possible risk-modifying interventions. Using our techniques, we have been able to show that reduction of the proportion of fat in the diet can result in reductions of breast density, although the direct connection to risk has not yet been made. The relationship between breast density and hormone-related and genetic factors is also of great interest. It is often not possible or ethical to obtain mammograms on some groups of women for whom information on density would be very useful. This includes younger women as well as groups in which it would be desirable to obtain such information at frequent intervals. For this reason, we are exploring the use of imaging approaches such as ultrasound and magnetic resonance imaging, which do not require ionizing radiation, to make measurements analogous to those now being performed by using X-ray mammograms.

MIDeepSeg: Minimally interactive segmentation of unseen objects from medical images using deep learning.

Interactive segmentation (IS) using minimal prompts like points and bounding boxes facilitates rapid image annotation, which is crucial for enhancing data-driven deep learning methods. Traditional IS methods, however, process only one target per interaction, leading to inefficiency when annotating multiple identical-class objects in remote sensing imagery (RSI). To address this issue, we present a new task—identical-class object detection (ICOD) for rapid IS in RSI. This task aims to only identify and detect all identical-class targets within an image, guided by a specific category target in the image with its mask. For this task, we propose an ICOD network (ICODet) with a two-stage object detection framework, which consists of a backbone, feature similarity analysis module (S3QFM), and an identical-class object detector. In particular, the S3QFM analyzes feature similarities from images and support objects at both feature-space and semantic levels, generating similarity maps. These maps are processed by a region proposal network (RPN) to extract target-level features, which are then refined through a simple feature comparison module and classified to precisely identify identical-class targets. To evaluate the effectiveness of this method, we construct two datasets for the ICOD task: one containing a diverse set of buildings and another containing multicategory RSI objects. Experimental results show that our method outperforms the compared methods on both datasets. This research introduces a new method for rapid IS of RSI and advances the development of fast interaction modes, offering significant practical value for data production and fundamental applications in the remote sensing community.

Object detection is a technology that automatically identifies and locates specific objects in images or videos and plays a core role in various fields, such as autonomous driving, security surveillance, and medical imaging. You Only Look Once (YOLO) has gained attention for achieving both accuracy and detection speed in real-time applications; however, during the resolution reduction process, detailed information is lost, and unnecessary signals are mixed in the multi-scale feature fusion stage, resulting in limited detection performance for small objects and complex background scenes. To alleviate these limitations, we propose YOLO-RECAP, which integrates a Content-Aware ReAssembly of FEatures (CARAFE) and Efficient Channel Attention (ECA) modules based on YOLOv11. CARAFE precisely restores boundaries and shapes during the upsampling stage by utilizing position-specific content information, whereas ECA effectively models interchannel interactions to emphasize important signals. For performance verification, VisDrone2019, Store Keeping Unit-110 K (SKU-110 K), Pascal Visual Object Classes (VOC), and Dataset for Object Detection in Aerial Images (DOTA)v1 were used. In addition, Latency is reported under an end-to-end setting that includes pre-processing, inference, and post-processing, to reflect practical deployment conditions. As a result, YOLO-RECAP achieved mAP50 of 0.316, mAP50@95 of 0.184, Latency of 16.5 ms, and 60.6 FPS on VisDrone2019; mAP50 of 0.895, mAP50@95 of 0.572, Latency of 18.2 ms, and 55.0 FPS on SKU-110 K; mAP50 of 0.770, mAP50@95 of 0.561, Latency of 15.6 ms, and 63.9 FPS on Pascal VOC; and mAP50 of 0.281, mAP50@95 of 0.157, Latency of 18.6 ms, and 53.9 FPS on DOTAv1. Qualitative bounding-box visualizations further indicate reduced missed detections and more stable predictions in cluttered and densely populated scenes. As a result, YOLO-RECAP provided a more stable and balanced detection performance than the existing YOLOv11 and recent detection models, especially for small objects and complex backgrounds. This code is available at https://github.com/Heon-ju/YOLO-RECAP.git .