A survey of recent interactive image segmentation methods

Interactive segmentation methods utilize user-provided positive and negative clicks to guide the model in accurately segmenting target objects. Compared to fully automatic medical image segmentation, these methods can achieve higher segmentation accuracy with limited image data, demonstrating significant potential in clinical applications. Typically, for each new click provided by the user, conventional interactive segmentation methods reprocess the entire network by re-inputting the click into the segmentation model, which greatly increases the user’s interaction burden and deviates from the intended goal of interactive segmentation tasks. To address this issue, we propose an efficient segmentation network, ESM-Net, for interactive medical image segmentation. It obtains high-quality segmentation masks based on the user’s initial clicks, reducing the complexity of subsequent refinement steps. Recent studies have demonstrated the strong performance of the Mamba model in various vision tasks; however, its application in interactive segmentation remains unexplored. In our study, we incorporate the Mamba module into our framework for the first time and enhance its spatial representation capabilities by developing a Spatial Augmented Convolution (SAC) module. These components are combined as the fundamental building blocks of our network. Furthermore, we designed a novel and efficient segmentation head to fuse multi-scale features extracted from the encoder, optimizing the generation of the predicted segmentation masks. Through comprehensive experiments, our method achieved state-of-the-art performance on three medical image datasets. Specifically, we achieved 1.43 NoC@90 on the Kvasir-SEG dataset, 1.57 NoC@90 on the CVC-ClinicDB polyp segmentation dataset, and 1.03 NoC@90 on the ADAM retinal disk segmentation dataset. The assessments on these three medical image datasets highlight the effectiveness of our approach in interactive medical image segmentation.

One of the most important goals in image segmentation is the process of separating the object parts from the image background. Image segmentation is also a fundamental stage in the development of other image applications such as object recognition, target tracking, computer vision, and biomedical image processing. Interactive image segmentation methods with additional user interaction are still popular in research. Interactive image segmentation aims to provide additional information through simple interactions, especially in images with complex objects. Interactive image segmentation with region merging processes has drawbacks, one of which is suboptimal region splitting due to soft color shades, blurred contours, and uneven lighting, referred to in this study as ambiguous regions. However, in the fuzzy region initialization stage after obtaining values from the marker process, there is a possibility of missing or suboptimal determination of fuzzy regions. This is because it only takes the highest gray level value for the background marker and the lowest gray level value for the object marker. In this study, fuzzy region merging using hierarchical clustering is proposed to find optimal initialization for fuzzy regions in image segmentation. Based on the experimental results, the proposed method can achieve optimal segmentation with an average misclassification error value of 2.62% for Natural Images and 9.33% for Dental Images.

Interactive image segmentation is a challenging task and receives increasing attention recently; however, two major drawbacks exist in interactive segmentation approaches. First, the segmentation performance of region of interest (ROI)-based methods is sensitive to the initial ROI: different ROIs may produce results with great difference. Second, most seed-based methods need intense interactions, and are not applicable in many cases. In this paper, we generalize the neutro-connectedness (NC) to be independent of top-down priors of objects and to model image topology with indeterminacy measurement on image regions, propose a novel method for determining object and background regions, which is applied to exclude isolated background regions and enforce label consistency, and put forward a hybrid interactive segmentation method, NC Cut (NC-Cut), which can overcome the above two problems by utilizing both pixelwise appearance information and region-based NC properties. We evaluate the proposed NC-Cut by employing two image data sets (265 images), and demonstrate that the proposed approach outperforms the state-of-the-art interactive image segmentation methods (Grabcut, MILCut, One-Cut, MGCmaxsum, and pPBC).

Deep learning-based segmentation algorithms for medical image require massive training datasets with accurate annotations, which is costly since it takes much human effort to manually labeling from scratch. Therefore, interactive image segmentation is important and may greatly improve the efficiency and accuracy of medical image labeling. Some interactive segmentation methods (e.g. Deep Extreme Cut and Deepgrow) may improve the labeling through minimal interactive input. However these methods only utilize the initial manually input information, while existing segmentation results (such as annotations produced by nonprofessionals or conventional segmentation algorithms) cannot be utilized. In this paper, an interactive segmentation method is proposed to make use of both existing segmentation results and human interactive information to optimize the segmentation results progressively. In this framework, the user only needs to click on the foreground or background of the target individual on the medical image, the algorithm could adaptively learn the correlation between them, and automatically completes the segmentation of the target. The main contributions of this paper are: (1) We adjusted and applied a convolutional neural network which takes medical image data and user's clicks information as input to achieve more accurate segmentation of medical images. (2) We designed an iterative training strategy to realize the applicability of the model to deal with different number of clicks data input. (3) We designed an algorithm based on false positive and false negative regions to simulate the user's clicks, so as to provide enough training data. By applying the proposed method, users can easily extract the region of interest or modify the segmentation results by multiple clicks. The experimental results of 6 medical image segmentation tasks show that the proposed method achieves more accurate segmentation results by at most five clicks.

Image segmentation is a necessary but difficult task in many image processing applications. Unlike conventional auto-segmentation, semi-supervised image segmentation involves a moderate amount of user interaction. In this paper, a novel interactive image segmentation method based on diffusion maps is proposed, which can better account for the distribution of pixels in feature space. Our method replaces the pixel features of Euclidean distance with diffusion maps and performs feature optimization and dimensionality reduction in order to achieve a final feature vector, with the assistance of user interaction. On this basis, L0 gradient minimization is applied to achieve steady segmentation results. Finally, the K-means clustering algorithm is applied to perfect the segmentation. Experiments show that our method can obtain better segmentation results than several state-of-the-art interactive image segmentation methods.

Interactive segmentation has recently been explored to effectively and efficiently harvest high-quality segmentation masks by iteratively incorporating user hints. While iterative in nature, most existing interactive segmentation methods tend to ignore the dynamics of successive interactions and take each interaction independently. We here propose to model iterative interactive image segmentation with a Markov decision process (MDP) and solve it with reinforcement learning (RL) where each voxel is treated as an agent. Considering the large exploration space for voxel-wise prediction and the dependence among neighboring voxels for the segmentation tasks, multi-agent reinforcement learning is adopted, where the voxel-level policy is shared among agents. Considering that boundary voxels are more important for segmentation, we further introduce a boundary-aware reward, which consists of a global reward in the form of relative cross-entropy gain, to update the policy in a constrained direction, and a boundary reward in the form of relative weight, to emphasize the correctness of boundary predictions. To combine the advantages of different types of interactions, i. e., simple and efficient for point-clicking, and stable and robust for scribbles, we propose a supervoxel-clicking based interaction design. Experimental results on four benchmark datasets have shown that the proposed method significantly outperforms the state-of-the-arts, with the advantage of fewer interactions, higher accuracy, and enhanced robustness.

In this paper, a novel interactive medical image segmentation method called SMOPL is proposed. This method only needs marking some pixels on foreground region for segmentation. To do this, SMOPL characterize the inherent correlations among foreground and background pixels as Hilbert-Schmidt independence. By maximizing the independence and minimizing the smoothness of labels on instance neighbor graph simultaneously, SMOPL gets the sufficiently smooth confidences of both positive and negative classes in absence of negative training examples. Then a image segmentation can be obtained by assigning each pixel to the label for which the greatest confidence is calculated. Experiments on real-world medical images show that SMOPL is robust to get a high-quality segmentation with only positive label examples.

Image segmentation for object to extract data from ultrasound acquired is an essential preprocessing step for the effective diagnosis. Various image segmentation methods have been studied. In this study, interactive image segmentation method by graph cut algorithm is proposed to develop a variety of applications of vascular ultrasound imaging and diagnostics. General imaging and vascular ultrasound imaging segmentation by entering constrain condition such as foreground and background. In the future it will be able to develop new ultrasound diagnostics.

It is hard for current interactive stereo image segmentation methods to deal with large scale images with fast feedback after each interaction. In this paper, we present an interactive stereo image segmentation method. Different from current methods, our method introduces a multi-scale graph structure for fast graph cut optimization. Besides, we use GPU parallel computing to handle single instruction multiple data tasks involved in the segmentation. Compared with state-of-the-art methods, our approach significantly accelerates segmentation speed. In the meanwhile, our method obtains segmentation accuracy comparable with state-of-the-art methods.

This paper proposes an interactive image segmentation method based on self-organizing maps (SOMs). In this paper, not only two dimensional SOMs but also higher dimensional SOMs are used for image segmentation. The proposed method was applied to actual images. The experimental results demonstrated that higher dimensional SOMs were able to achieve more accurate segmentation.

We propose an interactive object segmentation method which learns feature-specific segmentation parameters based on a single image. The first step is to design discriminative features for each pixel, which integrate four kinds of cues, i.e, the color Gaussian mixture model (GMM), the graph learning-based attribute, the texture GMM, and the geodesic distance. Then we formulate the segmentation problem as a conditional random field model in terms of fusing multiple features. While an image-specific parameter setting is practical in interactive segmentation, the efficiency of learning process highly depends on the type of user interaction and the designed features. We propose a feature-specific parameter learning strategy to learn model parameters, in which an offline training stage is not required and parameters are computed according to some sparsely labeled pixels on the basis of a single image. Extensive experiments show that the proposed segmentation model performs well for segmenting images with a weak boundary, texture, or cluttered background. Comparative experiment results demonstrate that our method can achieve both qualitative and quantitative improvements over other state-of-the-art interactive segmentation methods.

MIDeepSeg: Minimally interactive segmentation of unseen objects from medical images using deep learning.

Interactive image segmentation based on samples reconstruction and FLDA

This study investigates the impact of image complexity on the accuracy of interactive image segmentation algorithms. Image complexity plays a crucial role in segmentation performance, yet previous studies have primarily relied on subjective methods, leaving a gap in understanding how objective measures impact accuracy. The purpose of this research is to explore the relationship between image complexity and segmentation performance and to propose an adaptive approach for improving accuracy based on complexity measures. The study utilizes objective measures, namely entropy and fractal dimension, to quantify image complexity. An interactive image segmentation algorithm is employed, with a bounding box as the background and strokes as the foreground annotations. The number of strokes is dynamically adjusted based on complexity measures, ensuring a tailored segmentation approach. Comparative evaluations are conducted to assess the effectiveness of dynamic and fixed stroke allocation strategies. The principal results reveal a significant influence of image complexity on segmentation accuracy. The dynamic stroke allocation strategy outperforms fixed allocation, highlighting the importance of adapting to complexity. Moreover, the optimal combination of strokes and superpixel sizes is explored, providing valuable insights for practitioners. The findings emphasize the need to consider image complexity in segmentation algorithms to achieve accurate results. In conclusion, this study contributes to the understanding of the relationship between image complexity and interactive image segmentation. By introducing a dynamic stroke allocation approach and evaluating different configurations, the research provides insights into optimizing accuracy based on image complexity. The adaptive strategy improves segmentation performance and guides the development of robust algorithms. Future research can further refine the adaptive approach, explore additional complexity measures, and incorporate advanced machine learning techniques to enhance interactive image segmentation. Overall, this study advances the field by highlighting the importance of image complexity, providing guidance for practitioners, and paving the way for more efficient segmentation algorithms.

Many medical image analysis systems require segmentation of the structures of interest as a first step. For scans with gross pathology, automatic segmentation methods may fail. The authors' aim is to develop a versatile, fast, and reliable interactive system to segment anatomical structures. In this study, this system was used for segmenting lungs in challenging thoracic computed tomography (CT) scans. In volumetric thoracic CT scans, the chest is segmented and divided into 3D volumes of interest (VOIs), containing voxels with similar densities. These VOIs are automatically labeled as either lung tissue or nonlung tissue. The automatic labeling results can be corrected using an interactive or a supervised interactive approach. When using the supervised interactive system, the user is shown the classification results per slice, whereupon he/she can adjust incorrect labels. The system is retrained continuously, taking the corrections and approvals of the user into account. In this way, the system learns to make a better distinction between lung tissue and nonlung tissue. When using the interactive framework without supervised learning, the user corrects all incorrectly labeled VOIs manually. Both interactive segmentation tools were tested on 32 volumetric CT scans of pigs, mice and humans, containing pulmonary abnormalities. On average, supervised interactive lung segmentation took under 9 min of user interaction. Algorithm computing time was 2 min on average, but can easily be reduced. On average, 2.0% of all VOIs in a scan had to be relabeled. Lung segmentation using the interactive segmentation method took on average 13 min and involved relabeling 3.0% of all VOIs on average. The resulting segmentations correspond well to manual delineations of eight axial slices per scan, with an average Dice similarity coefficient of 0.933. The authors have developed two fast and reliable methods for interactive lung segmentation in challenging chest CT images. Both systems do not require prior knowledge of the scans under consideration and work on a variety of scans.