Co-Sparse Textural Similarity for Interactive Segmentation

Interactive segmentation methods utilize user-provided positive and negative clicks to guide the model in accurately segmenting target objects. Compared to fully automatic medical image segmentation, these methods can achieve higher segmentation accuracy with limited image data, demonstrating significant potential in clinical applications. Typically, for each new click provided by the user, conventional interactive segmentation methods reprocess the entire network by re-inputting the click into the segmentation model, which greatly increases the user’s interaction burden and deviates from the intended goal of interactive segmentation tasks. To address this issue, we propose an efficient segmentation network, ESM-Net, for interactive medical image segmentation. It obtains high-quality segmentation masks based on the user’s initial clicks, reducing the complexity of subsequent refinement steps. Recent studies have demonstrated the strong performance of the Mamba model in various vision tasks; however, its application in interactive segmentation remains unexplored. In our study, we incorporate the Mamba module into our framework for the first time and enhance its spatial representation capabilities by developing a Spatial Augmented Convolution (SAC) module. These components are combined as the fundamental building blocks of our network. Furthermore, we designed a novel and efficient segmentation head to fuse multi-scale features extracted from the encoder, optimizing the generation of the predicted segmentation masks. Through comprehensive experiments, our method achieved state-of-the-art performance on three medical image datasets. Specifically, we achieved 1.43 NoC@90 on the Kvasir-SEG dataset, 1.57 NoC@90 on the CVC-ClinicDB polyp segmentation dataset, and 1.03 NoC@90 on the ADAM retinal disk segmentation dataset. The assessments on these three medical image datasets highlight the effectiveness of our approach in interactive medical image segmentation.

We propose an interactive object segmentation method which learns feature-specific segmentation parameters based on a single image. The first step is to design discriminative features for each pixel, which integrate four kinds of cues, i.e, the color Gaussian mixture model (GMM), the graph learning-based attribute, the texture GMM, and the geodesic distance. Then we formulate the segmentation problem as a conditional random field model in terms of fusing multiple features. While an image-specific parameter setting is practical in interactive segmentation, the efficiency of learning process highly depends on the type of user interaction and the designed features. We propose a feature-specific parameter learning strategy to learn model parameters, in which an offline training stage is not required and parameters are computed according to some sparsely labeled pixels on the basis of a single image. Extensive experiments show that the proposed segmentation model performs well for segmenting images with a weak boundary, texture, or cluttered background. Comparative experiment results demonstrate that our method can achieve both qualitative and quantitative improvements over other state-of-the-art interactive segmentation methods.

Many medical image analysis systems require segmentation of the structures of interest as a first step. For scans with gross pathology, automatic segmentation methods may fail. The authors' aim is to develop a versatile, fast, and reliable interactive system to segment anatomical structures. In this study, this system was used for segmenting lungs in challenging thoracic computed tomography (CT) scans. In volumetric thoracic CT scans, the chest is segmented and divided into 3D volumes of interest (VOIs), containing voxels with similar densities. These VOIs are automatically labeled as either lung tissue or nonlung tissue. The automatic labeling results can be corrected using an interactive or a supervised interactive approach. When using the supervised interactive system, the user is shown the classification results per slice, whereupon he/she can adjust incorrect labels. The system is retrained continuously, taking the corrections and approvals of the user into account. In this way, the system learns to make a better distinction between lung tissue and nonlung tissue. When using the interactive framework without supervised learning, the user corrects all incorrectly labeled VOIs manually. Both interactive segmentation tools were tested on 32 volumetric CT scans of pigs, mice and humans, containing pulmonary abnormalities. On average, supervised interactive lung segmentation took under 9 min of user interaction. Algorithm computing time was 2 min on average, but can easily be reduced. On average, 2.0% of all VOIs in a scan had to be relabeled. Lung segmentation using the interactive segmentation method took on average 13 min and involved relabeling 3.0% of all VOIs on average. The resulting segmentations correspond well to manual delineations of eight axial slices per scan, with an average Dice similarity coefficient of 0.933. The authors have developed two fast and reliable methods for interactive lung segmentation in challenging chest CT images. Both systems do not require prior knowledge of the scans under consideration and work on a variety of scans.

Deep learning-based segmentation algorithms for medical image require massive training datasets with accurate annotations, which is costly since it takes much human effort to manually labeling from scratch. Therefore, interactive image segmentation is important and may greatly improve the efficiency and accuracy of medical image labeling. Some interactive segmentation methods (e.g. Deep Extreme Cut and Deepgrow) may improve the labeling through minimal interactive input. However these methods only utilize the initial manually input information, while existing segmentation results (such as annotations produced by nonprofessionals or conventional segmentation algorithms) cannot be utilized. In this paper, an interactive segmentation method is proposed to make use of both existing segmentation results and human interactive information to optimize the segmentation results progressively. In this framework, the user only needs to click on the foreground or background of the target individual on the medical image, the algorithm could adaptively learn the correlation between them, and automatically completes the segmentation of the target. The main contributions of this paper are: (1) We adjusted and applied a convolutional neural network which takes medical image data and user's clicks information as input to achieve more accurate segmentation of medical images. (2) We designed an iterative training strategy to realize the applicability of the model to deal with different number of clicks data input. (3) We designed an algorithm based on false positive and false negative regions to simulate the user's clicks, so as to provide enough training data. By applying the proposed method, users can easily extract the region of interest or modify the segmentation results by multiple clicks. The experimental results of 6 medical image segmentation tasks show that the proposed method achieves more accurate segmentation results by at most five clicks.

Interactive image segmentation is a challenging task and receives increasing attention recently; however, two major drawbacks exist in interactive segmentation approaches. First, the segmentation performance of region of interest (ROI)-based methods is sensitive to the initial ROI: different ROIs may produce results with great difference. Second, most seed-based methods need intense interactions, and are not applicable in many cases. In this paper, we generalize the neutro-connectedness (NC) to be independent of top-down priors of objects and to model image topology with indeterminacy measurement on image regions, propose a novel method for determining object and background regions, which is applied to exclude isolated background regions and enforce label consistency, and put forward a hybrid interactive segmentation method, NC Cut (NC-Cut), which can overcome the above two problems by utilizing both pixelwise appearance information and region-based NC properties. We evaluate the proposed NC-Cut by employing two image data sets (265 images), and demonstrate that the proposed approach outperforms the state-of-the-art interactive image segmentation methods (Grabcut, MILCut, One-Cut, MGCmaxsum, and pPBC).

Introduction: Wound segmentation is a pivotal process of photography, which is an important and prevalent method for skin wound analysis and documentation in forensic science. However, due to the complexity and diversity of forensic wound images, no automatic segmentation algorithm is capable of judging the boundaries of the wound accurately. Meanwhile, the currently approach using lasso tool, though accurate, is time-consuming and labor-intensive for practical application. Materials and Methods: In this paper, interactive segmentation algorithms are applied to skin wound extraction. This application improves the performance of traditional segmentation methods and reduces the time of manual method. In order to verify the segmentation, a software platform hosting image preprocessing, image segmentation and wound area calculation is developed. This platform integrates the manual method and three interactive segmentation methods, and is completely available for forensic wound measurement based on photography technique. Results: The segmentation accuracy is evaluated by precision, recall and F-measure, which are higher than 0.96, 0.95, and 0.96 respectively. The error of the area calculation is less than 5%, with respect to the golden standard. Discussion: These results demonstrate that interactive segmentation is practicable and precise enough to meet the demands of skin wounds measurement based on photography in forensic medicine.KeywordsInteractive segmentationWound measurementForensic medicinePhotography

MIDeepSeg: Minimally interactive segmentation of unseen objects from medical images using deep learning.

Understanding and analyzing 2D/3D sensor data is crucial for a wide range of machine learning-based applications, including object detection, scene segmentation, and salient object detection. In this context, interactive object segmentation is a vital task in image editing and medical diagnosis, involving the accurate separation of the target object from its background based on user annotation information. However, existing interactive object segmentation methods struggle to effectively leverage such information to guide object-segmentation models. To address these challenges, this paper proposes an interactive image-segmentation technique for static images based on multi-level semantic fusion. Our method utilizes user-guidance information both inside and outside the target object to segment it from the static image, making it applicable to both 2D and 3D sensor data. The proposed method introduces a cross-stage feature aggregation module, enabling the effective propagation of multi-scale features from previous stages to the current stage. This mechanism prevents the loss of semantic information caused by multiple upsampling and downsampling of the network, allowing the current stage to make better use of semantic information from the previous stage. Additionally, we incorporate a feature channel attention mechanism to address the issue of rough network segmentation edges. This mechanism captures richer feature details from the feature channel level, leading to finer segmentation edges. In the experimental evaluation conducted on the PASCAL Visual Object Classes (VOC) 2012 dataset, our proposed interactive image segmentation method based on multi-level semantic fusion demonstrates an intersection over union (IOU) accuracy approximately 2.1% higher than the currently popular interactive image segmentation method in static images. The comparative analysis highlights the improved performance and effectiveness of our method. Furthermore, our method exhibits potential applications in various fields, including medical imaging and robotics. Its compatibility with other machine learning methods for visual semantic analysis allows for integration into existing workflows. These aspects emphasize the significance of our contributions in advancing interactive image-segmentation techniques and their practical utility in real-world applications.

This study investigates the impact of image complexity on the accuracy of interactive image segmentation algorithms. Image complexity plays a crucial role in segmentation performance, yet previous studies have primarily relied on subjective methods, leaving a gap in understanding how objective measures impact accuracy. The purpose of this research is to explore the relationship between image complexity and segmentation performance and to propose an adaptive approach for improving accuracy based on complexity measures. The study utilizes objective measures, namely entropy and fractal dimension, to quantify image complexity. An interactive image segmentation algorithm is employed, with a bounding box as the background and strokes as the foreground annotations. The number of strokes is dynamically adjusted based on complexity measures, ensuring a tailored segmentation approach. Comparative evaluations are conducted to assess the effectiveness of dynamic and fixed stroke allocation strategies. The principal results reveal a significant influence of image complexity on segmentation accuracy. The dynamic stroke allocation strategy outperforms fixed allocation, highlighting the importance of adapting to complexity. Moreover, the optimal combination of strokes and superpixel sizes is explored, providing valuable insights for practitioners. The findings emphasize the need to consider image complexity in segmentation algorithms to achieve accurate results. In conclusion, this study contributes to the understanding of the relationship between image complexity and interactive image segmentation. By introducing a dynamic stroke allocation approach and evaluating different configurations, the research provides insights into optimizing accuracy based on image complexity. The adaptive strategy improves segmentation performance and guides the development of robust algorithms. Future research can further refine the adaptive approach, explore additional complexity measures, and incorporate advanced machine learning techniques to enhance interactive image segmentation. Overall, this study advances the field by highlighting the importance of image complexity, providing guidance for practitioners, and paving the way for more efficient segmentation algorithms.

It has been well established that there is a positive correlation between the dense appearance of breast stroma and parenchyma on a mammogram and the risk of breast cancer. Subjective assessment by radiologists indicated relative risks on the order of 4 to 6 for the group of women whose mammograms showed a density of over 75% or more of the projected area compared to those with an absence of density. In order to obtain a more quantitative, continuous and reproducible means of estimating breast density, which is sensitive to small changes, we have developed quantitative methods for the analysis of mammographic density, which can be applied to digitized mammograms. These techniques have been validated in a nested case-control study on 708 women aged 40-59 years (on entry) who participated in a national mammographic screening study. An interactive image segmentation method and two completely automated techniques based on image texture and grey scale histogram measures have been developed and evaluated. While our methods all show statistically significant risk factors for dense breasts, the interactive method currently provides the highest risk values (relative risk 4.0, 95% confidence interval (CI) = 2.12-7.56) compared to a measure based on the shape of the image histogram (relative risk 3.35, 95% CI = 1.57-7.12) or the fractal dimension of the mammogram (relative risk 2.54, 95% CI = 1.14-5.68). All methods were highly consistent between images of the left and right breast and between the two standard views (cranio-caudal and medio-lateral oblique) of each breast, so that studies can be done by sampling only one of the four views per examination. There is a large number of factors in addition to breast density which affect the appearance of the mammogram. In particular, the assessment of density is made difficult where the breast is not uniformly compressed, e.g. at the periphery. We have designed and are currently evaluating an image processing algorithm that effectively corrects for this problem and have considered methods for controlling some of the variables of image acquisition in prospective studies. Measurements of breast density may be helpful in assigning risk groups to women. Such measurements might guide the frequency of mammographic screening, aid the study of breast cancer aetiology, and be useful in monitoring possible risk-modifying interventions. Using our techniques, we have been able to show that reduction of the proportion of fat in the diet can result in reductions of breast density, although the direct connection to risk has not yet been made. The relationship between breast density and hormone-related and genetic factors is also of great interest. It is often not possible or ethical to obtain mammograms on some groups of women for whom information on density would be very useful. This includes younger women as well as groups in which it would be desirable to obtain such information at frequent intervals. For this reason, we are exploring the use of imaging approaches such as ultrasound and magnetic resonance imaging, which do not require ionizing radiation, to make measurements analogous to those now being performed by using X-ray mammograms.

Interactive segmentation offers greater clinical potential in computational pathology compared to traditional automatic segmentation. By incorporating interactive input, it addresses the limitations of fully automatic segmentation models, which often fail to meet pathologists' requirements and rely heavily on large-scale, pixel-level annotated datasets. However, current interactive segmentation methods struggle to balance interaction cost and segmentation performance, and they fail to adapt effectively to slide-level segmentation, a task that is even more crucial in routine pathology analysis. In this study, we propose a Prototype-Guided Interactive Segmentation (ProGIS) framework for pathological image segmentation, designed to deliver precise segmentation results efficiently with minimal interaction signals. ProGIS identifies all same-type tissue connected components in a single interaction and supports multi-class segmentation without predefined categories during inference. Moreover, ProGIS can be easily adapted for slide-level interactive segmentation. Specifically, ProGIS consists of three modules: Prototype Initialization, Prototype Navigation, and Local Refinement. First, the Prototype Initialization module identifies categorical prototypes, which are then utilized in the Prototype Navigation module to identify all tissue connected components belonging to the same type. The local refinement module further refines the segmentation results using detailed correction signals to ensure the accuracy of challenging-to-distinguish regions. We evaluate our framework on two regions of interest level and two slide-level pathological segmentation datasets, achieving new state-of-the-art performance with fewer interactions than existing methods. Our code is available at https://github.com/JSGe-AI/ProGIS.

This paper presents a new edge-based interactive image segmentation method. First an edge image is generated with an edge detector, and then primary object edges can be selected by separating the object and background edges. Finally we obtain the complete object edge by connecting broken object edges. In order to reduce the influence of the noise, image denoising can be done before the edge detecting. Natural and medical image segmentation demonstrates that our method incorporates the advantages of the automatic and manual image segmentation methods. For most of images, a little of manual object edge connection is needed to obtain object edges. It is an effective and feasible interactive image segmentation method.

In this paper, a novel interactive medical image segmentation method called SMOPL is proposed. This method only needs marking some pixels on foreground region for segmentation. To do this, SMOPL characterize the inherent correlations among foreground and background pixels as Hilbert-Schmidt independence. By maximizing the independence and minimizing the smoothness of labels on instance neighbor graph simultaneously, SMOPL gets the sufficiently smooth confidences of both positive and negative classes in absence of negative training examples. Then a image segmentation can be obtained by assigning each pixel to the label for which the greatest confidence is calculated. Experiments on real-world medical images show that SMOPL is robust to get a high-quality segmentation with only positive label examples.

Interactive segmentation (IS) using minimal prompts like points and bounding boxes facilitates rapid image annotation, which is crucial for enhancing data-driven deep learning methods. Traditional IS methods, however, process only one target per interaction, leading to inefficiency when annotating multiple identical-class objects in remote sensing imagery (RSI). To address this issue, we present a new task—identical-class object detection (ICOD) for rapid IS in RSI. This task aims to only identify and detect all identical-class targets within an image, guided by a specific category target in the image with its mask. For this task, we propose an ICOD network (ICODet) with a two-stage object detection framework, which consists of a backbone, feature similarity analysis module (S3QFM), and an identical-class object detector. In particular, the S3QFM analyzes feature similarities from images and support objects at both feature-space and semantic levels, generating similarity maps. These maps are processed by a region proposal network (RPN) to extract target-level features, which are then refined through a simple feature comparison module and classified to precisely identify identical-class targets. To evaluate the effectiveness of this method, we construct two datasets for the ICOD task: one containing a diverse set of buildings and another containing multicategory RSI objects. Experimental results show that our method outperforms the compared methods on both datasets. This research introduces a new method for rapid IS of RSI and advances the development of fast interaction modes, offering significant practical value for data production and fundamental applications in the remote sensing community.

Interactive image segmentation based on samples reconstruction and FLDA