Effects of Different Superpixel Algorithms on Interactive Segmentations

Extensive research has been conducted in an effort to evaluate methods and techniques for image segmentation. However, while most literature has focused on evaluating automatic and semi-automatic algorithms, works evaluating interactive segmentation algorithms are less numerous. Note that interactive segmentation can improve results by adding prior knowledge from users into the process. Although this user guidance improves segmentation results, it also makes difficult to conduct objective evaluations. For this reason, some works only present non-canonical evaluations. In this paper, we present an objective and empirical evaluation of seed-based interactive segmentation algorithms. We first compare popular metrics that are employed in image-segmentation evaluations in order to define which one reflects most accurately the performance of segmentation algorithms. Then, in the aim of presenting reproducible results, we introduce a novel seed-based user input dataset that extends the well-known GrabCut dataset. In addition, we evaluate and contrast four state-of-the-art interactive segmentation algorithms. The analysis of the results demonstrates that Jaccard coefficient and Precision-Recall curves provide a good insight into the performance of the evaluated algorithms. Finally, the GrabCut algorithm presents the most robust and useful segmentation among all the evaluated algorithms.

This study investigates the impact of image complexity on the accuracy of interactive image segmentation algorithms. Image complexity plays a crucial role in segmentation performance, yet previous studies have primarily relied on subjective methods, leaving a gap in understanding how objective measures impact accuracy. The purpose of this research is to explore the relationship between image complexity and segmentation performance and to propose an adaptive approach for improving accuracy based on complexity measures. The study utilizes objective measures, namely entropy and fractal dimension, to quantify image complexity. An interactive image segmentation algorithm is employed, with a bounding box as the background and strokes as the foreground annotations. The number of strokes is dynamically adjusted based on complexity measures, ensuring a tailored segmentation approach. Comparative evaluations are conducted to assess the effectiveness of dynamic and fixed stroke allocation strategies. The principal results reveal a significant influence of image complexity on segmentation accuracy. The dynamic stroke allocation strategy outperforms fixed allocation, highlighting the importance of adapting to complexity. Moreover, the optimal combination of strokes and superpixel sizes is explored, providing valuable insights for practitioners. The findings emphasize the need to consider image complexity in segmentation algorithms to achieve accurate results. In conclusion, this study contributes to the understanding of the relationship between image complexity and interactive image segmentation. By introducing a dynamic stroke allocation approach and evaluating different configurations, the research provides insights into optimizing accuracy based on image complexity. The adaptive strategy improves segmentation performance and guides the development of robust algorithms. Future research can further refine the adaptive approach, explore additional complexity measures, and incorporate advanced machine learning techniques to enhance interactive image segmentation. Overall, this study advances the field by highlighting the importance of image complexity, providing guidance for practitioners, and paving the way for more efficient segmentation algorithms.

Nuclear forensics relies on different signatures to identify the source of nuclear material. Such signatures include crystalline structure, chemical composition, and particle morphology. One way to quantify morphology in electron microscope imagery is through image segmentation, where pixels are assigned to several partitions (or groups) that correspond to particles, grains, and other objects of interest within the image. Once pixels are assigned to segments, it is relatively straightforward to quantify other quantities of interest, such as grain size, circularity, etc. However, the range and diversity of microscope images make it difficult to obtain an accurate segmentation automatically. The accuracy of segmentation can be improved through supervised learning, but this requires many images to be manually segmented. Another way to improve the accuracy is to use interactive segmentation. Interactive segmentation requires a human to provide image-specific user input to improve performance. However, the amount of user input (effort) is generally far less than is required for supervised learning. In this paper, we investigate several parallelization strategies to automatically explore the user input parameter space of interactive segmentation algorithms across a large number of images. Specifically, we investigate four different parallelization mechanisms in a high-performance computing (HPC) environment and use the Amdahl fraction to evaluate efficiency on multiple processor cores across multiple nodes. Ultimately, the parallelization strategy that was most efficient utilized the message passing interface integrated with the segmentation and quantification code. This strategy had an Amdahl fraction of 0.985 and a performance of about 0.251 s/image. These results indicate that the parameter space of interactive segmentation algorithms can be efficiently explored using HPC. This opens the door to future work where user input is reduced and where interactive image segmentation algorithms are automatically applied to large image sets.

The image segmentation of histopathological tissue images has always been a challenge due to the overlapping of tissue color distributions, the complexity of extracellular texture and the large image size. In this paper, we introduce a new region-merging algorithm, namely, the Regional Pattern Merging (RPM) for interactive color image segmentation and annotation, by efficiently retrieving and applying the user’s prior knowledge of stroke-based interaction. Low-level color/texture features of each region are used to compose a regional pattern adapted to differentiating a foreground object from the background scene. This iterative region-merging is based on a modified Region Adjacency Graph (RAG) model built from initial segmented results of the mean shift to speed up the merging process. The foreground region of interest (ROI) is segmented by the reduction of the background region and discrimination of uncertain regions. We then compare our method against state-of-the-art interactive image segmentation algorithms in both natural images and histological images. Taking into account the homogeneity of both color and texture, the resulting semi-supervised classification and interactive segmentation capture histological structures more completely than other intensity or color-based methods. Experimental results show that the merging of the RAG model runs in a linear time according to the number of graph edges, which is essentially faster than both traditional graph-based and region-based methods.

A new algorithm for interactive image segmentation is proposed. Besides the traditional appearance and gradient information, a new Generic Shape Prior (GSP) knowledge which implies the location and the shape information of the object is combined into the framework. The GSP can be further categorized into the Regional and the Contour GSP to fit the interactive application, where a hierarchical graph-cut based optimization procedure is established, for its global optimization using the regional GSP to obtain good global segmentation results, and the local one using the Contour GSP to refine boundaries of global results. Moreover, the global optimization is based on superpixels which significantly reduce the computational complexity but preserve necessary image structures; the local one only considers a subset pixels around a contour segment, they both speed up the system. Results show our method performs better on both speed and accuracy.

This paper proposes an algorithm for interactive image segmentation. The task is formulated as a problem of graph-based transductive classification. Specifically, given an image window, the color of each pixel in it will be reconstructed linearly with those of the remaining pixels in this window. The optimal reconstruction weights will be kept unchanged to linearly reconstruct their class labels. The label reconstruction errors are estimated in each window. These errors are further collected together to develop a learning model. Then, the class information about the user specified foreground and background pixels are integrated into a regularization framework. Under this framework, a globally optimal labeling is finally obtained. The computational complexity is analyzed, and an approach for speeding up the algorithm is presented. Comparative experimental results illustrate the validity of our algorithm.

In this paper, we present a robust and accurate algorithm for interactive image segmentation. The level set method is clearly advantageous for image objects with a complex topology and fragmented appearance. Our method integrates discriminative classification models and distance transforms with the level set method to avoid local minima and better snap to true object boundaries. The level set function approximates a transformed version of pixelwise posterior probabilities of being part of a target object. The evolution of its zero level set is driven by three force terms, region force, edge field force, and curvature force. These forces are based on a probabilistic classifier and an unsigned distance transform of salient edges. We further propose a technique that improves the performance of both the probabilistic classifier and the level set method over multiple passes. It makes the final object segmentation less sensitive to user interactions. Experiments and comparisons demonstrate the effectiveness of our method.

Convolutional neural networks (CNNs) are the backbone of most modern interactive segmentation algorithms. However, the limited receptive field of CNNs restricts their ability to capture long-range semantic relationships. Recently, transformers have gained significant attention for their capacity to capture long-range dependencies. Nevertheless, CNNs still outperform Transformer in extracting local information. An effective interactive segmentation algorithm should accurately capture fine-grained local details alongside global semantic relationships. Therefore, we propose GLClick, a global–local click-based interactive image segmentation model that integrates local and global information through a novel fusion mechanism. We design an efficient global–local feature fusion module (GLFM) that integrates fine-grained features from various layers of ResNet50 with those from the Transformer feature pyramid. This approach maintains ResNet50’s ability to extract local features while effectively leveraging the Transformer to capture global context. Additionally, we enhance the multi-layer perceptron (MLP) to improve performance. Extensive experiments on diverse benchmark datasets demonstrate significant improvements in interactive image segmentation, confirming the effectiveness of our approach. Moreover, we conduct experiments on medical image datasets, further illustrating the model’s versatility and effectiveness across different domains.

Interactive image segmentation based on samples reconstruction and FLDA

In material science and bio-medical domains the quantity and quality of microscopy images is rapidly increasing and there is a great need to automatically detect, delineate and quantify particles, grains, cells, neurons and other functional objects within these images. These are challenging problems for image processing because of the variability in object appearance that inevitably arises in real world image acquisition and analysis. One of the most promising (and practical) ways to address these challenges is interactive image segmentation. These algorithms are designed to incorporate input from a human operator to tailor the segmentation method to the image at hand. Interactive image segmentation is now a key tool in a wide range of applications in microscopy and elsewhere. Historically, interactive image segmentation algorithms have tailored segmentation on an image-by-image basis, and information derived from operator input is not transferred between images. But recently there has been increasing interest to use machine learning in segmentation to provide interactive tools that accumulate and learn from the operator input over longer periods of time. These new learning algorithms reduce the need for operator input over time, and can potentially provide a more dynamic balance between customization and automation for different applications. This paper reviews the state of the art in this area, provides a unified view of these algorithms, and compares the segmentation performance of various design choices.

Interactive Image Segmentation has attracted much attention in the vision and graphics community recently. A typical application for interactive image segmentation is foreground/background segmentation based on user specified brush labellings. The problem can be formulated within the binary Markov Random Field (MRF) framework which can be solved efficiently via graph cut [1]. However, no attempt has yet been made to handle segmentation of multiple regions using graph cuts. In this paper, we propose a multiclass interactive image segmentation algorithm based on the Potts MRF model. Following [2], this can be converted to a multiway cut problem first proposed in [2] and solved by expansion-move algorithms for approximate inference [2]. A faster algorithm is proposed in this paper for efficient solution of the multiway cut problem based on partial optimal labeling. To achieve this, we combine the one-vs-all classifier fusion framework with the expansion-move algorithm for label inference over large images. We justify our approach with both theoretical analysis and experimental validation.

In this paper we propose an interactive approach for object class segmentation of natural images on touch-screen capable mobile devices. The key research question to which this paper tries to give an answer is: can we effectively correct the errors committed by an automatic or semi-automatic figure-ground segmentation algorithm while also providing real time feedback to the user on a low computational power mobile device? Many research works focused on improving automatic or semi-automatic figure-ground segmentation algorithms, but none tried to take advantage of the existing touch-screen technology integrated in most modern mobile devices to optimize the segmentation results of these algorithms. Our key idea is to use super-pixels as interactive buttons that can be quickly tapped by the user to be added or removed from an initial low quality segmentation mask, with the aim of correcting the segmentation errors and produce a satisfying final result. We performed an extensive analysis of the proposed approach by implementing it both on a desktop computer and a mid-range Android device, even though our method is extremely simple, the results we obtained are comparable with those achieved by other state-of-the-art interactive segmentation algorithms. As such, we believe that the proposed approach can be exploited by most image editing mobile applications to provide a simple but highly effective method for interactive object class segmentation.

Introduction: Wound segmentation is a pivotal process of photography, which is an important and prevalent method for skin wound analysis and documentation in forensic science. However, due to the complexity and diversity of forensic wound images, no automatic segmentation algorithm is capable of judging the boundaries of the wound accurately. Meanwhile, the currently approach using lasso tool, though accurate, is time-consuming and labor-intensive for practical application. Materials and Methods: In this paper, interactive segmentation algorithms are applied to skin wound extraction. This application improves the performance of traditional segmentation methods and reduces the time of manual method. In order to verify the segmentation, a software platform hosting image preprocessing, image segmentation and wound area calculation is developed. This platform integrates the manual method and three interactive segmentation methods, and is completely available for forensic wound measurement based on photography technique. Results: The segmentation accuracy is evaluated by precision, recall and F-measure, which are higher than 0.96, 0.95, and 0.96 respectively. The error of the area calculation is less than 5%, with respect to the golden standard. Discussion: These results demonstrate that interactive segmentation is practicable and precise enough to meet the demands of skin wounds measurement based on photography in forensic medicine.KeywordsInteractive segmentationWound measurementForensic medicinePhotography

This paper proposes a novel interactive image segmentation algorithm based on the Grow Cut of two different scale graphs. Firstly, Watershed algorithm based on color information has been used to partition the image into many different regions which will be considered as the cells of Grow Cut, instead of image pixels. Then a segmentation result can be obtained by using Grow Cut on the aforementioned regions. Finally an automatic edge correction can be used on the segmentation result by Grow Cut of pixel-scale graph. Because the number of nodes and edges for the Grow Cut algorithm is reduced by more than fifty times compared to the pixel based method, the running time of our proposed algorithm is much less than the original Grow Cut. The segmentation performance of our proposed is much better than the original Grow Cut. Experimental results on Berkeley image dataset demonstrated the effectiveness of proposed method.

Interactive segmentation methods utilize user-provided positive and negative clicks to guide the model in accurately segmenting target objects. Compared to fully automatic medical image segmentation, these methods can achieve higher segmentation accuracy with limited image data, demonstrating significant potential in clinical applications. Typically, for each new click provided by the user, conventional interactive segmentation methods reprocess the entire network by re-inputting the click into the segmentation model, which greatly increases the user’s interaction burden and deviates from the intended goal of interactive segmentation tasks. To address this issue, we propose an efficient segmentation network, ESM-Net, for interactive medical image segmentation. It obtains high-quality segmentation masks based on the user’s initial clicks, reducing the complexity of subsequent refinement steps. Recent studies have demonstrated the strong performance of the Mamba model in various vision tasks; however, its application in interactive segmentation remains unexplored. In our study, we incorporate the Mamba module into our framework for the first time and enhance its spatial representation capabilities by developing a Spatial Augmented Convolution (SAC) module. These components are combined as the fundamental building blocks of our network. Furthermore, we designed a novel and efficient segmentation head to fuse multi-scale features extracted from the encoder, optimizing the generation of the predicted segmentation masks. Through comprehensive experiments, our method achieved state-of-the-art performance on three medical image datasets. Specifically, we achieved 1.43 NoC@90 on the Kvasir-SEG dataset, 1.57 NoC@90 on the CVC-ClinicDB polyp segmentation dataset, and 1.03 NoC@90 on the ADAM retinal disk segmentation dataset. The assessments on these three medical image datasets highlight the effectiveness of our approach in interactive medical image segmentation.