E-GrabCut: an economic method of iterative video object extraction

In this paper, a novel object-oriented hierarchical video segmentation and representation algorithm is proposed based on a four-component video model, where the local variance contrast and the frame difference contrast are selected for generating the 2D spatiotemporal entropy. The extracted object is first represented by a group of (4/spl times/4) blocks coarsely, then the intra-block edge extraction on edge blocks and the joint spatiotemporal similarity test among neighboring blocks are further performed for determining meaningful real objects. This proposed hierarchical segmentation algorithm may be very useful for MPEG-4 applications. A novel fast algorithm is also introduced for reducing the search burden. Moreover, this unsupervised algorithm also makes automatic image and video segmentation possible.

A new video segmentation algorithm using a combined seg- mentation measure for content-based coding is presented. The com- bined segmentation measure is a weighted sum of intensity, motion, and a change segmentation measure. The change segmentation measure is defined from a change image, which is obtained from the proposed change detector. The proposed change detector performs morphological erosion filtering to eliminate many inaccurate components, included in the resulting image obtained from a conventional change detector. Since the change segmentation measure is defined as an absolute change value difference between a pixel and its neighboring region, it can be an efficient segmentation measure for the accurate segmentation of neigh- boring moving objects and static background regions. Therefore, the pro- posed combined segmentation measure can determine exact bound- aries even though the estimated motion vectors around the boundaries of moving objects and static background regions are inaccurate and the intensities around the boundaries are similar. From the experimental re- sults for a video segmentation, since the combined segmentation mea- sure can accurately segment the boundaries between moving objects and static background regions, the motion compensated images pro- duced by the proposed segmentation algorithm show improved image quality compared to the conventional segmentation. © 2000 Society of Photo-Optical Instrumentation Engineers. (S0091-3286(00)01208-3)

Video segmentation can distinguish the video content preliminarily. It gets ready for the following Analysis scene and object recognition. The accurate video segmentation would do the direct impact on successor task. When the video segmentation algorithm is accurate, the video in painting effect based on motion mask is well. Therefore, more and more people pay close attention to the improvement about segmentation algorithm. In order to improve the difficulty of background modeling and inaccurate update in traditional background difference method. This paper raises a method which combine the SURF algorithm with background subtraction.

Semi-automated segmentation or more commonly known as interactive image segmentation is an algorithm that extracts a region of interest (ROI) from an image based on the input information from the user. The said algorithm will be repetitively fed with such input information until required region of interest is successfully segmented. To accelerate this segmentation procedure as well as enhancing the result, pre-processing steps can be applied. The application of superpixel is an example of such pre-processing step. Superpixel can be defined as a collection of pixels that share common features such as texture and colours. Though employed as pre-processing step in many interactive segmentation algorithms, to date, no study has been conducted to assess the effects of such incorporations on the segmentation algorithms. Thus, this study aims to address this issue. In this study, five different types of superpixels ranging from watershed, density, graph, clustering and energy optimization categories are evaluated. The superpixels generated by these five algorithms will be used on two interactive image segmentation algorithms: i) Maximal Similarity based Region Merging (MSRM) and ii) Graph-Based Manifold Ranking (GBMR) with single and multiple strokes on various images from the Berkeley image dataset. The result of testing had shown that MSRM achieved better result compared to GBMR in both single and multiple input strokes using SEEDS superpixel algorithm. This study summary concluded that at different superpixel algorithms produced different results and that it is not possible to single out one particular superpixel algorithm that can work well for all the interactive segmentation algorithms. As such, the key to achieving a decent segmentation result lies in choosing the right superpixel algorithms for a given interactive segmentation algorithm.

The November 1999 Journal published in: “Adding Hyperlinks to Digital Television” by V. M. Bove, Jr., J. Dakss, S. Agamanolis, and E. Chalom: “Hyperlinked video is where objects are made selectable by some form of user interface, and the user's interactions with these objects modify the presentation of the video. Identifying and tracking the objects remains one of the chief difficulties in authoring hyperlinked video; we solve this problem through the use of a video tracking and segmentation algorithm that uses color, texture, motion, and position parameters. An author uses a computer mouse to scribble roughly on each desired object in a frame of video, and the system generates a segmentation mask for that frame and following frames. This technique was applied in the production of a soap opera program, with the result that the user can inquire about purchasing clothing and furnishings used in the show.”

Identifying different audio segments in videos is the first step for many important tasks such as event detection and speech transcription. Approaches using Mel-Frequency Cepstral coefficients (MFCCs) with Gaussian mixture models (GMMs) and hidden Markov models (HMMs) perform reasonably well in stationary conditions but do not scale to a broad range of environmental conditions. This paper focuses on the audio segmentation in broadcast soccer videos into audio classes such as Silence, Speech Only, Speech Over Crowd, Crowd Only and Excited, with an alternative feature set which is simplistic as well as robust to changes in the environment conditions. Support Vector Machines (SVMs), Neural Networks and Random Forest are used for the classification. The accuracy achieved with SVMs, Neural Networks and Random Forest are 83.80%, 86.07%, and 88.35% respectively. The proposed features and Random Forest classifier are found to achieve better accuracy compared to the other classifiers.

Extensive research has been conducted in an effort to evaluate methods and techniques for image segmentation. However, while most literature has focused on evaluating automatic and semi-automatic algorithms, works evaluating interactive segmentation algorithms are less numerous. Note that interactive segmentation can improve results by adding prior knowledge from users into the process. Although this user guidance improves segmentation results, it also makes difficult to conduct objective evaluations. For this reason, some works only present non-canonical evaluations. In this paper, we present an objective and empirical evaluation of seed-based interactive segmentation algorithms. We first compare popular metrics that are employed in image-segmentation evaluations in order to define which one reflects most accurately the performance of segmentation algorithms. Then, in the aim of presenting reproducible results, we introduce a novel seed-based user input dataset that extends the well-known GrabCut dataset. In addition, we evaluate and contrast four state-of-the-art interactive segmentation algorithms. The analysis of the results demonstrates that Jaccard coefficient and Precision-Recall curves provide a good insight into the performance of the evaluated algorithms. Finally, the GrabCut algorithm presents the most robust and useful segmentation among all the evaluated algorithms.

The paper proposed an approach to extracting human body from an image without interaction of user. Image segmentation algorithm was an Interactive segmentation algorithm, which was employed to extract the foreground of an image, but it works wrong frequently when the pixels around the foreground are similar to the pixels of the background. We advanced a new cost function to correct the mistake. Using the human detection method, we proposed a human body extraction method from image without interaction. Through experiments, our segmentation algorithm can achieve better segmentation results in two situations. The experiment shows that the size of user input box has less influence on this proposed algorithm.KeywordsHuman action recognitionSport analysisHuman detectionImage segmentation algorithm

This paper presents a novel algorithm for unsupervised texture segmentation. The proposed algorithm incorporates the local binary pattern operator under a segmentation framework based on the active contour without edges model. The experiments performed, show that it can be used for fast segmentation of two-textured images, outperforming recent texture segmentation algorithms, with a segmentation quality that reaches 99% on average

Video segmentation research has emerged over the last decade for biomedical image and video processing, especially in biological organism tracking. However, due to the difficulties in generating the video segmentation ground truth, the general lack of segmentation datasets with annotated ground-truth severely limits the evaluation of segmentation algorithms. This paper proposes an efficient and scalable crowdsourced approach to generate video segmentation ground-truth to facilitate database generation for general biological organism segmentation and tracking algorithm evaluation. To illustrate the proposed approach, an annotated zebrafish larvae video segmentation dataset has been generated and made freely available online. To enable the evaluation of algorithms against a ground-truth, a set of segmentation evaluation metrics are also presented. The segmentation performance of five leading segmentation algorithms is then evaluated by the metrics on the generated zebrafish video segmentation dataset.

There has been remarkable progress in the field of Semantic segmentation in recent years. Yet, it remains a challenging problem to apply segmentation to the video-based applications. Videos usually involve significantly larger volume of data compared to images. Particularly, a video contains around 30 frames per second. Segmentation of the similar frames unnecessarily adds to the time required for segmentation of complete video. In this paper, we propose a contour detection-based approach for detection of salient frames for faster semantic segmentation of videos. We propose to detect the salient frames of the video and pass only the salient frames through the segmentation block. Then, the segmented labels of the salient frames are mapped to the non-salient frames. The salient frame is defined by the variation in the pixel values of the background subtracted frames. The background subtraction is done using MOG2 background subtractor algorithm for background subtraction in various lighting conditions. We demonstrate the results using the Pytorch model for semantic segmentation of images. We propose to concatenate the semantic segmentation model to our proposed framework. We evaluate our result by comparing the time taken and the mean Intersection over Union (mIoU) for segmentation of the video with and without passing the video input through our proposed framework. We evaluate the results of Saliency Detection Block using Retention and Condensation ratio as the quality metrics.

Based on compressed video segment detection, the decompression step can be omitted, features can be directly extracted from the original video data stream, and the detection rate can be accelerated. This paper analyzes the characteristics of video data, video segmentation and key frame selection. Based on the analysis of the key technologies of video retrieval technology, the compressed video stream extraction method based on video segmentation and key frame is introduced. A method of extracting key frames from MPEG compressed video based on DC coefficients and motion vectors is proposed. Experiments show that the proposed method can reduce the computational burden, and can better represent the video content.

The purpose of this study is to reveal how the performance of lung nodule segmentation algorithm impacts the performance of lung nodule detection, and to provide guidelines for choosing an appropriate segmentation algorithm with appropriate parameters in a computer-aided detection (CAD) scheme. The database consisted of 85 CT scans with 111 nodules of 3 mm or larger in diameter from the standard CT lung nodule database created by the Lung Image Database Consortium. The initial nodule candidates were identified as those with strong response to a selective nodule enhancement filter. A uniform viewpoint reformation technique was applied to a three-dimensional nodule candidate to generate 24 two-dimensional (2D) reformatted images, which would be used to effectively distinguish between true nodules and false positives. Six different algorithms were employed to segment the initial nodule candidates in the 2D reformatted images. Finally, 2D features from the segmented areas in the 24 reformatted images were determined, selected, and classified for removal of false positives. Therefore, there were six similar CAD schemes, in which only the segmentation algorithms were different. The six segmentation algorithms included the fixed thresholding (FT), Otsu thresholding (OTSU), fuzzy C-means (FCM), Gaussian mixture model (GMM), Chan and Vese model (CV), and local binary fitting (LBF). The mean Jaccard index and the mean absolute distance (Dmean) were employed to evaluate the performance of segmentation algorithms, and the number of false positives at a fixed sensitivity was employed to evaluate the performance of the CAD schemes. For the segmentation algorithms of FT, OTSU, FCM, GMM, CV, and LBF, the highest mean Jaccard index between the segmented nodule and the ground truth were 0.601, 0.586, 0.588, 0.563, 0.543, and 0.553, respectively, and the corresponding Dmean were 1.74, 1.80, 2.32, 2.80, 3.48, and 3.18 pixels, respectively. With these segmentation results of the six segmentation algorithms, the six CAD schemes reported 4.4, 8.8, 3.4, 9.2, 13.6, and 10.4 false positives per CT scan at a sensitivity of 80%. When multiple algorithms are available for segmenting nodule candidates in a CAD scheme, the "optimal" segmentation algorithm did not necessarily lead to the "optimal" CAD detection performance.

In this paper, an unsupervised image segmentation algorithm is proposed, which combines spatial constraints with a kernel fuzzy c-means (KFCM) clustering algorithm. Conventional KFCM clustering segmentation algorithm does not incorporate the spatial context information of image, which makes it sensitive to the noise and intensity variations. In order to overcome the shortcomings, the contents of image is characterized by Gaussian mixture model, and the parameters of model are estimated by modified expectation maximization (EM) algorithm, which overcomes the classical EM algorithm drawbacks that easily trap in local maxima and be susceptible to initial value. According to the maximum a posterior theorem, we can get the pixel maximum posterior probability. We redefine the objective function of the KFCM algorithm which incorporates the pixel maximum posteriori probability, by minimizing the fuzzy objective function, the fuzzy segmentation algorithm is derived. The experimental results on a synthetic image and a real magnetic resonance image show that the proposed algorithm is more effective than the conventional FCM algorithm without local spatial constraints.

Lack of temporal coherence in video segmentation algorithms often leads to flickering or to discrepancies on the segmentation mask boundaries. Active contour video segmentation algorithms can lead to very smooth segmentation masks when they are defined in three dimensions (two spatial and one temporal). In this paper, we will use an active contour method to attract an active surface toward a non-smoothed segmentation mask boundary. This active surface will produce a new segmentation mask which will be smoother than the first one. To achieve this task, we introduce a new region-based term for active contour segmentation in the variational framework. This term attracts the evolving curve to a reference contour. The energy criterion and the evolution equation are defined in n-dimension and we investigate the particular case of two regions. The use of active surfaces to smooth video segmentation masks appears to be a very powerful tool for video postprocessing. The resulting masks are smoother than the original ones, the discrepancies of the segmentation masks are removed and the flickering on the boundary of the segmentation masks are considerably reduced.