Object segmentation based on Gaussian mixture model and conditional random fields

In this work, we use Hidden Markov Models (HMM), Conditional Random Field (CRF), Gaussian Mixture Models (GMM) and Mathematical Methods of Statistics (MMS) for Chinese and Japanese text summarization. The purpose of this work is to study the applicability of mentioned three trainable models for cross-language text summarization. For model training, we use several training features such as sentence position, sentence centrality, number of Name Entity and so on. For model testing, Chinese on-line news and Japanese news are used as test data which are extracted from web pages. We evaluate each model by measuring the precision at the compression rate 10%, 20% and 30%. MMS is a baseline method. The results show that HMM, CRF and GMM have remarkable increases than MMS on both Chinese and Japanese text summarization by using the same training features. Especially, GMM model make a best performance in all tests.

A vision system is an integral component of many autonomous robots. It enables the robot to perform essential tasks such as mapping, localization, or path planning. A vision system also assists with guiding the robot's grasping and manipulation tasks. As an increased demand is placed on service robots to operate in uncontrolled environments, advanced vision systems must be created that can function effectively in visually complex and cluttered settings. This thesis presents the development of segmentation algorithms to assist in online model acquisition for guiding robotic manipulation tasks. Specifically, the focus is placed on localizing door handles to assist in robotic door opening, and on acquiring partial object models to guide robotic grasping. . First, a method for localizing a door handle of unknown geometry based on a proposed 3D segmentation method is presented. Following segmentation, localization is performed by fitting a simple box model to the segmented handle. The proposed method functions without requiring assumptions about the appearance of the handle or the door, and without a geometric model of the handle. Next, an object segmentation algorithm is developed, which combines multiple appearance (intensity and texture) and geometric (depth and curvature) cues. The algorithm is able to segment objects without utilizing any a priori appearance or geometric information in visually complex and cluttered environments. The segmentation method is based on the Conditional Random Fields (CRF) framework, and the graph cuts energy minimization technique. A simple and efficient method for initializing the proposed algorithm which overcomes graph cuts' reliance on user interaction is also developed. Finally, an improved segmentation algorithm is developed which incorporates a distance metric learning (DML) step as a means of weighing various appearance and geometric segmentation cues, allowing the method to better adapt to the available data. The improved method also models the distribution of 3D points in space as a distribution of algebraic distances from an ellipsoid fitted to the object, improving the method's ability to predict which points are likely to belong to the object or the background. Experimental validation of all methods is performed. Each method is evaluated in a realistic setting, utilizing scenarios of various complexities. Experimental results have demonstrated the effectiveness of the handle localization method, and the object segmentation methods.

In crop disease image segmentation, traditional convolutional neural networks have the problem of low image accuracy. For this reason, this paper proposes a segmentation method that combines conditional random fields with segnet networks. First, input the training set in the data set into the segnet network for training, and obtain the initial segmentation result and the network with updated parameters; secondly, the image segmented by the segnet network is sent to the conditional random field in the form of pixels and classification vectors corresponding to the pixels At the same time, construct an energy function to represent the relative relationship between pixels, and then keep training; in order to get the conditional random field in the training set, you need to adjust and optimize the image segmented by the segnet network to get it; then in the verification set, the segnet network and the weight parameters of the conditional random field model are further adjusted and optimized to obtain the final conditional random field and segnet network; finally, the final conditional random field and segnet network are applied to the crop disease image segmentation. The test results show that compared with the Otsu threshold and Grabcut segmentation methods, the segmentation accuracy P, recall rate R, and comprehensive evaluation index F of this method are all the maximum, and the average processing time of a single image is 0.86 s; compared with pspnet and deeplab methods, Under the premise that the segmentation accuracy P, the recall rate R, and the comprehensive evaluation index F are basically unchanged, the average processing time of a single image is significantly reduced. The results show that the best comprehensive effect can be achieved by segmenting crop disease images in complex environments using the method in this paper.

Accurately predicting water consumption in residential and commercial buildings is essential for identifying possible leaks, minimizing water wastage, and for paving the way for a sustainable future. In this paper, we present SWaP, a Smart Water Prediction system that predicts future hourly water consumption based on historical data. To perform this prediction task, in SWaP, we design discriminative probabilistic graphical and deep learning models, in particular, sparse Gaussian Conditional Random Fields (GCRFs) and Long Short Term Memory (LSTM) based deep Recurrent Neural Network (RNN) models, to successfully encode dependencies in the water consumption data. We evaluate our system on water consumption data collected from multiple buildings in a university campus and demonstrate that both the GCRF and LSTM based deep models are able to accurately predict future hourly water consumption in advance using just the last 24 hours of data at test time. SWaP achieves superior prediction performance for all buildings in comparison to the linear regression and ARIMA baselines in terms of Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE), with the GCRF and LSTM models providing 50% and 44% improvements on average, respectively. We also demonstrate that augmenting our models with temporal features such as time of the day and day of the week can improve the overall average prediction performance. Additionally, based on our evaluation, we observe that the GCRF model outperforms the LSTM based deep learning model, while simultaneously being faster to train and execute at test time. The computationally efficient and interpretable nature of GCRF models in SWaP make them an ideal choice for practical deployment.

The conventional object segmentation methods often degrade their performance due to the requirement of user interaction and/or the incomplete colour appearance models. In this Letter, the authors propose a novel design method of a colour appearance model for accurate and fully‐automatic object segmentation by using saliency maps. The authors initialise the Gaussian mixture models (GMMs) to describe the colour appearance of the foreground objects and the background, respectively, where the mean vectors, covariance matrices, and mixing coefficients are updated adaptively such that more salient pixels have larger weights to update the GMM for the foreground objects while less salient pixels have larger weights to update the GMM for the background, respectively. Experiments are performed on MSRC, iCoseg, and PASCAL datasets and we show that the proposed method outperforms the existing methods quantitatively and qualitatively.

The field of computer vision has undergone rapid development. Starting from recognition tasks such as classification, detection, and segmentation, the focus of visual analysis has gradually shifted towards learning spatiotemporal information. This thesis presents research on spatiotemporal learning, with a particular emphasis on motion estimation and visual recognition. First, we address the problem of video object tracking. Previous methods have primarily re-lied on learning improved appearance representations, while the spatiotemporal relationships of individual objects have been underexplored. We propose leveraging optical flow features to achieve higher generalization in semi-supervised video object segmentation, directly incorporating these features into both the target representation and the decoder network. Our experiments and analysis show that enriching feature representations with spatiotemporal information improves segmentation quality and generalization capability. Next, we investigate spatiotemporal learning in 3D for motion estimation, specifically scene flow estimation. Scene flow estimation as an important research topic in 3D computer vision is crucial for applications such as robotics, autonomous driving, embodied navigation, and tracking. We investigate the problem in different perspectives: 1. What is the best formulation for solving the problem and how to learn a better spatiotemporal feature representation? 2. Can we introduce uncertainty estimation to the task, which is of crucial importance for safety-critical downstream tasks? 3. How to scale the estimation to large-scale data, e.g., autonomous scenes, and leverage the temporal information without introducing much computation overheads? To answer these questions, we explore the use of transformers for improved feature representation, diffusion models for uncertainty estimation, and efficient feature learning methods for multi-frame, large-scale autonomous driving scenarios. Finally, we extend our research to joint visual segmentation, tracking, and open-vocabulary recognition in LiDAR sequences, particularly for autonomous scenes. In such environments, precise segmentation, tracking, and recognition of objects are essential for downstream analysis and control. Current human-annotated open-source datasets allow for reasonable tracking of traffic participants such as cars and pedestrians. However, we aim to advance beyond this towards segmenting and tracking any object in LiDAR data. To this end, we propose a pseudo-labeling engine that leverages the 2D visual foundation model SAM v2 and the vision-language model CLIP to automatically label LiDAR streams. We further introduce the SAL-4D model, capable of segmenting, tracking, and recognizing any object in a zero-shot manner. In summary, we explore the learning of spatiotemporal information in both 2D image and 3D point cloud domains. In the image domain, we demonstrate that spatiotemporal information improves video object segmentation quality and generalization. In the 3D point cloud domain, we show that spatiotemporal learning enables more accurate motion estimation and facilitates the first method for zero-shot segmentation, tracking, and open-vocabulary recognition of arbitrary objects.

Automatic segmentation of corpus collasum using Gaussian mixture modeling and Fuzzy C means methods

In this paper, we propose an effective source-aware domain enhancement and adaptation (SDEA) approach to increase the accuracy of the existing convolutional neural network-based (CNN-based) object segmentation methods. We first scoop out the source elements, such as the falling-leaves, manhole covers, cirrus clouds, and advertisements, which often cause invalid object segmentation and make the existing object segmentation methods provide unreliable information to the ADAS (automatic driving assistance systems) applications. Secondly, we create a new GTA5-like (Grand Theft Auto V-like) dataset with the scenarios including these source elements. Furthermore, we perform a domain adaptation on the created GTA5-like dataset to generate a photo-realistic GTA5-like dataset, namely GTA5 s SDEA . Without the need to relabel the pixel-annotations for GTA5 s SDEA , we combine GTA5 s SDEA with the realistic dataset, namely Camvid, to constitute a newly enhanced dataset. After retraining the existing CNN-based object segmentation methods by using our enhanced dataset, it can achieve substantial segmentation accuracy improvement. The comprehensive experimental results have demonstrated the clear accuracy improvement merit by applying our SDEA approach to the state-of-the-art object segmentation methods on FCN (Fully Convolutional Networks), SegNet-basic, AdaptSegNet, and Gated-AdaptSegNet, providing more reliable information to ADAS applications.

Image segmentation has always been an important research direction in the field of images processing, however, due to the long cycle of algorithm, the image segmentation techniques have never been widely applied. According to the problem above, a image segmentation algorithm of Gaussian Mixture Model (GMM) based on Map/Reduce is proposed to improve the real-time performance. Firstly, the architecture of Hadoop image processing interface (Hipi) is presented to improve the ability of Map/Reduce to process small image. The K-means is present to generate the init value of Gaussian Mixture Model to avoid local optimal. Secondly, the cloud computational model of Map/Reduce is proposed to realize the parallel of GMM, and the fusion method of the sub-model for GMM is presented to keep the accuracy of segmentation. Finally, the experiment is carried out based on the cloud platform of aliyun. Results show the image segmentation algorithm of GMM based on Map/Reduce can greatly improve the real-time performance with keeping the accuracy of segmentation.

Current speech recognition systems, for example, typically use Gaussian mixture models (GMMs), to estimate the observation (or emission) probabilities of hidden Markov models (HMMs), and GMMs are generative models that have only one layer of latent variables. Instead of developing more powerful models, most of the research effort has gone into finding better ways of estimating the GMM parameters so that error rates are decreased or the margin between different classes is increased. The same observation holds for natural language processing (NLP) in which maximum entropy (MaxEnt) models and conditional random fields (CRFs) have been popular for the last decade. Both of these approaches use shallow models whose success largely depends on the use of carefully handcrafted features.

Interactive image segmentation has gained much popularity since it can enhance the segmentation performance with human interactions. In this paper, we propose a novel framework that can effectively estimate the binary labels based on Conditional Random Field (CRF) models combining region-level information with pixels-level information. The user scribbles are treated as soft constraints. We use pixel-layer with local CRF and region-layer with fully connected CRF to model the graph. Then we combine the pixel information and region information together to calculate the pixels' likelihood term. In contrast to using relations between the pixel-layer and the region-layer during the probability computation, we calculate the probabilities of pixels and regions separately to reduce computational complexity. After the likelihood computation, a graph cut based model is used to refine the pixel-level label predictions of the image in order to avoid label inconsistency in regions and capture the boundary details. Due to the less relationship between layers and utilizing the refinement, the proposed model can provide high accuracy and user inputs insensitive results. Our experiments show that the proposed approach can efficiently and accurately segment the images compared with other state of art methods with only a few scribbles. Graph cut based fine tuning improves our segmentation accuracy and our method needs less computational cost than traditional multi-layer-based methods.

Accurate segmentation of zebrafish from bright-field microscope images is crucial to many applications in the life sciences. Early zebrafish stages are used, and in these stages the zebrafish is partially transparent. This transparency leads to edge ambiguity as is typically seen in the larval stages. Therefore, segmentation of zebrafish objects from images is a challenging task in computational bio-imaging. Popular computational methods fail to segment the relevant edges, which subsequently results in inaccurate measurements and evaluations. Here we present a hybrid method to accomplish accurate and efficient segmentation of zebrafish specimens from bright-field microscope images. We employ the mean shift algorithm to augment the colour representation in the images. This improves the discrimination of the specimen to the background and provides a segmentation candidate retaining the overall shape of the zebrafish. A distance-regularised level set function is initialised from this segmentation candidate and fed to an improved level set method, such that we can obtain another segmentation candidate which preserves the explicit contour of the object. The two candidates are fused using heuristics, and the hybrid result is refined to represent the contour of the zebrafish specimen. We have applied the proposed method on two typical datasets. From experiments, we conclude that the proposed hybrid method improves both efficiency and accuracy of the segmentation of the zebrafish specimen. The results are going to be used for high-throughput applications with zebrafish.

Learning feature representations from unlabeled data for volcano-seismic event classification

Structured prediction of sparse dependent variables for traffic state estimation in large-scale networks

Road detection is a crucial problem for autonomous navigation system (ANS) and advance driver-assistance system (ADAS). In this paper, we propose a hierarchical road detection method for robust road detection in challenging scenarios. Given an on-board road image, we first train a Gaussian mixture model (GMM) to obtain road probability density map (RPDM), and next oversegment the image into superpixels. Based on RPDM and superpixels, initial seeds are selected in an unsupervised way, and the seed superpixels iteratively try to occupy their neighbors according to GrowCut framework, the road segment is obtained after convergency. Finally, we refine the road segment with a conditional random field (CRF), which enforces the shape prior on the road segmentation task. Experiments on two challenging databases demonstrate that the proposed method exhibits high robustness compared with the state-of-the-art.