Combining Color and Spatial Image Features for Unsupervised Image Segmentation with Mixture Modelling and Spectral Clustering

Segmentation is a low-level processing aimed at the partition of an image in disjoint regions, each one homogeneous with respect to some properties like intensity, texture, shape, etc. Such a task is needed in many high-level processing and applications in such diverse fields as remote-sensing, medical imaging, video coding, and industrial automation, just to name a few. In this work of thesis, we first deal with the problem of color based image segmentation, that is based on some form of homogeneity in the color/spectral properties of the image, and then move to the more complex task of texture based segmentation, where the aim is to recognize complex structures in the image which are typically non homogeneous in terms of spectral properties. Concerning color based segmentation, we focus on a specific class of Markov Random Field (MRF) probability models, namely the Tree-Structured Markov Random Fields (TS-MRF), recently proposed to provide a hierarchical multiscale description of images. Based on such a model, the unsupervised image segmentation is carried out by means of a sequence of nested class splits, where each class is modeled as a local binary MRF. As a first contribution, we first extend the model to generic tree structures removing the unnecessary binary constraint, and then devise a new TS-MRF unsupervised segmentation technique which improves upon the original algorithm by selecting a better tree structure and eliminating spurious classes. Such results are obtained by using the Mean-Shift procedure to estimate the number of pdf modes at each node, and to obtain a more reliable initial clustering for subsequent MRF optimization. To this end, we devise a new reliable and fast clustering algorithm based on the Mean-Shift technique, which makes use of a variable-bandwidth strategy based on the k-Nearest Neighbors (k-NN) technique, and is implemented with a speed-up strategy that cuts significantly the computational complexity. Experimental results on synthetic data and real life images from the remote sensing domain prove the potential of the proposed method. Turning to texture based segmentation, we present a novel multiscale texture model, and a related algorithm for the unsupervised segmentation of color images. Elementary textures are characterized by their spatial interactions with neighboring regions along selected directions. Such interactions are modeled in turn by means of a set of Markov chains, one for each direction, whose parameters are collected in a feature vector that synthetically describes the texture. Based on the feature vectors, the texture are then recursively merged, giving rise to larger and more complex textures, which appear at different scales of observation: accordingly, the model is named Hierarchical Multiple Markov Chain (H-MMC). The Texture Fragmentation and Reconstruction (TFR) algorithm, addresses the unsupervised segmentation problem based on the H-MMC model. The ``fragmentation'' step allows one to find the elementary textures of the model, while the ``reconstruction'' step defines the hierarchical image segmentation based on a probabilistic measure (texture score) which takes into account both region scale and inter-region interactions. The performance of the proposed method was assessed through the Prague segmentation benchmark, based on mosaics of real natural textures, and also tested on real-world natural images from the Berkeley dataset. Finally, an application of the TFR algorithm for the unsupervised segmentation of multiresolution remotely sensed images is proposed, based on the significant presence of textural content in last generation high-resolution satellite images. The obtained results further assess the potential of the proposed method for such real life applications.

Deep expectation-maximization network for unsupervised image segmentation and clustering

In this paper, an unsupervised segmentation method using clustering is presented for color images. We propose to use a neural network based approach to automatic feature selection to achieve adaptive segmentation of color images. With a self-organizing feature map (SOFM), multiple color features can be analyzed, and the useful feature sequence (feature vector) can then be determined. The encoded feature vector is used in the final segmentation using fuzzy clustering. The proposed method has been applied in segmenting different types of color images, and the experimental results show that it outperforms the classical clustering method. Our study shows that the feature encoding approach offers great promise in automating and optimizing the segmentation of color images.

Conventional unsupervised image segmentation methods use color and geometric information and apply clustering algorithms over pixels. They preserve object boundaries well but often suffer from over-segmentation due to noise and artifacts in the images. In this paper, we contribute on a preprocessing step for image smoothing, which alleviates the burden of conventional unsupervised image segmentation and enhance their performance. Our approach relies on a convolutional autoencoder (CAE) with the total variation loss (TVL) for unsupervised learning. We show that, after our CAE-TVL preprocessing step, the over-segmentation effect is significantly reduced using the same unsupervised image segmentation methods. We evaluate our approach using the BSDS500 image segmentation benchmark dataset and show the performance enhancement introduced by our approach in terms of both increased segmentation accuracy and reduced computation time. We examine the robustness of the trained CAE and show that it is directly applicable to other natural scene images.

The traditional multi-resolution Markov random field (MRMRF) model uses two-component Markov random field model on each resolution, and requires training data to estimate the necessary model parameters, which is unsuitable for unsupervised image segmentation. Under this circumstance, a new multi-resolution Markov random field model with variable potential for unsupervised texture image segmentation is presented. The new model solves this problem by introducing a variable potential function for multi-level logistic distribution (MLL) model on each scale. Using this method, the new model can automatically estimate model parameters and produce accurate unsupervised segmentation results. The results obtained on synthetic texture images and remote sensing images demonstrate that a better segmentation is achieved by our model than the traditional MRMRF model.

The application of a Markov random field (MRF) state model in an expectation-maximization (EM)-based approach to unsupervised image segmentation is investigated. In the calculation of the marginal distribution of the state field, it is shown that the use of the expected state values for interacting pixel sites in the computation of the MRF energy function may be interpreted as a mean-field approximation. The implications of calculating a self-consistent expectation of the state field are considered. EM convergence criteria are considered, and a criterion based upon divergence is proposed. Experimental results based on synthetic data illustrate the performance advantage of the mean-field approximation and the computational advantage of using self-consistent expectations. >

The aim of our paper is to present a new unsupervised Bayesian image segmentation method using a recent model by hidden fuzzy Markov fields. The main problem of parameter estimation is solved using a recent general method of estimation regarding hidden data, called iterative conditional estimation (ICE). This has been successfully applied in classical hidden Markov fields based segmentations. The first part of our work involves estimating the parameters defining the Markovian distribution of the fuzzy picture without noise. We then combine this algorithm with the ICE method in order to estimate all the parameters of the noisy picture.

We propose a novel approach to investigate and implement unsupervised image content understanding and segmentation of color industrial images like medical imaging, forensic imaging, security and surveillance imaging, biotechnical imaging, biometrics, mineral and mining imaging, material science imaging, and many more. In this particular work, our focus will be on medical images only. The aim is to develop a computer aided diagnosis (CAD) system based on a newly developed Multidimensional Spatially Variant Finite Mixture Model (MSVFMM) using Markov Random Fields (MRF) Model. Unsupervised means automatic discovery of classes or clusters in images rather than generating the class or cluster descriptions from training image sets. The aim of this work is to produce precise segmentation of color medical images on the basis of subtle color and texture variation. Finer segmentation of images has tremendous potential in medical imaging where subtle information related to color and texture is required to analyze the image accurately. In this particular work, we have used CIE-Luv and Daubechies wavelet transforms as color and texture descriptors respectively. Using the combined effect of a CIE-Luv color model and Daubechies transforms, we can segment color medical images precisely in a meaningful manner. The evaluation of the results is done through comparison of the segmentation quality with another similar alternative approach and it is found that the proposed approach is capable of producing more faithful segmentation.

Improving the runtime of MRF based method for MRI brain segmentation

Unsupervised segmentation of noisy and textured images using Markov random fields

Image segmentation is the process of partitioning a digital image into a set of homogeneous regions (according to some homogeneity criterion) to facilitate a subsequent higher-level analysis. In this context, the present paper proposes an unsupervised and graph-based method of image segmentation, which is driven by an application goal, namely, the generation of image segments associated with a user-defined and application-specific goal. A graph, together with a random grid of source elements, is defined on top of the input image. From each source satisfying a goal-driven predicate, called seed, a propagation algorithm assigns a cost to each pixel on the basis of similarity and topological connectivity, measuring the degree of association with the reference seed. Then, the set of most significant regions is automatically extracted and used to estimate a statistical model for each region. Finally, the segmentation problem is expressed in a Bayesian framework in terms of probabilistic Markov random field (MRF) graphical modeling. An ad hoc energy function is defined based on parametric models, a seed-specific spatial feature, a background-specific potential, and local-contextual information. This energy function is minimized through graph cuts and, more specifically, the alpha-beta swap algorithm, yielding the final goal-driven segmentation based on the maximum a posteriori (MAP) decision rule. The proposed method does not require deep a priori knowledge (e.g., labelled datasets), as it only requires the choice of a goal-driven predicate and a suited parametric model for the data. In the experimental validation with both magnetic resonance (MR) and synthetic aperture radar (SAR) images, the method demonstrates robustness, versatility, and applicability to different domains, thus allowing for further analyses guided by the generated products.

This work deals with unsupervised sonar image segmentation. We present a new estimation segmentation procedure using the recent iterative method of estimation called Iterative Conditional Estimation (ICE) [1]. This method takes into account the variety of the laws in the distribution mixture of a sonar image and the estimation of the parameters of the label field (modeled by a Markov Random Field (MRF)). For the estimation step, we use a maximum likelihood technique to estimate the noise model parameters, and the least squares method proposed by Derin et al. [2] to estimate the MRF prior model. Then, in order to obtain an accurate segmentation map and to speed up the convergence rate, we use a multigrid strategy exploiting the previously estimated parameters. This technique has been successfully applied to real sonar images 1, and is compatible with an automatic processing of massive amounts of data.

Graph convolutional network – Long short term memory neural network- multi layer perceptron- Gaussian progress regression model: A new deep learning model for predicting ozone concertation

Road detection aims at detecting the (drivable) road surface ahead vehicle and plays a crucial role in driver assistance system. To improve the accuracy and robustness of road detection approaches in complex environments, a new road detection method based on CNN (convolutional neural network) and MRF (markov random field) is proposed. The original road image is segmented into super-pixels of uniform size using simple linear iterative clustering (SLIC) algorithm. On this basis, we train the CNN which can automatically learn the features that are most beneficial to classification. Then, the trained CNN is applied to classify road region and non-road region. Finally, based on the relationship between the super-pixels neighborhood, we utilize MRF to optimize the classification results of CNN. Quantitative and qualitative experiments on the publicly datasets demonstrate that the proposed method is robust in complex environments. Furthermore, compared with state-of-the-art algorithms, the approach provides the better performance.

Unsupervised brain tumor segmentation using a symmetric-driven adversarial network