Assessing Representation Learning and Clustering Algorithms for Computer-Assisted Image Annotation—Simulating and Benchmarking MorphoCluster

Joint multi-view representation and image annotation via optimal predictive subspace learning

Image distance metric learning based on neighborhood sets for automatic image annotation

This article presents an automatic image level annotation approach that takes advantage of both context and semantics presented in segmented images. The proposed approach is based on the optimization of classes’ scores using particle swarm optimization. In addition, random forest classifier and normalized cuts algorithm have been applied for automatic image classification, annotation, and clustering. For the proposed approach, each input image is segmented using the normalized cuts segmentation algorithm in order to create a descriptor for each segment. Two parameter selection models have been selected for particle swarm optimization algorithm and many voting techniques have been implemented to find the most suitable set of annotation words per image. Experimental results, using Corel5k benchmark annotated images dataset, demonstrate that applying optimization algorithms along with random forest classifier achieved noticeable increase in image annotation performance measures compared to related researches on the same dataset.

State representation learning for control: An overview

We propose novel deep speaker representation learning that considers perceptual similarity among speakers for multi-speaker generative modeling. Following its success in accurate discriminative modeling of speaker individuality, knowledge of deep speaker representation learning (i.e., speaker representation learning using deep neural networks) has been introduced to multi-speaker generative modeling. However, the conventional discriminative algorithm does not necessarily learn speaker embeddings suitable for such generative modeling, which may result in lower quality and less controllability of synthetic speech. We propose three representation learning algorithms that utilize a perceptual speaker similarity matrix obtained by large-scale perceptual scoring of speaker-pair similarity. The algorithms train a speaker encoder to learn speaker embeddings with three different representations of the matrix: a set of vectors, the Gram matrix, and a graph. Furthermore, we propose an active learning algorithm that iterates the perceptual scoring and speaker encoder training. To obtain accurate embeddings while reducing costs of scoring and training, the algorithm selects unscored speaker-pairs to be scored next on the basis of the sequentially-trained speaker encoder's similarity prediction results. Experimental evaluation results show that 1) the proposed representation learning algorithms learn speaker embeddings strongly correlated with perceptual speaker-pair similarity, 2) the embeddings improve synthetic speech quality in speech autoencoding tasks better than conventional d-vectors learned by discriminative modeling, 3) the proposed active learning algorithm achieves higher synthetic speech quality while reducing costs of scoring and training, and 4) among the proposed similarity {vector, matrix, graph} embedding algorithms, the first achieves the best speaker similarity for synthetic speech and the third gives the most improvement in the synthetic speech naturalness.

This paper presents a novel multi-label classification framework for domains with large numbers of labels. Automatic image annotation is such a domain, as the available semantic concepts are typically hundreds. The proposed framework comprises an initial clustering phase that breaks the original training set into several disjoint clusters of data. It then trains a multi-label classifier from the data of each cluster. Given a new test instance, the framework first finds the nearest cluster and then applies the corresponding model. Empirical results using two clustering algorithms, four multi-label classification algorithms and three image annotation data sets suggest that the proposed approach can improve the performance and reduce the training time of standard multi-label classification algorithms, particularly in the case of large number of labels.

The development of technology generates huge amounts of non-textual information, such as images. An efficient image annotation and retrieval system is highly desired. Clustering algorithms make it possible to represent visual features of images with finite symbols. Based on this, many statistical models, which analyze correspondence between visual features and words and discover hidden semantics, have been published. These models improve the annotation and retrieval of large image databases. However, image data usually have a large number of dimensions. Traditional clustering algorithms assign equal weights to these dimensions, and become confounded in the process of dealing with these dimensions. In this paper, we propose a top-down, subspace clustering algorithm as a solution to this problem. For a given cluster, we determine relevant features based on histogram analysis and assign greater weight to relevant features as compared to less relevant features. We have implemented four different models to link visual tokens with keywords based on the clustering results of our clustering algorithm and K-means algorithm, and evaluated performance using precision, recall and correspondence accuracy using benchmark dataset. The results show that our algorithm is better than traditional ones for automatic image annotation and retrieval.

Using image classification approach for automatic image annotation is one promising method. In order to improve image annotation accuracy, recent researchers propose to use AdaBoost algorithm for the ensemble of classifiers. But in these researches, only fewer features are used. We construct multi-class classifiers for all the image low-level feature of multimedia content description interface and their potential combinations respectively, k-nearest neighbor classifier is used as the base classifier and dasiaone vs. onepsila scheme is chosen to build multi-class classifiers. A bi-coded chromosome genetic algorithm is used to select the optimal classifier subset as weak classifiers and corresponding weights, which are used for the combination of an ensemble classifier by weighted majority voting scheme. The results of experiment over 2000 classified Corel images show that the approach selects 4 of 325 multi-class classifiers as weak classifiers as well as corresponding optimized weights to generate an ensemble classifier. The ensemble classifier created by the bi-coded chromosome genetic algorithm has higher accuracy than that by AdaBoost algorithm.

Automatic image annotation is one of the research fields helping to extract the meaning of images, which aims at the production of a set of semantic annotations for an image to help better present the concept. Over the past few decades, researchers have developed a number of approaches for automatic image annotation. Nevertheless, previous studies have not fully accounted for visual features and annotated features. Therefore, it is still possible to achieve a better annotation performance by combining visual and annotated information. In this study, our aim is to associate multiple semantic tags with a given image. In particular, we detect how to obtain the image annotation by utilizing visual and annotated information. To take advantage of visual information, we first designed a modified neural network method to acquire the features of the image content. In addition, to obtain the annotated features, we exploit an aggregated network embedding approach that consists of annotation embedding, social embedding, profile embedding, and semantic embedding. Finally, to produce an accurate image annotation, we integrate the two aforementioned methods, that is, combining the visual and annotated information, to build a unified cooperative training framework. The experimental results on three real-world datasets clarify that our presented method is superior to the current popular image annotation approaches.

Dynamic Adaboost learning with feature selection based on parallel genetic algorithm for image annotation

The annotation of magnetic resonance imaging (MRI) images plays an important role in deep learning-based MRI segmentation tasks. Semi-automatic annotation algorithms are helpful for improving the efficiency and reducing the difficulty of MRI image annotation. However, the existing semi-automatic annotation algorithms based on deep learning have poor pre-annotation performance in the case of insufficient segmentation labels. In this paper, we propose a semi-automatic MRI annotation algorithm based on semi-weakly supervised learning. In order to achieve a better pre-annotation performance in the case of insufficient segmentation labels, semi-supervised and weakly supervised learning were introduced, and a semi-weakly supervised learning segmentation algorithm based on sparse labels was proposed. In addition, in order to improve the contribution rate of a single segmentation label to the performance of the pre-annotation model, an iterative annotation strategy based on active learning was designed. The experimental results on public MRI datasets show that the proposed algorithm achieved an equivalent pre-annotation performance when the number of segmentation labels was much less than that of the fully supervised learning algorithm, which proves the effectiveness of the proposed algorithm.

The using of deep learning method belongs to the application and research of artificial intelligence technology in medical field for assisting medical image information processing. Due to the larger difference between the image underlying character features and high-level semantic concepts described, the existing image annotation algorithms perform not very ideal, therefore, the we propose an image annotation framework based on multimodule deep transfer learning method, aiming at the application of deep convolution neural network technology to optimize the neural network parameters, and improve the tagging accuracy in detail. The proposed multimodule deep learning annotation framework uses two stages to optimiz neural network parameters: (1) using deep neural network to optimize the single-mode condition parameters; (2) using the phase correlation, to realize the multi-mode state, the experiment of the optimal combination of public data sets shows that the scheme can effectively improve the performance of image annotation.

The development of technology generates huge amounts of non-textual information, such as images. An efficient image annotation and retrieval system is highly desired. Clustering algorithms make it possible to represent visual features of images with finite symbols. Based on this, many statistical models, which analyze correspondence between visual features and words and discover hidden semantics, have been published. These models improve the annotation and retrieval of large image databases. However, image data usually have a large number of dimensions. Traditional clustering algorithms assign equal weights to these dimensions, and become confounded in the process of dealing with these dimensions. In this paper, we propose weighted feature selection algorithm as a solution to this problem. For a given cluster, we determine relevant features based on histogram analysis and assign greater weight to relevant features as compared to less relevant features. We have implemented various different models to link visual tokens with keywords based on the clustering results of K-means algorithm with weighted feature selection and without feature selection, and evaluated performance using precision, recall and correspondence accuracy using benchmark dataset. The results show that weighted feature selection is better than traditional ones for automatic image annotation and retrieval.

Performing semantic segmentation on point clouds is the primary method by which machines perceive 3D scenes in a fine-grained manner. Deep learning algorithms usually require many pointwise annotations obtained with specialized tools, which is a laborious and inefficient process. To this end, we develop two frameworks for training point cloud semantic segmentation networks, one that utilizes fewer projected image annotations and another that employs sparse scribble image annotations, making the process more flexible and user friendly. However, back-projecting 2D-pixel labels to 3D points during loss calculations always introduces errors. To increase the back-projection accuracy of our approach, we first identify and record potential pixel-point correspondence errors and then develop strategies for constructing an accurate back-projection mapping matrix. Specifically, we filter out occluded and noisy points to avoid incorrect label allocations and permit multiclass assignments to adjust the ambiguity of boundary points. By incorporating an accurate back-projection mechanism into the loss functions of the proposed training frameworks, our networks can perform well with only four projected image annotations or even sparse scribble image annotations for each scene. This results in state-of-the-art performance compared with that of other weakly supervised point cloud semantic segmentation approaches, and the outcomes are even comparable to those produced by fully supervised methods on the S3DIS and ScanNet-v2 datasets.

The virtual power plant (VPP) is an important technology for realizing smart grids. VPPs are not limited by geographical location, which greatly aggregates the distributed energy resources. To divide a VPP across regions, we no longer limit it to power lines. This paper proposes a three-stage strategy for the classification of VPPs that includes graph model construction, communication capacity assessment and cross-regional classification. To construct the VPP graph model, complex network theory and communication matrices are used. In addition, a network representation learning and clustering algorithm is used to classify the VPP decision areas. Then, the classification results are evaluated using modularity and network communication sequence entropy to identify VPP boundaries. Finally, the validity of the method is verified by simulating the test network and the real network.