Scalable multi-label patent classification via iterative large language model-assisted active learning

Most existing work on multi-label learning focused on supervised learning which requires manual annotation samples that is labor-intensive, time-consuming and costly. To address such a problem, we present a novel method that incorporates active learning into the semi-supervised learning for multi-label image classification. What’s more, aiming at the curse of dimensionality existing in high-dimensional data, we explore a dimensionality reduction technique with non-negative sparseness constraint to extract a group of features that can completely describe the data and hence make the learning model more efficiently. Experimental results on common data sets validate that the proposed algorithm is relatively effective to improve the performance of the learner in multi-label classification, and the obtained learner is with reliability and robustness after data dimensionality using NNS-DR (Non-Negative Sparseness for Dimensionality Reduction).

Multi-label classification and interactive NLP-based visualization of electric vehicle patent data

International audience

We study the problem of active learning for multilabel classification. We focus on the real-world scenario where the average number of positive (relevant) labels per data point is small leading to positive label sparsity. Carrying out mutual information based near-optimal active learning in this setting is a challenging task since the computational complexity involved is exponential in the total number of labels. We propose a novel inference algorithm for the sparse Bayesian multilabel model of [17]. The benefit of this alternate inference scheme is that it enables a natural approximation of the mutual information objective. We prove that the approximation leads to an identical solution to the exact optimization problem but at a fraction of the optimization cost. This allows us to carry out efficient, non-myopic, and near-optimal active learning for sparse multilabel classification. Extensive experiments reveal the effectiveness of the method.

Deep learning can discover intricate patterns hidden in big data, and has much better scalability than traditional machine learning when the volume of data increases dramatically. Thus, deep learning has gained many successes in various domains and applications such as image classification, text classification, and machine translation. In this paper, we use deep learning to classify geographical features (e.g., mountains, rivers, landmarks, and cities) from text, using geolocated Wikipedia entries as the case study application. We employ one of the most commonly used deep learning architectures, convolutional neural networks (CNNs) and its integration with active learning (creating what we call active CNNs), to train the geographical feature classifiers on the Wikipedia text data set obtained from GeoNames (which provides the feature type for each geolocated entity). We evaluate the performance of CNNs and active CNNs with multiple metrics (i.e., accuracy, F1 score, and confusion matrix). Our experiment results demonstrated that CNNs and active CNNs can effectively classify geo-referenced text entities into predefined geographical features. In addition, our experiment results show that active CNNs outperform CNNs for hard to distinguish classes. In our experiment, we also compared results for hierarchical multi-class classification and flat multiclass classification, and the results show that hierarchical multiclass classification significantly outperforms flat multi-class classification for the data set we used.

In multi-label image classification, each image is always associated with multiple labels and labels are usually correlated with each other. The intrinsic relation among labels can definitely contribute to classifier training. However, most previous studies on active learning for multi-label image classification purely mine label correlation based on observed label distribution. They ignore the mapping relation between examples and their labels. This mapping relation also implicates label relationship. Ignoring the mapping relation leads to an uncomprehensive label correlation estimation and results in a bad performance for classification. In this paper, we propose a novel multi-label active learning with low-rank mapping for image classification, called LMMAL, to solve this issue. More precisely, we train a low-rank mapping matrix to signify the mapping relation between the feature space and the label space of a certain multi-label dataset. Using this low-rank mapping relation, we exploit a full label correlation. Subsequently, an effective sampling strategy is designed by integrating this potential information with uncertainty to select the most informative example-label pairs. In addition, we extend LMMAL with automatic labeling (denoted as AL-LMMAL) to further reduce the annotation workload of active learning. Empirical results demonstrate the effectiveness of our approaches.

With the rapid development of social networks, the networked multi-label classification algorithms have gained wide attention. The existing networked multi-label classification algorithms mostly only consider the homogeneity or heterogeneity of the network without taking the imbalance of the network into account, and this is actually pretty common in real network environments, which deserves more attention. Moreover, the selection strategy of training set is very critical for multi-label classification algorithm, because it will directly affect both the parameter updating inside the classifier and the precision of the classifier. The application of active learning to the selection of training set can effectively improve the precision of the classifier. Similarly, the application of imbalanced data processing strategies to the selection of training sets also makes classifiers more suitable for imbalanced data networks. Thereout, we propose an algorithm BSHD (Block Sampling with selecting the Highest Degree nodes), which is an active learning based imbalanced networked multi-label classification algorithm. In this algorithm, we divide the network according to the edge density and utilize the oversampling and undersampling to dispose each block. Then we select the nodes with the highest degree from each block to form the training set. Experimental results show that our proposed BSHD outperforms other state-of-arts approaches.

Material extrusion could enable on-demand production of complex and personalized parts but is limited by low reliability, particularly in higher-volume production. Machine learning-based methods may enhance reliability, but are often themselves insufficiently reliable for use in production. Foundation artificial intelligence models have enabled significant improvements in performance across many tasks. Here, a vision-based control system is reported, coupling active learning and uncertainty awareness with a foundation model to continually learn to build a specific part better. The resulting framework is called Iterative Learning, as it improves performance by learning from its own errors during repeated build cycles of the same part. The iterative learning approach is shown to enable robust error detection and correction while being more space, time and computationally efficient compared to a naive fine-tuning approach. This provides a path showing how foundation models may be adapted to enhance reliability across a wider range of additive manufacturing processes.

This paper studies active learning in structured probabilistic models such as Conditional Random Fields (CRFs). This is a challenging problem because unlike unstructured prediction problems such as binary or multi-class classification, structured prediction problems involve a distribution with an exponentially-large support, for instance, over the space of all possible segmentations of an image. Thus, the entropy of such models is typically intractable to compute. We propose a crude yet surprisingly effective histogram approximation to the Gibbs distribution, which replaces the exponentially-large support with a coarsened distribution that may be viewed as a histogram over M bins. We show that our approach outperforms a number of baselines and results in a 90%-reduction in the number of annotations needed to achieve nearly the same accuracy as learning from the entire dataset.

To detect specific types of bugs and vulnerabilities, static analysis tools must be correctly configured with security-relevant methods (SRM), e.g., sources, sinks, sanitizers and authentication methods–usually a very labour-intensive and error-prone process. This work presents the semi-automated tool SWAN_ASSIST, which aids the configuration with an IntelliJ plugin based on active machine learning. It integrates our novel automated machine-learning approach SWAN, which identifies and classifies Java SRM. SWAN_ASSIST further integrates user feedback through iterative learning. SWAN_ASSIST aids developers by asking them to classify at each point in time exactly those methods whose classification best impact the classification result. Our experiments show that SWAN_ASSIST classifies SRM with a high precision, and requires a relatively low effort from the user. A video demo of SWAN_ASSIST can be found at https://youtu.be/fSyD3V6EQOY. The source code is available at https://github.com/secure-software-engineering/swan.

Margin-based active learning remains the most widely used active learning paradigm due to its simplicity and empirical successes. However, most works are limited to binary or multiclass prediction problems, thus restricting the applicability of these approaches to many complex prediction problems where active learning would be most useful. For example, machine learning techniques for natural language processing applications often require combining multiple interdependent prediction problems—generally referred to as learning in structured output spaces. In many such application domains, complexity is further managed by decomposing a complex prediction into a sequence of predictions where earlier predictions are used as input to later predictions—commonly referred to as a pipeline model. This work describes methods for extending existing margin-based active learning techniques to these two settings, thus increasing the scope of problems for which active learning can be applied. We empirically validate these proposed active learning techniques by reducing the annotated data requirements on multiple instances of synthetic data, a semantic role labeling task, and a named entity and relation extraction system.

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2004 SIAM International Conference on Data Mining (SDM)Class-Specific Ensembles for Active Learning in Digital ImageryAmit Mandvikar and Huan LiuAmit Mandvikar and Huan Liupp.412 - 421Chapter DOI:https://doi.org/10.1137/1.9781611972740.38PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract In many real-world tasks of image classification, limited amounts of labeled data are available to train automatic classifiers. Consequently, extensive human expert involvement is required for instance labeling. Detecting Egeria densa in digital imagery is one such real-world classification task. It presents an additional challenge due to subtle spectral changes in Egeria, which makes it difficult to find a single accurate classifier. A novel solution is proposed to employ an ensemble of classifiers for each class (class-specific ensembles), combined with an active learning scheme. The class-specific ensembles are implicitly diverse. Diversity is required to increase the overall accuracy when combining predictions. The combined predictions of the ensembles can be used to reduce the uncertainty in detecting Egeria. Iterative active learning is then suggested to adapt the ensembles to the new images, unseen to the active learner. A novel solution to build compact ensembles is also presented, which are needed to expedite the re-training of the active learner. The combined results are accurate and compact ensembles, which require significantly less expert involvement for image region classification. Previous chapter Next chapter RelatedDetails Published:2004ISBN:978-0-89871-568-2eISBN:978-1-61197-274-0 https://doi.org/10.1137/1.9781611972740Book Series Name:ProceedingsBook Code:PR117Book Pages:xiv + 537

Interactive labelling of a multivariate dataset for supervised machine learning using linked visualisations, clustering, and active learning

Recognizing human emotions from pieces of natural language has been a challenging task in artificial intelligence, for the difficulty of acquiring high-quality training examples. In this paper, we propose a novel method based on active learning to progressively improve the performance of supervised text emotion classification models, with as few human labor as possible in annotating the training examples. Specifically, the active learning algorithm interactively communicates with the supervised emotion classification model to find the potentially most effective training examples from a huge set of unlabeled data and increases the training data by acquiring emotion labels for these examples from the human experts. Our experiment of multi-label emotion classification on Japanese tweets suggests that the proposed method is effective in steadily improving the supervised classification results by incrementally feeding a classification model with the new tweets of well-balanced emotion labels.

Multilabel image classification has been a hot topic in the field of computer vision and image understanding in recent years. To achieve better classification performance with fewer labeled images, multilabel active learning is used for this scenario. Several active learning methods have been proposed for multilabel image classification. However, all of them assume that either all training images have complete labels or label correlations are given at the beginning. These two assumptions are unrealistic. In fact, it is very difficult to obtain complete labels for each example, in particular when the size of labels in a multilabel dataset is very large. Typically, only partial labels are available. This is one type of “weak label” problem. To solve this weak label problem inside multilabel active learning, this paper proposes a novel solution called AE-WLMAL. AE-WLMAL explores conditional label correlations on the weak label problem with the help of input features and then utilizes label correlations to construct a unified sampling strategy and evaluate the informativeness of each example-label pair in a multilabel dataset for active sampling. In addition, a pruning strategy is adopted to further improve its computation efficiency. Moreover, AE-WLAML exploits label correlations to infer labels for unlabeled images, which further reduces human labeling cost. Our experimental results on seven real-world datasets show that AE-WLMAL consistently outperforms existing approaches.