Large-Scale Hierarchical Text Classification Based on Path Semantic Vector and Prior Information

Due to the complex label hierarchy and intensive labeling cost in practice, the hierarchical text classification (HTC) suffers a poor performance especially when low-resource or few-shot settings are considered. Recently, there is a growing trend of applying prompts on pre-trained language models (PLMs), which has exhibited effectiveness in the few-shot flat text classification tasks. However, limited work has studied the paradigm of prompt-based learning in the HTC problem when the training data is extremely scarce. In this work, we define a path-based few-shot setting and establish a strict path-based evaluation metric to further explore few-shot HTC tasks. To address the issue, we propose the hierarchical verbalizer (“HierVerb”), a multi-verbalizer framework treating HTC as a single- or multi-label classification problem at multiple layers and learning vectors as verbalizers constrained by hierarchical structure and hierarchical contrastive learning. In this manner, HierVerb fuses label hierarchy knowledge into verbalizers and remarkably outperforms those who inject hierarchy through graph encoders, maximizing the benefits of PLMs. Extensive experiments on three popular HTC datasets under the few-shot settings demonstrate that prompt with HierVerb significantly boosts the HTC performance, meanwhile indicating an elegant way to bridge the gap between the large pre-trained model and downstream hierarchical classification tasks.

Hierarchical classification has been becoming a popular research topic nowadays, particularly on the web as text categorization. For a large web corpus, there can be a hierarchy with hundreds of thousands of topics, so it is common to handle this task using a flat classification approach, inducing a binary classifier only for the leaf-node classes. However, it always suffers from such low prediction accuracy due to an imbalanced issue in the training data. In this paper, we propose two novel strategies: (i) “Top-Level Pruning” to narrow down the candidate classes, and (ii) “Exclusive Top-Level Training Policy” to build more effective classifiers by utilizing the top-level data. The experiments on the Wikipedia dataset show that our system outperforms the traditional flat approach unanimously on all hierarchical classification metrics.

Hierarchy-Aware and Label Balanced Model for Hierarchical Text Classification

This paper presents an investigation into the summarisation of the free text element of questionnaire data using hierarchical text classification. The process makes the assumption that text summarisation can be achieved using a classification approach whereby several class labels can be associated with documents which then constitute the summarisation. A hierarchical classification approach is suggested which offers the advantage that different levels of classification can be used and the summarisation customised according to which branch of the tree the current document is located. The approach is evaluated using free text from questionnaires used in the SAVSNET (Small Animal Veterinary Surveillance Network) project. The results demonstrate the viability of using hierarchical classification to generate free text summaries.

Hierarchical multi-label text classification (HMTC) has become rather challenging when it requires handling large sets of closely related categories. We present a novel software that provides a hybrid embedding-based text representation for HMTC, shortened as HE-HMTC. It made full use of categories’ structure and their labels semantics to enrich the text representation, and therefore attempted to improve the classification performance of the text. In addition, HE-HMTC facilitates improving the accuracy of HMTC tasks. Moreover, our HE-HMTC can easily be generalized in other hierarchical classification tasks and achieve superior performance.

Hierarchical text categorization is a task that aims to assign predefined categories to text documents with hierarchical constraint. Recently, deep learning techniques has shown many success results in various fields, especially, in text categorization. In our previous work called Shared Hidden Layer Neural Network (SHL-NN), it has shown that sharing information between levels can improve a performance of the model. However, this work is based on a sequence of unsupervised word embedding vectors, so the performance should be limited. In this paper, we propose a supervised document embedding specifically designed for hierarchical text categorization based on Autoencoder, which is trained from both words and labels. To enhance the embedding vectors, the document embedding strategies are invented to utilize the class hierarchy information in the training process. To transfer the prediction result from the parent classes, the shared information technique has been improved to be more flexible and efficient. The experiment was conducted on three standard benchmarks: WIPO-C, WIPO-D and Wiki comparing to two baselines: SHL-NN and a top-down based SVM framework with TF-IDF inputs called “HR-SVM.” The results show that the proposed model outperforms all baselines in terms of F1 macro.

Hierarchical Classification is a very important classification task for arranging data in a hierarchical structure. Hierarchical arrangement of data is one of the best methods to achieve better understanding of complex data. In this paper, we propose the HMAC method to perform Hierarchical Multi-label Associative Classification. This method uses multiple and negative rules to predict class-set and to filter exceptional cases in order to improve both accuracy and explanatory ability of the resulting classifier. Redundant rule pruning method for negative rules and rule ranking method for hierarchical associative classification are developed. Moreover, we propose a rule evaluation measure, Sim, that tread-off between F-measure and Jaccard's coefficient to encode hierarchical structure meaning of actual and predicted node collections. We show that Sim is robust and simple rule evaluation measure. Experimental results show that HMAC improves accuracy and explanatory ability of hierarchical classifiers compared with various rule ranking and pruning techniques.

Hierarchical classification for acoustic scenes using deep learning

In this paper, we model learning to rank algorithms based on structural dependencies in hierarchical multi-label text categorization (TC). Our method uses the classification probability of the binary classifiers of a standard top-down approach to generate k-best hypotheses. The latter are generated according to their global probability while at the same time satisfy the structural constraints between father and children nodes. The rank is then refined using Support Vector Machines and tree kernels applied to a structural representation of hypotheses, i.e., a hierarchy tree in which the outcome of binary one-vs-all classifiers is directly marked in its nodes. Our extensive experiments on the whole Reuters Corpus Volume 1 show that our models significantly improve over the state of the art in TC, thanks to the use of structural dependecies.

Hierarchical text classification is an essential yet challenging subtask of multi-label text classification with a taxonomic hierarchy. Existing methods have difficulties in modeling the hierarchical label structure in a global view. Furthermore, they cannot make full use of the mutual interactions between the text feature space and the label space. In this paper, we formulate the hierarchy as a directed graph and introduce hierarchy-aware structure encoders for modeling label dependencies. Based on the hierarchy encoder, we propose a novel end-to-end hierarchy-aware global model (HiAGM) with two variants. A multi-label attention variant (HiAGM-LA) learns hierarchy-aware label embeddings through the hierarchy encoder and conducts inductive fusion of label-aware text features. A text feature propagation model (HiAGM-TP) is proposed as the deductive variant that directly feeds text features into hierarchy encoders. Compared with previous works, both HiAGM-LA and HiAGM-TP achieve significant and consistent improvements on three benchmark datasets.

Along with the exponential growth of text data on the Web, particularly of the user-generated content, comes an increasing need for hierarchically organizing documents, retrieving documents accurately, and discovering evolutionary trends of various popular topics from the data. However, all of these are challenging due to the diversity, heterogeneity, noisiness and time-sensitivity of Web 2.0 data. Motivated by this, we tackle the challenges at a fundamental level, by proposing a novel topic modeling method with ontological guidance. It may be used to discover topic language models formalizing various terms relevant to given topics using the Web data. The topic model takes into account both the ontological relationships amongst the topics defined in a topic taxonomy and also word co-occurrence patterns in the data to automatically identify the portions in the data relevant to the topics. Then, it estimates language models for these topics from these relevant portions. At an application level, we use the topic model to propose novel approaches for three different tasks, namely hierarchical text classification without labeled data, information retrieval with pseudo-relevance feedback, and discovering topic evolutionary trends. Our classification experiment on the IPTC (International Press and Telecommunications Council) taxonomy, containing more than 1100 topics, shows that our approach achieves a performance of 67% in terms of the hierarchical version of the F-1 measure, without using any labeled data. Our retrieval experiments on five benchmark datasets show that compared to baseline retrieval (without pseudo-relevance feed-back), our approach improves on average 39% in terms of mean average precision. Finally, for the last task, using blog data, our approach discovers meaningful insights on how the crowd responds to various news topics such as the language used to discuss each topic, how this language drifts over time, and when the crowd’s focus on a topic increases, reaches a peak, and declines.

Hierarchical text classification concerning the relationship among categories has become an interesting problem recently. Most research has focused on tree-structured categories, but in reality directed acyclic graph (DAG) – structured categories, where a child category may have more than one parent category, appear more often. In this paper, we introduce three approaches, namely, flat, tree-based, and DAG-based, for solving the multi-label text classification problem in which categories are organized as a DAG, and documents are classified into both leaf and internal categories. We also present experimental results of the methods using SVMs as classifiers on the Reuters-21578 collection and our data set of research papers in Artificial Intelligence.Keywordstext classificationhierarchiesmulti-labelsSVM

Reservoir fluid identification is a critical aspect of oil and gas geophysical exploration. Accurate fluid identification directly impacts the interpretation of subsurface geological conditions, reduces exploration risks, and provides essential guidance for formulating oil and gas development strategies. Therefore, reliable and precise fluid identification is indispensable across different stages of oil and gas exploration and production. This study proposes a hierarchical classification method based on conditional Variational Autoencoder (cVAE) and time series forest (TSF) algorithms to address reservoir fluid identification under complex geological conditions. The main contributions of this work are as follows: (i) the cVAE is used to pre-process the logging data to suppress local high-frequency disturbances and isolated anomalies that may exist in the logging curves, thereby improving the quality of the input data; and (ii) hierarchical classification strategy is utilized to perform the fluid identification task in two steps. The first step involves a top-level classification to distinguish the gas bearing layer from the non-gas layer. The second step refines this classification into subcategories, including the gas layer (GL), gas–water layer (GW), gas-bearing water layer (GBW), water layer (WL), and non-reservoir layer (DW). This can fully address the challenges of imbalanced datasets and improve the recognition accuracy of minority classes. Additionally, integrating the TSF algorithm within the hierarchical classification framework effectively captures the sequential characteristics of well logging data, improving the model’s ability to recognize complex geological patterns. A real-world application in a block of the Yinggehai Basin in the South China Sea demonstrated the superior performance of the proposed model. Experimental results show that the method achieves an accuracy of over 84% in all four wells, enabling accurate and reliable reservoir fluid classification.

Question retrieval aims to increase the accessibility of the community Question Answer (cQA) archives and has attracted increasing research interests recently. In this paper, we present a novel method for improving the question retrieval performance by investigating the question term selection and weighting as well as reranking results. Different from previous work, we propose a hierarchical question classification method with a sparse regularization to mimc user's question labeling in cQAs. Based on the hierarchical classification, we explore the local context of the question for term selection and reranking results and then integrating them into our proposed general question retrieval framework. The experimental results on a Yahoo! Answers dataset show the effectiveness of our method as compared to existing general question retrieval models and some state-of-the-art methods of utilizing category information for question retrieval.

Seismic attributes analysis and classification for exploration and reservoir characterization have been widely published. Applications vary from standard horizon-based facies classification maps to more recent 3D multi-attribute facies classification volumes. The approach is usually the same and run in a two-step procedure. First, an unsupervised classification aims at revealing the natural clustering of the data, and second, a supervised scheme is applied where training and validation data are used to redefine the class cloud point centers based on well log data flagging a specific fluid or lithology. The limitations of these approaches are that they are focused on one aspect of the seismic response, usually fluid, and tend to neglect the geological framework. In these workflows, more attention is put on the reservoir facies for fluid and potential lithology detection, while the seismic seismostratigraphic signature is overlooked or not used as a constrain in the attribute analysis. We present a case study in which both texture facies and fluid prediction are linked by performing a hierarchical classification and estimation scheme whereby a multiattributes volume, which captures seismic stratigraphy and texture information, is combined with AVO attributes to map fluid response into a single, coherent seismostratigraphic and reservoir facies volume. This methodology is applied for exploration data screening in offshore Borneo in the Greater Samarang sub-block (East Baram Delta, offshore Sabah, Malaysia). In this case study, geological framework, seismic geomorphology, seismic stratigraphy, and combined fluid response from AVO data calibrated with well data facilitate the development of new play concepts in the highstand system tracts and in the morphology generated by incisions in the shoreface deposits during the low stands.