Random-Walk Term Weighting for Improved Text Classification

This paper describes a new approach for estimating term weights in a text classification task. The approach uses term co-occurrence as a measure of dependency between word features. A random walk model is applied on a graph encoding words and co-occurrence dependencies, resulting in scores that represent a quantification of how a particular word feature contributes to a given context. We argue that by modeling feature weights using these scores, as opposed to the traditional frequency-based scores, we can achieve better results in a text classification task. Experiments performed on four standard classification datasets show that the new random-walk based approach outperforms the traditional term frequency approach to feature weighting.

φq-field theory for portfolio optimization: “fat tails” and nonlinear correlations

Topic modelling is the state of the art technique for understanding, organizing, and extracting information from text collections. Traditional topic modeling approaches apply probabilistic techniques to generate the list of topics from collections. Nevertheless, human understands, summarizes and discovers the topics based on the meaning of the content. Hence, the quality of the topic models can be improved by grasping the meaning from the content. In this paper, we propose an approach to identify sets of meaningful terms based on ontology, called Semantic-based Patterns, which represent the content of a collection of documents. A set of related semantic-based patterns can be used to represent a latent topic in the collection. The proposed Topic Representation using Semantic-based Patterns aims to generate semantically meaningful patterns based on ontology rather than term co-occurrence as what existing topic modelling methods do. The semantically meaningful patterns were evaluated by applying the information filtering to semantic-based topic representation. The semantic based patterns were used as features for information filtering and were evaluated by comparing against popular information filtering baseline systems. Topic quality was evaluated in terms of topic coherence and perplexity. The experimental results verified that the quality of the proposed patterns was better than features used in baseline systems for information filtering. Further, the quality of topic representation outperforms the generated topics of other topic modeling approaches.

In the text classification, The similarity between the text need to be calculated, but the existing classification methods only consider the similarity between feature words and categories and does not involve the semantic similarity between feature words. In this paper, a new classification model LDA (Latent Dirichlet Allocation) - KNN (K-Nearest Neighbor) is proposed. LDA is used to solve the problem of semantic similarity measurement in traditional text categorization. The sample space is modeled and selected by this model. In the reduced feature space, KNN classifier is used to classify the sample. The experiment was based on the Matlab software platform, and the data set was obtained from the Chinese corpus of Fudan University, and the high precision classification result was obtained with the average value of 0.933. LDA-KNN model is compared with MI(Mutual Information)-KNN model and LSI(Latent Semantic Index)-KNN model. The results show that LDA-KNN model has superior classification performance in automatic text categorization.

Web services are self-contained software components that support business process automation over the Internet, and mashup is a popular technique that creates value-added service compositions to fulfill complicated business requirements. For mashup developers, looking for desired component services from a sea of service candidates is often challenging. Therefore, web service recommendation has become a highly demanding technique. Traditional approaches, however, mostly rely on static and potentially subjectively described texts offered by service providers. In this paper, we propose a novel way of dynamically reconstructing objective service profiles based on mashup descriptions, which carry historical information of how services are used in mashups. Our key idea is to leverage mashup descriptions and structures to discover important word features of services and bridge the vocabulary gap between mashup developers and service providers. Specifically, we jointly model mashup descriptions and component service using author topic model in order to reconstruct service profiles. Exploiting word features derived from the reconstructed service profiles, a new service recommendation algorithm is developed. Experiments over a real-world data set from ProgrammableWeb.com demonstrate that our proposed service recommendation algorithm is effective and outperforms the state-of-the-art methods. Note to Practitioners —Service recommendation accuracy for mashup creation is often limited due to poor quality of service descriptions. Mashup descriptions contain valuable information about functions and features of its component services, which can be leveraged to enhance descriptive quality of original service profiles. Based on the assumption, this paper proposes a novel two-phase service recommendation framework to facilitate mashup creation. Specifically, our approach reconstructs service profiles by extracting appropriate words from historical mashup descriptions. Then, a novel service recommendation algorithm is developed by exploiting popularity and relevance measures hidden in the reconstructed profiles. Moreover, we propose the rules of dominant words discovery and employ it to further refine our algorithm.

The last decades witnessed the rapid growth of Web technologies, methodologies and applications. This special issue collects eight research papers which are extensions of the best papers of the WISE 2013 conference(14 International Conference of Web Information Systems Engineering, Nanjing, China). The first paper, by Huang et al, “Mining streams of short text for analysis of world-wide event evolutions”, addresses the limitations of existing techniques such as LDA in detecting and tracking events through short text analysis. The authors propose a novel algorithm named FTCCT (Finding Topic Clusters using Co-occurring Terms) to automatically generate topics from a short text corpus, and develop an Event EvolutionMining (EEM) algorithm to discover hot events and their evolutions. Evolutions of the hot events are also visualized in the paper. The second paper, by Li et al., “Social event identification and ranking on flickr”, focuses on social event modeling and ranking. A new method is introduced to effectively identify events by considering the spreading effect of event in the spatio-temporal space, and a selfexciting point process model is developed to capture the triggering relationships among events. Furthermore, event impact is defined and estimated via random walk based on the triggering relationships. Huang et al. study user attribute identification or profile inference in the third paper, “A multi-source integration framework for user occupation inference in social media systems”. The authors propose a comprehensive framework to infer the user occupation from his/her social activities recorded in the micro-blog system, by identifying some beneficial content features and proposing a machine learning classification model.

A software defect prediction method using binary gray wolf optimizer and machine learning algorithms

In the era of electronic commerce, online customer reviews (OCRs) have become a prevalent and valuable information source for both customers and merchants to make business decisions. This paper proposes an enhanced collaborative filtering approach based on sentiment assessment to discover the potential preferences of customers, and to predict customers’ future requirements for business services or products (collectively referred to as “entities”). Specifically, this approach involves three major steps: aspect-level sentiment assessment, customer preference mining and personalized recommendation. First, the aspect-level sentiment assessment transforms OCRs to a structured aspect-level review vector. Second, customer preference mining uses the vector to extract aspect-level feature words from sentiments and assigns polarity score to each sentiment. Finally, the feature words and sentiment polarity score are used to calculate customer preference and customers’ similarities. Personalized recommendation for services and products are generated according to customer similarity. Experiments are conducted based on the data from one of the most popular electronic commerce websites in China ( www.JD.com ). The results demonstrate that the proposed approach outperforms traditional collaborative filtering approaches in effectively recommending entities to target customers especially in the long term.

The set of features which the traditional feature selection algorithm of chi-square selected is not complete. This causes the low performance for the final text classification. Therefore, this paper proposes a method. The method utilizes word2vec to generate word vector to improve feature selection algorithm of the chi square. The algorithm applies the word vector generated by word2vec to the process of the traditional feature selection and uses these words to supplement the set of features as appropriate. Ultimately, the set of features obtained by this method has better discriminatory power. Because, the feature words with the better discriminatory power has the strong ability of distinguishing categories as its semantically similar words. On this base, multiple experiments have been carried out in this paper. The experimental results show that the performance of text classification can increase after extension of feature words.

Word vector is an important tool for natural language processing (NLP) tasks such as text classification. However, existing static language models such as Word2vec cannot solve the polysemy problem, leading to a decline in text classification performance. To solve this problem, this paper proposes a method for making Chinese word vector dynamic (MCWVD). The part of speech (POS) is used to solve the ambiguity problem caused by different POS. The POS structure graph is constructed and the syntactic structure information of POS features is extracted by GCN (Graph Convolutional Network). POS vector and word vector are concatenated into PW (POS-Word) vector. Parametric matrix is added to improve the fusion effect of POS and word features. Multilayer attention is used to distinguish the importance of different features and further update the vector expression of word vectors about the current context. Experiments on Chinese datasets THUCNews and SogouNews show that MCWVD effectively improves the accuracy of text classification and achieves better performance than CoVe (Context Vectors) and ELMo (Embeddings from Language Models). MCWVD also achieves similar performance to BERT and GPT-1 (Generative Pre-Training), but with a much lower computational cost and only 4% of BERT parameters.

Several real-world complex systems have graph-structured data, including social networks, biological networks, and knowledge graphs. A continuous increase in the quantity and quality of these graphs demands learning models to unlock the potential of this data and execute tasks, including node classification, graph classification, and link prediction. This tutorial presents machine learning on graphs, focusing on how representation learning - from traditional approaches (e.g., matrix factorization and random walks) to deep neural architectures - fosters carrying out those tasks. We also introduce representation learning over dynamic and knowledge graphs. Lastly, we discuss open problems, such as scalability and distributed network embedding systems.

Short text understanding is a key task and popular issue in current natural language processing. Because the content of short texts is characterized by sparsity and semantic limitation, the traditional search methods that analyze only the semantics of literal text for short text understanding and similarity matching have certain restrictions. In this paper, we propose a combined method based on knowledge-based conceptualization and a transformer encoder. Specifically, for each term in a short text, we obtain its concepts and enrich the short text information from a knowledge base based on cooccurrence terms and concepts, construct a convolutional neural network (CNN) to capture local context information, and introduce the subnetwork structure based on a transformer embedding encoder. Then, we embed these concepts into a low-dimensional vector space to obtain more attention from these concepts based on a transformer. Finally, the concept space and transformer encoder space construct the understanding models. An experiment shows that the method in this paper can effectively capture more semantics of short texts and can be applied to a variety of applications, such as short text information retrieval and short text classification.

Robust stereo matching using adaptive random walk with restart algorithm

This paper proposes a matching technique for learning causal associations between word features and class labels in document classification. The goal is to identify more meaningful and generalizable features than with only correlational approaches. Experiments with sentiment classification show that the proposed method identifies interpretable word associations with sentiment and improves classification performance in a majority of cases. The proposed feature selection method is particularly effective when applied to out-of-domain data.

Many applications require semantic understanding of short texts, and inferring discriminative and coherent latent topics is a critical and fundamental task in these applications. Conventional topic models largely rely on word co-occurrences to derive topics from a collection of documents. However, due to the length of each document, short texts are much more sparse in terms of word co-occurrences. Recent studies show that the Dirichlet Multinomial Mixture (DMM) model is effective for topic inference over short texts by assuming that each piece of short text is generated by a single topic. However, DMM has two main limitations. First, even though it seems reasonable to assume that each short text has only one topic because of its shortness, the definition of “shortness” is subjective and the length of the short texts is dataset dependent. That is, the single-topic assumption may be too strong for some datasets. To address this limitation, we propose to model the topic number as a Poisson distribution, allowing each short text to be associated with a small number of topics (e.g., one to three topics). This model is named PDMM. Second, DMM (and also PDMM) does not have access to background knowledge (e.g., semantic relations between words) when modeling short texts. When a human being interprets a piece of short text, the understanding is not solely based on its content words, but also their semantic relations. Recent advances in word embeddings offer effective learning of word semantic relations from a large corpus. Such auxiliary word embeddings enable us to address the second limitation. To this end, we propose to promote the semantically related words under the same topic during the sampling process, by using the generalized Pólya urn (GPU) model. Through the GPU model, background knowledge about word semantic relations learned from millions of external documents can be easily exploited to improve topic modeling for short texts. By directly extending the PDMM model with the GPU model, we propose two more effective topic models for short texts, named GPU-DMM and GPU-PDMM. Through extensive experiments on two real-world short text collections in two languages, we demonstrate that PDMM achieves better topic representations than state-of-the-art models, measured by topic coherence. The learned topic representation leads to better accuracy in a text classification task, as an indirect evaluation. Both GPU-DMM and GPU-PDMM further improve topic coherence and text classification accuracy. GPU-PDMM outperforms GPU-DMM at the price of higher computational costs.