TopicStriKer: A topic kernels-powered approach for text classification

The geographical identification of content in Social Networks have enabled to bridge the gap between online social platforms and the physical world. Although vast amounts of data in such networks are due to breaking news or global occurrences, local events witnessed by users in situ are also present in these streams and of great importance for many city entities. Nowadays, unsupervised machine learning techniques, such as Tweet-SCAN, are able to retrospectively detect these local events from tweets. However, these approaches have limited abilities to reason about unseen observations in a principled way due to the lack of a proper probabilistic foundation. Probabilistic models have also been proposed for the task, but their event identification capabilities are far from those of Tweet-SCAN. In this paper, we identify two key factors which, when combined, boost the accuracy of such models. As a first key factor, we notice that the large amount of meaningless social data requires explicitly modeling non-event observations.Therefore, we propose to incorporate a background model that captures spatio-temporal fluctuations of non-event tweets. As a second key factor, we observe that the shortness of tweets hampers the application of traditional topic models. Thus, we integrate event detection and topic modeling, assigning topic proportions to events instead of assigning them to individual tweets. As a result, we propose Warble, a new probabilistic model and learning scheme for retrospective event detection that incorporates these two key factors. We evaluate Warble in a data set of tweets located in Barcelona during its festivities. The empirical results show that the model outperforms other state-of-the-art techniques in detecting various types of events while relying on a principled probabilistic framework that enables to reason under uncertainty.

The demand for wearable devices that can detect anxiety and stress when driving is increasing. Recent studies have attempted to use multiple biosignals to detect driving stress. However, collecting multiple biosignals can be complex and is associated with numerous challenges. Determining the optimal biosignal for assessing driving stress can save lives. To the best of our knowledge, no study has investigated both longitudinal and transitional stress assessment using supervised and unsupervised ML techniques. Thus, this study hypothesizes that the optimal signal for assessing driving stress will consistently detect stress using supervised and unsupervised machine learning (ML) techniques. Two different approaches were used to assess driving stress: longitudinal (a combined repeated measurement of the same biosignals over three driving states) and transitional (switching from state to state such as city to highway driving). The longitudinal analysis did not involve a feature extraction phase while the transitional analysis involved a feature extraction phase. The longitudinal analysis consists of a novel interaction ensemble (INTENSE) that aggregates three unsupervised ML approaches: interaction principal component analysis, connectivity-based clustering, and K-means clustering. INTENSE was developed to uncover new knowledge by revealing the strongest correlation between the biosignal and driving stress marker. These three MLs each have their well-known and distinctive geometrical basis. Thus, the aggregation of their result would provide a more robust examination of the simultaneous non-causal associations between six biosignals: electrocardiogram (ECG), electromyogram, hand galvanic skin resistance, foot galvanic skin resistance, heart rate, respiration, and the driving stress marker. INTENSE indicates that ECG is highly correlated with the driving stress marker. The supervised ML algorithms confirmed that ECG is the most informative biosignal for detecting driving stress, with an overall accuracy of 75.02%.

Examining the effectiveness of machine learning techniques in analyzing engineering students’ decision-making processes through topic modeling during simulation-based design tasks is crucial for advancing educational methods and tools. Thus, this study presents a comparative analysis of different supervised and unsupervised machine learning techniques for topic modeling, along with human validation. Hence, this manuscript contributes by evaluating the effectiveness of these techniques in identifying nuanced topics within the argumentation framework and improving computational methods for assessing students’ abilities and performance levels based on their informed decisions. This study examined the decision-making processes of engineering students as they participated in a simulation-based design challenge. During this task, students were prompted to use an argumentation framework to articulate their claims, evidence, and reasoning, by recording their informed design decisions in a design journal. This study combined qualitative and computational methods to analyze the students’ design journals and ensured the accuracy of the findings through the researchers’ review and interpretations of the results. Different machine learning models, including random forest, SVM, and K-nearest neighbors (KNNs), were tested for multilabel regression, using preprocessing techniques such as TF-IDF, GloVe, and BERT embeddings. Additionally, hyperparameter optimization and model interpretability were explored, along with models like RNNs with LSTM, XGBoost, and LightGBM. The results demonstrate that both supervised and unsupervised machine learning models effectively identified nuanced topics within the argumentation framework used during the design challenge of designing a zero-energy home for a Midwestern city using a CAD/CAE simulation platform. Notably, XGBoost exhibited superior predictive accuracy in estimating topic proportions, highlighting its potential for broader application in engineering education.

Email has become one of the best ever and most efficient forms of communication. However, there are many challenges of grouping email messages. One of the challenging issues of email grouping is to identify when the clustering process accumulates accurate and sufficient information for grouping to be determined. In this work, an unsupervised machine learning technique has been developed based on unsupervised clustering method (UCM) with email vector space model (EVSM). The proposed EVSM is a novel approach as it groups emails in conjunction with UCM. Nowadays, a typical email user receives about 20-40 email messages per day. For some people depending on their job function, hundreds of messages are usual. Thus, users spend a significant part of their working time on browsing through emails. As the popularity of email communication is growing, the time spent on reading and replying emails will only increase which could lead to email overload and unstructured mailbox. As a result, there has recently been a growing interest in creating automated system to help users manage their mailbox extensively. Examples of such systems are PEA [1], MailCat [2], Re:Agent [3]. Moreover, the main tool for email management is text classification [4, 5]. EVSM is a system that implements self learning approach by The creating a space in which both email messages and queries are represented by vectors to automatically create email message groups into one or more discrete set of predefined categories. The key of using unsupervised clustering method (UCM) to cluster emails in combination with email vector space model (EVSM) will improve the performance of the email groupings being generated. Another problem that needs to be solved is the word and phrase similarity in different email messages. An email naturally has no label indicating the group it can belong to. The label can only be set by humans for accuracy, which needs heavy manual labor. This challenging issue is especially significant for email users who need to be labeling email messages frequently in order to manage and organize their inbox. 2. Email Vector Space Model (EVSM) This work developed a novel approach that uses both the structure and content of email messages for email grouping. Since emails composes of a structure in the form of headers and the message body, they can be represented using feature selection and the relationships between various terms (e.g., the occurrence of a term in the subject or body of the message etc) can be represented in the form of a cluster. EVSM has been developed as a viable technique for pattern extraction, frequent words and phrase selections. This approach is based on the premise that representative – common and recurring – structures/patterns/features can be extracted from a pre-grouped email dataset and the same can be used effectively for grouping incoming emails. It suggest that an email dataset consists of representative emails or knowledge sets which contains typically a set of keywords representing a specific meaning, and the structure and content of these representative emails or knowledge sets can be extracted by adapting unsupervised email vector space model techniques to work with the developed UCM. The work also, hypothesise that the notion of clustering is critical for this to work as it helps in grouping similar structures within emails instead of looking for exact/identical matches that may be difficult to find in the email messages.

ML-Diabetes is a machine learning-based predictive model for the early detection of diabetes. Diabetes is a chronic metabolic disorder that affects millions of people worldwide. Early detection of diabetes can help prevent its complications and improve patient outcomes. ML-Diabetes is designed to use demographic and clinical data to predict the likelihood of a patient developing diabetes. The model uses a combination of supervised and unsupervised machine learning techniques to analyse and classify data. ML-Diabetes uses a dataset containing demographic and clinical information of patients, including age, sex, BMI, blood pressure, and glucose levels. The dataset is preprocessed and cleaned to remove missing values and outliers. The processed data is then split into training and testing sets, and the model is trained on the training set. The model uses a combination of supervised and unsupervised machine learning techniques, including logistic regression, decision trees, and k-means clustering, to predict the likelihood of a patient developing diabetes. The model is evaluated on the testing set using various performance metrics, including accuracy, precision, recall, and F1-score. The results show that ML-Diabetes is a reliable and accurate predictive model for the early detection of diabetes. The model achieves an accuracy of 85%, precision of 90%, recall of 80%, and F1-score of 85%. The model can be used by healthcare professionals to screen patients for diabetes and provide early interventions to prevent complications.

Operational modal analysis (OMA) is required to maintain large-scale and necessary civil infrastructures. The non-contact method of OMA using digital image correlation and point tracking algorithms requires a speckle pattern placed on the structure. Alternatively, advanced computer vision methods like optical flow and phase-based video motion magnification (PBVMM) techniques are used to measure modal parameters. Despite the importance of PBVMM, the users should know the range of frequencies in which the natural structure frequency lies. A methodology based on an unsupervised machine learning technique is developed to extract the modal parameters blindly from its recorded digital video. The proposed methodology uses complex steerable pyramids and an unsupervised machine learning technique, also known as principal component analysis, and analytical mode decomposition with a random decrement technique to blindly extract the modal parameters of a structure. This study validated the proposed methodology using a multi-degree of freedom (DOF) numerical model. The results are compared with theoretical and estimated values and are in good agreement. Furthermore, it is implemented on a laboratory benchmark SDOF, MDOF, and real-time videos of the London Millennium and Tacoma Narrows bridges for blindly extracting the modal frequencies and damping ratios.

The rapid development of large language models (LLMs) has opened up new opportunities for automated questionnaire design in academic research. However, questions remain about the validity and quality of AI-generated tools, especially when applied to theory-driven frameworks. The aim of this study is to assess the performance of AI-generated questionnaires and human-designed questionnaires in single and integrated theoretical models. To this end, we employ a hybrid approach that combines expert evaluation and unsupervised machine learning techniques. Specifically, we use latent dirichlet allocation (LDA) for topic modeling to assess semantic coverage. And, KMeans clustering was used to detect redundancy and assess semantic consistency. We created four questionnaires: two based on validated manual writing tools and two generated by the GPT-4. This covered the Unified Theory of Acceptance and Use of Technology (UTAUT) and its extended models. 310 The questionnaires assessed program quality on multiple dimensions. And, these judgments were objectively validated using machine learning outputs. Results show that AI-generated questionnaires are fluent and objective, but perform poorly in terms of accuracy, clarity, and comprehensiveness, especially under complex modeling conditions. Redundancy and semantic drift increased with theory integration. However, the AI performed well in areas requiring standardization and neutrality.

Customer relationship management (CRM) is the tool to enhance customer relationship in any business. Due to the exponential growth of data volume, in any field, it is significant to develop new techniques to discover the customer knowledge, automation of the system and moreover customer satisfaction to win customer lifetime value. CRM with machine learning could bring a catalytic change in business. Several supervised and unsupervised machine learning techniques are utilized to improve the customer experience and profitability of business. This paper reviews the available literature on the CRM with machine learning techniques for customer identification, customer attraction, and customer retention and customer development. This study reveals that supervised learning techniques are 48.48% utilized, unsupervised learning techniques are utilized 15.15%, and 9.09% utilized other techniques in CRM. Paradigm is also shifted toward the deep learning from machine learning as 28.28% text has been reported to deep learning. Decision tree-based algorithm and support vector machine algorithms are most utilized algorithm of supervised learning. E-commerce and telecommunication sectors are the most important areas identified with the exponential growth of the users and hence need a suitable machine learning techniques for customer satisfaction and business profitability.

A massive of text that is generated every minute is increasing dramatically. Therefore, it is more and more important to find an effective model to automatic classify the amount of text. Topic models is the most powerful techniques in text classification. There are many research results in the field of topic model have been published in scholarly journals. The Latent Dirichlet Allocation (LDA) is one of the most popular topic models in text classification. Researchers have proposed many topic evolution models based on LDA to solve some specific problems in applications of text classification. And some joint models which based on topic models combined other algorithms have been studied to enhance the performance of text classification. In this paper, we investigated three categories topic models for text classification and briefly introduced their advantages and disadvantages in the applications of text mining. Also, we introduce the generated process of documents and illustrate the graphical model for each topic models.

In rarefied gas flows, discontinuity phenomena such as velocity slip and temperature jump commonly appear in the gas layer adjacent to a solid boundary. Due to the physical complexity of the interactions at the gas-solid interface, particularly in the case of systems with local nonequilibrium state, boundary models with limited number of parameters cannot completely describe the reflection of gas molecules at the boundary. In this work, the Gaussian mixture (GM) model, which is an unsupervised machine learning technique, is employed to construct a statistical gas-solid surface scattering model based on the collisional data obtained from molecular dynamics (MD) simulations. The GM model is applied to study Couette flow for different inert gases (Ar and He) confined between two parallel infinite gold walls at different temperatures. A direct comparison between the results obtained from the GM model and the Cercignani-Lampis-Lord (CLL) scattering kernel against the MD collisional data in terms of the distribution of the predicted postcollisional velocities, and accommodation coefficients has shown that the results from the GM model are an excellent match with the MD results outperforming the CLL scattering kernel. As an example, for He gas, while the predicted energy accommodation coefficient by the CLL model is more than two times higher than the MD predictions, the value computed by the GM model is in excellent agreement with the MD results. This superior performance of the GM model confirms its high potential to derive a generalized boundary condition in systems encountered with highly nonequilibrium and complex gas flow conditions.

Technological developments have affected the decision-making phase in football matches. The viewing pleasure of fans and the result of the match is a matter of debate determined by the referees. Throughout time, the objectiveness of referees was questioned by the technological tools. Video Assistant Referee (VAR) is an example approach to ensure whether the referees react to similar positions of different matches in the same manner or not. One of the problems of this topic is the controversial decisions of some referees leading to unexpected results. In this research, as a different approach to the referees’ objectivity problem, referees are tried to be classified based on the statistical outcome of the matches using unsupervised machine learning techniques. Meaningful clusters should not be found to be able to state the referees are objective. This study is conducted on 10-years English Premier League between 2009–2018 data. Principal Component Analysis is going to be used for grouping the variables to perform exploratory data analysis. K-Means, hierarchical clustering, and Fuzzy C-Means are going to be used for dividing the referees into various subgroups. R programming language is used for examining data. In conclusion of the analysis, four different referee groups are defined.KeywordsFootballEnglish Premier LeagueRefereeClusteringMachine learningUnsupervised learning

Who cares about coal? Analyzing 70 years of German parliamentary debates on coal with dynamic topic modeling

Online discussion forums play an important role in building and sharing domain knowledge. An extensive amount of information can be found in online forums, covering every aspect of life and professional discourse. This chapter introduces the application of supervised and unsupervised machine learning techniques to analyze forum questions. This chapter starts with supervised machine learning techniques to classify forum posts into pre-defined topic categories. As a supporting technique, web scraping is also discussed to gather data from an online forum. After this, this chapter introduces unsupervised learning techniques to identify latent topics in documents. The combination of supervised and unsupervised machine learning approaches offers us deeper insights of the data obtained from online forums. This chapter demonstrates these techniques through a case study on a very large online discussion forum called LabVIEW from the systems modeling community. In the end, the authors list future trends in applying machine learning to understand the expertise captured in online expert communities.

For many text classification tasks, there is a major problem posed by the lack of labeled data in a target domain. Although classifiers for a target domain can be trained on labeled text data from a related source domain, the accuracy of such classifiers is usually lower in the cross-domain setting. Recently, string kernels have obtained state-of-the-art results in various text classification tasks such as native language identification or automatic essay scoring. Moreover, classifiers based on string kernels have been found to be robust to the distribution gap between different domains. In this paper, we formally describe an algorithm composed of two simple yet effective transductive learning approaches to further improve the results of string kernels in cross-domain settings. By adapting string kernels to the test set without using the ground-truth test labels, we report significantly better accuracy rates in cross-domain English polarity classification.

BackgroundTopic models are a class of unsupervised machine learning models, which facilitate summarization, browsing and retrieval from large unstructured document collections. This study reviews several methods for assessing the quality of unsupervised topic models estimated using non-negative matrix factorization. Techniques for topic model validation have been developed across disparate fields. We synthesize this literature, discuss the advantages and disadvantages of different techniques for topic model validation, and illustrate their usefulness for guiding model selection on a large clinical text corpus.Design, setting and dataUsing a retrospective cohort design, we curated a text corpus containing 382,666 clinical notes collected between 01/01/2017 through 12/31/2020 from primary care electronic medical records in Toronto Canada.MethodsSeveral topic model quality metrics have been proposed to assess different aspects of model fit. We explored the following metrics: reconstruction error, topic coherence, rank biased overlap, Kendall’s weighted tau, partition coefficient, partition entropy and the Xie-Beni statistic. Depending on context, cross-validation and/or bootstrap stability analysis were used to estimate these metrics on our corpus.ResultsCross-validated reconstruction error favored large topic models (K ≥ 100 topics) on our corpus. Stability analysis using topic coherence and the Xie-Beni statistic also favored large models (K = 100 topics). Rank biased overlap and Kendall’s weighted tau favored small models (K = 5 topics). Few model evaluation metrics suggested mid-sized topic models (25 ≤ K ≤ 75) as being optimal. However, human judgement suggested that mid-sized topic models produced expressive low-dimensional summarizations of the corpus.ConclusionsTopic model quality indices are transparent quantitative tools for guiding model selection and evaluation. Our empirical illustration demonstrated that different topic model quality indices favor models of different complexity; and may not select models aligning with human judgment. This suggests that different metrics capture different aspects of model goodness of fit. A combination of topic model quality indices, coupled with human validation, may be useful in appraising unsupervised topic models.