A Competency Mining Method Based on Latent Dirichlet Allocation (LDA) Model

This study aimed to assess the different needs of patients with breast cancer and their families in online health communities at different treatment phases using a Latent Dirichlet Allocation (LDA) model. Using Python, breast cancer-related posts were collected from two online health communities: patient-to-patient and patient-to-doctor. After data cleaning, eligible posts were categorized based on the treatment phase. Subsequently, an LDA model identifying the distinct need-related topics for each phase of treatment, including data preprocessing and LDA topic modeling, was established. Additionally, the demographic and interactive features of the posts were manually analyzed. We collected 84,043 posts, of which 9504 posts were included after data cleaning. Early diagnosis and rehabilitation treatment phases had the highest and lowest number of posts, respectively. LDA identified 11 topics: three in the initial diagnosis phase and two in each of the remaining treatment phases. The topics included disease outcomes, diagnosis analysis, treatment information, and emotional support in the initial diagnosis phase; surgical options and outcomes, postoperative care, and treatment planning in the perioperative treatment phase; treatment options and costs, side effects management, and disease prognosis assessment in the non-operative treatment phase; diagnosis and treatment options, disease prognosis, and emotional support in the relapse and metastasis treatment phase; and follow-up and recurrence concerns, physical symptoms, and lifestyle adjustments in the rehabilitation treatment phase. The needs of patients with breast cancer and their families differ across various phases of cancer therapy. Therefore, specific information or emotional assistance should be tailored to each phase of treatment based on the unique needs of patients and their families.

The study aims to investigate the integration of artificial intelligence technology into Weibo sentiment analysis, aiming to enhance the effectiveness of Weibo in human-computer interaction education. Initially, the Weibo sentiment dictionary is created, and a conventional model for sentiment analysis of user-forwarded Weibo is introduced, specifically the Latent Dirichlet Allocation (LDA) model. Then, the deep learning models in the field of artificial intelligence, namely, the convolutional neural network (CNN) model and the long short-term memory network (LSTM) model, are proposed. Tencent Weibo data set is obtained through Application Programming Interface (API) crawler. The experimental environment of the deep learning model is analyzed and the data set is preprocessed. The results show that when the number of topics is 120, the relative maximum value of F1 is 69.92% and 69.96% with and without the introduction of emotional features in the LDA model, respectively. The accuracy of CNN model and LSTM model is 0.793 and 0.849, respectively. In the three cases of user characteristics, user characteristics + Weibo features, and use characteristics + Weibo features + relationship characteristics, the polarity of the forwarded comments of the LDA model doesn’t change much. In conclusion, the LDA model demonstrates universality and accuracy in sentiment analysis of user-forwarded Weibo, while LSTM proves to be more suitable for sentiment classification in this context. Leveraging the LDA deep learning model, LSTM effectively analyzes the sentiment of users forwarding Weibo. These findings serve as an experimental foundation for the efficient integration of Weibo in human-computer interaction education.

A questionnaire text mining model is built based on latent semantic analysis (LSA). Take open questionnaire as the research object and improve Latent Dirichlet Allocation (LDA) model according to the text characteristics, a BM25-LDA model is built. By comparing the LDA model and BM25-LDA model on the open questionnaire text, we can verify that the performance of BM25-LDA mode is better than that of original LDA model.

Understanding and predicting the effect of global change phenomena on biodiversity is challenging given that biodiversity data are highly multivariate, containing information from tens to hundreds of species in any given location and time. The Latent Dirichlet Allocation (LDA) model has been recently proposed to decompose biodiversity data into latent communities. While LDA is a very useful exploratory tool and overcomes several limitations of earlier methods, it has limited inferential and predictive skill given that covariates cannot be included in the model. We introduce a modified LDA model (called LDAcov) which allows the incorporation of covariates, enabling inference on the drivers of change of latent communities, spatial interpolation of results, and prediction based on future environmental change scenarios. We show with simulated data that our approach to fitting LDAcov is able to estimate well the number of groups and all model parameters. We illustrate LDAcov using data from two experimental studies on the long‐term effects of fire on southeastern Amazonian forests in Brazil. Our results reveal that repeated fires can have a strong impact on plant assemblages, particularly if fuel is allowed to build up between consecutive fires. The effect of fire is exacerbated as distance to the edge of the forest decreases, with small‐sized species and species with thin bark being impacted the most. These results highlight the compounding impacts of multiple fire events and fragmentation, a scenario commonly found across the southern edge of Amazon. We believe that LDAcov will be of wide interest to scientists studying the effect of global change phenomena on biodiversity using high‐dimensional datasets. Thus, we developed the R package LDAcov to enable the straightforward use of this model.

How to recognize human action from videos captured by modern cameras efficiently and effectively is a challenge in real applications. Traditional methods which need professional analysts are facing a bottleneck because of their shortcomings. To cope with the disadvantage, methods based on computer vision techniques, without or with only a few human interventions, have been proposed to analyse human actions in videos automatically. This paper provides a method combining the three dimensional Scale Invariant Feature Transform (SIFT) detector and the Latent Dirichlet Allocation (LDA) model for human motion analysis. To represent videos effectively and robustly, we extract the 3D SIFT descriptor around each interest point, which is sampled densely from 3D Space-time video volumes. After obtaining the representation of each video frame, the LDA model is adopted to discover the underlying structure-the categorization of human actions in the collection of videos. Public available standard datasets are used to test our method. The concluding part discusses the research challenges and future directions.

Objective: To analyze the literature on rational drug use in China from 2021 to June 2024, sort out the recent situation of rational drug use in China, and find out the potential problems of rational drug use in China, in order to further promote the standardized management of rational drug use. Methods: Based on the literature on rational drug use in CNKI, the title, abstract, keyword and discussion were taken as the research objects, the LDA model was used for text mining, and 304 articles were included after removing irrelevant literature, and 6 topics were mined by using the latent Dirichlet allocation (LDA)model, and the theme research trends were found for visual display. Results: By analyzing and summarizing relevant data, it provided a reference for optimizing rational drug use strategies.

Latent Dirichlet allocation (LDA) is a statistical model that is often used to discover topics or themes in a large collection of documents. In the LDA model, topics are modeled as discrete distributions over a finite vocabulary of words. The LDA is also a popular choice to model other datasets spanning a discrete domain, such as population genetics and social networks. However, in order to model data spanning a continuous domain with the LDA, discrete approximations of the data need to be made. These discrete approximations to continuous data can lead to loss of information and may not represent the true structure of the underlying data. We present an augmented version of the LDA topic model, where topics are represented using Gaussian mixture models (GMMs), which are multi-modal distributions spanning a continuous domain. This augmentation of the LDA topic model with Gaussian mixture topics is denoted by the GMM-LDA model. We use Gibbs sampling to infer model parameters. We demonstrate the utility of the GMM-LDA model by applying it to the problem of clustering sleep states in electroencephalography (EEG) data. Results are presented demonstrating superior clustering performance with our GMM-LDA algorithm compared to the standard LDA and other clustering algorithms.

To improve hydro-chemical modeling and forecasting, there is a need to better understand flood-induced variability in water chemistry and the processes controlling it in watersheds. In the literature, assumptions are often made, for instance, that stream chemistry reacts differently to rainfall events depending on the season; however, methods to verify such assumptions are not well developed. Often, few floods are studied at a time and chemicals are used as tracers. Grouping similar events from large multivariate datasets using principal component analysis and clustering methods helps to explain hydrological processes; however, these methods currently have some limits (definition of flood descriptors, linear assumption, for instance). Most clustering methods have been used in the context of regionalization, focusing more on mapping results than on understanding processes. In this study, we extracted flood patterns using the probabilistic Latent Dirichlet Allocation (LDA) model, its first use in hydrology, to our knowledge. The LDA method allows multivariate temporal datasets to be considered without having to define explanatory factors beforehand or select representative floods. We analyzed a multivariate dataset from a long-term observatory (Kervidy-Naizin, western France) containing data for four solutes monitored daily for 12 years: nitrate, chloride, dissolved organic carbon, and sulfate. The LDA method extracted four different patterns that were distributed by season. Each pattern can be explained by seasonal hydrological processes. Hydro-meteorological parameters help explain the processes leading to these patterns, which increases understanding of flood-induced variability in water quality. Thus, the LDA method appears useful for analyzing long-term datasets.

Remote sensing images are primary data sources for land use classification. High spatial resolution images enable more accurate analysis and identification of land cover types. However, a higher spatial resolution also brings new challenges to the existing classification methods. In the low-level feature spaces of remote sensing images, it is difficult to improve classification performance by modifying classifiers. Probabilistic topic models can connect low-level features and high-level semantics of remote sensing images. Latent Dirichlet allocation (LDA) models are representatives of probabilistic topic models. However, at present, probabilistic topic models are mainly adopted for scene classification and image retrieval in remote sensing image analysis only. In this study, multiscale segmentation was employed to construct bag-of-words (BoW) representations of high-resolution images. The segmented patches were then utilized as “image documents.” A structural topic model was used with an LDA model to import spatial information from the image documents at two levels: topical prevalence and topical content in the form of covariates. In this way, latent topic features in image documents can be more accurately deduced. The proposed method showed more satisfactory classification performance than standard LDA models and demonstrated a certain degree of robustness against the changes in the segmentation scale. Acknowledgement for the data support from “Yangtze River Delta Science Data Center, National Earth System Science Data Center, National Science & Technology Infrastructure of China ( http://nnu.geodata.cn:8008 )”.

Scene classification from images is a challenging problem in computer vision due to its significant variability of scale, illumination, and view. Recently, Latent Dirichlet Allocation (LDA) model has grown popular in computer vision field, especially in scene labeling and classification. However, the effectiveness of the LDA model for the scene classification has not yet been addressed thoroughly. Especially, there is little experimental evaluation on the model's performance for different types of features. Fusion of multiple types of features is usually necessary in the scene classification due to the complexity of scene images. In this paper, we investigate the effectiveness of the LDA model in scene classification by using 7 types of features (i.e. uniform grid based interest points, Harris corner based interest points, scale invariant feature transform (SIFT), texture, shape, color, and location) and their various combinations. Furthermore, we compare the performance of the LDA model with Support Vector Machine (SVM) classifier. All experiments are performed on the UIUC Sport Scene database. The experiments demonstrate that the performance of the LDA model 1) is significantly lower than the SVM classifier for the scene classification over different types of features; and 2) decreases by fusing multiple features while improvement shown in SVM classifier.

Latent Dirichlet Allocation (LDA) model, a formal generative model, has been used to improve ad-hoc information retrieval recently. However, its feasibility and effectiveness for geographic information retrieval has not been explored. This paper proposes an LDA-based document model to improve geographic information retrieval by inheriting the LDA model with text retrieval model. The proposed model has been evaluated on GeoCLEF2007 collection. This is a part of the experiments of Columbus Project of Microsoft Research Asia (MSRA) in GeoCLEF2007 (a cross-language geographical retrieval track which is part of Cross Language Evaluation Forum). This is the second time we participate in this event. Since the queries in GeoCLEF2007 are similar to those in GeoCLEF2006, we leverage most of the methods that we used in GeoCLEF2006, including MSRAWhitelist, MSRAExpansion, MSRALocation and MSRAText approaches. The difference is that MSRAManual approach is not included this time, and we use MSRALDA instead. The results show that the application of LDA model in GeoCLEF monolingual English task performs stably but needs to be further explored.

Under the goal of carbon peak and carbon neutrality, developing battery electric vehicles (BEVs) is an important way to reduce carbon emissions in the transportation sector. To popularize BEVs as soon as possible, it is necessary to study selection strategies for BEVs from the perspective of consumers. Therefore, the Latent Dirichlet Allocation (LDA) model based on fine-grained sentiment analysis is combined with the multi-criteria decision-making (MCDM) model to assess ten types of BEV alternatives. Fine-grained sentiment analysis is applied to find the vehicle attributes that consumers care about the most based on the word-of-mouth data. The LDA model is suggested to divide topics and construct the indicator system. The MCDM model is used to rank vehicles and put forward the corresponding optimization path to increase consumer purchases of BEVs in China. The results show that (a) via the LDA model based on fine-grained sentiment analysis, attributes that consumers care most about are divided into five topics: dynamics, technology, safety, comfort, and cost; (b) based on the DEMATEL technique, the dimensions in the order of importance are as follows: safety, technology, dynamics, comfort, and cost; (c) the price is the most important criteria that affect customers’ satisfaction by the DANP model; and (d) based on the VIKOR model, the selection strategies present that Aion S is highlighted as the best choice, and the optimization path is discussed to promote the performance of BEVs to increase customers’ satisfaction. The findings can provide a reference for improving the sustainable development of the automobile industry in China. The proposed framework serves as the basis for further discussion of BEVs.

We present a new way called Persona Analysis with Text Topic Modelling (PATTM), which tries to learn the role of personae according to the literal descriptions. It is similar to Latent Dirichlet Allocation (LDA) and Author Topic (AT) model, with the attribute of allowing all text to join in the topic modelling process, even when there is no persona in the text. We experiment on the “Libya Event” data set which contains more than 4,000 texts collected from the Internet. The PATTM gives lower perplexity than LDA and AT model on the data set.

Two new extensions of latent Dirichlet allocation (LDA), denoted topic-supervised LDA (ts-LDA) and class-specific-simplex LDA (css-LDA), are proposed for image classification. An analysis of the supervised LDA models currently used for this task shows that the impact of class information on the topics discovered by these models is very weak in general. This implies that the discovered topics are driven by general image regularities, rather than the semantic regularities of interest for classification. To address this, ts-LDA models are introduced which replace the automated topic discovery of LDA with specified topics, identical to the classes of interest for classification. While this results in improvements in classification accuracy over existing LDA models, it compromises the ability of LDA to discover unanticipated structure of interest. This limitation is addressed by the introduction of css-LDA, an LDA model with class supervision at the level of image features. In css-LDA topics are discovered per class, i.e., a single set of topics shared across classes is replaced by multiple class-specific topic sets. The css-LDA model is shown to combine the labeling strength of topic-supervision with the flexibility of topic-discovery. Its effectiveness is demonstrated through an extensive experimental evaluation, involving multiple benchmark datasets, where it is shown to outperform existing LDA-based image classification approaches.

Topic models have recently shown to be really useful tools for the analysis of microarray experiments. In particular they have been successfully applied to gene clustering and, very recently, also to samples classification. In this latter case, nevertheless, the basic assumption of functional independence between genes is limiting, since many other a priori information about genes’ interactions may be available (co-regulation, spatial proximity or other a priori knowledge). In this paper a novel topic model is proposed, which enriches and extends the Latent Dirichlet Allocation (LDA) model by integrating such dependencies, encoded in a categorization of genes. The proposed topic model is used to derive a highly informative and discriminant representation for microarray experiments. Its usefulness, in comparison with standard topic models, has been demonstrated in two different classification tests.KeywordsTopic ModelLatent Dirichlet AllocationMultinomial DistributionProbabilistic Latent Semantic AnalysisLatent Dirichlet Allocation ModelThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.