Optimizing People Sourcing Through Semantic Matching of Job Description Documents and Candidate Profile Using Improved Topic Modelling Techniques

Topic modeling is the statistical model for discovering hidden topics or keywords in a collection of documents. Topic modeling is also considered a probabilistic model for learning, analyzing, and discovering topics from the document collection. The most popular techniques for topic modeling are latent semantic analysis (LSA), probabilistic latent semantic analysis (pLSA), latent Dirichlet allocation (LDA), and the recent deep learning-based lda2vec. LDA is most commonly used in extractive multi-document summarization to determine whether the extracted sentence reflects the concept of the input document. In this paper, we will try to explore various multi-document summarization techniques that use LDA as a topic modeling method for improving final summary coverage and to reduce redundancy. Finally, we compared LDA and LSA using the Genism toolkit, and our experiment results show that LDA outperforms LSA if we increase the number of features considered for sentence selection.KeywordsTopic modelingText summarizationLDALSA

Stack Overflow has undoubtedly become a big trend among the programmers and researchers. The problems or doubts faced by any programmer are put on this forum making it available to many knowledgeable people and hence giving out possibly correct solutions. This paper focuses on Tag based Answer Recommendation System on Stack Overflow. This model follows Topic Modeling methodology wherein two techniques were attempted, i.e., LSA(Latent semantic analysis) and LDA(Latent Dirichlet Allocation). These methods deal with various User Queries to find out the most similar questions from the database and hence, deliver the most valid answers related to the question as output. The results from both the Topic Modeling techniques were analyzed with the help of Cosine Similarity as metrics and hence, we found out that LDA performed better than LSA, thereby giving better and more accurate results.

Due to the rapid advancement in social media and technology, it generates a large amount of data in different areas of applications. Social media analysis and text mining are all about collecting the most valuable data and drawing actionable conclusions. Text mining also referred to as data mining it is which contains various nodes in the form of data which is often linked together to form a pattern. High-quality information is typically derived through the devising of patterns and trends through means such as statistical pattern learning. In this study we have analyzed and mounted social media data from Twitter, new articles, and Reddit which suggest that domestic abuse is acting as an opportunistic infection, flourishing in the condition created by the pandemic. The computing tweet sentiments of domestic violence amongst various social media platforms is a major factor of concern. We have used several topic modeling techniques such as Latent Semantic Analysis (LSA) uses a bag of words model, Hierarchical Dirichlet Process (HDP) is a nonparametric Bayesian model for clustering problems, and Latent Dirichlet Allocation (LDA) is a generative probabilistic model for collections of discrete data. Therefore, in this project, we tend to propose a deeper insight into the rise in domestic violence on social media and to provide a holistic approach to tackle this situation.

Text analysis is an important component in the evolution of recommender systems, as it enables meaningful information to be extracted from vast amounts of textual data. This study performs a comparative analysis of two main topic modeling techniques, Latent Dirichlet Allocation (LDA) and BERTopic in the context of news recommender systems. Using a dataset of Moroccan news articles, we evaluate the ability of these models to generate coherent and interpretable topics. Our results demonstrate that BERTopic outperforms LDA in terms of topic consistency and semantic richness, particularly when applied to large and diverse datasets. Moreover, the integration of BERTopic into a content-based recommender system results in improved relevance and accuracy of recommendations. This research highlights the potential of advanced topic modeling techniques for refining recommender systems and suggests future directions, including the exploration of hybrid models and the integration of sentiment analysis for further improvements.

The holy Quran is one of the Holy Books of God. It is considered one of the main references for an estimated 1.6 billion of Muslims around the world. The Holy Quran language is Arabic. Specialized as well as non-specialized people in religion need to search and lookup certain information from the Holy Quran. Most research projects concentrate on the translation of the holy Quran in different languages. Nevertheless, few research projects pay attention to original text of the holy Quran in Arabic language. Keyword search is one of the Information Retrieval (IR) methods but will retrieve what is called exact search. Semantic search aims at finding deeper meanings of a text, and it is a hot field of study in Natural Language Processing (NLP). In this paper topic modeling techniques are explored to setup a framework for semantic search in the holy Quran. As the Holy Quran is the word of God, its meanings are unlimited. In this paper the words of chapter Joseph (Peace Be Upon Him (PBUH)) from the Holy Quran is analyzed based on topic modeling techniques as a case study. Latent Dirichlet Allocation (LDA) topic modeling technique has been applied in this paper into two structures (Hizb Quarters and verses) of Joseph chapter as: words, roots and stems. The log-Likelihood has been calculated for the two structures of the chapter. Results show that the best structure to use is verses, which gives the least energy for data. Some of the results of the attained topics are shown. These results suggest that topic modeling techniques failed to capture in an accurate manner the coherent topics of the chapter.

This study examines emerging topics and trends in Artificial Intelligence (AI)-enabled healthcare services, with a particular emphasis on service quality, diagnosis, and treatment enhancement due to the emergence of technology. This study aimed to identify the primary themes found in scholarly publications and examine how those themes have significance in the healthcare sector. Bibliometric and topic modelling techniques were used to analyse the extracted set of relevant publications. For bibliometric analysis, the bibliometric package Biblioshiny was used in R version 4.4.1, and Latent Dirichlet Allocation (LDA) was utilised for topic modelling. Academic papers from Scopus are included in the dataset, covering the period from 2011 to 2024. The research analysis highlights that there is a substantial emphasis placed on the use of artificial intelligence in the healthcare industry, notably in the areas of improving diagnosis, treatment personalisation, and operational efficiency related to services provided in the healthcare sector.

This project explores the application of sophisticated topic models to determine the ideal amount of subjects and evaluate them across several datasets pertaining to trends and current events. We propose a comprehensive approach that leverages the topic models such as Latent Dirichlet Allocation (LDA), Non-negative Matrix Factorization (NMF), Latent Semantic Analysis (LSA), Probabilistic Latent Dirichlet Allocation (PLDA) and Probabilistic Latent Semantic Indexing (PLSI) to accurately identify andanalyse topics. Our approach utilizes various topic modelling techniques to enhance the robustness and precision of topic detection. The results demonstrate the effectiveness of topic models in capturing nuanced trends and themes within complex datasets. A user-friendly frontend, built with Flask and SQLite, enables interactive input and visualization of topic modeling results, offering an accessible platformfor real-time topic analysis and insights. Thismethodology provides a powerful tool for understanding and responding to emerging trends in dynamic datasets

Continuous social text streams, such as tweets, provide a timeline of discussions. Topic modeling techniques such as Latent Dirichlet Allocation (LDA) have been used to extract the topics being discussed on social media streams. Recently, Online LDA has been proposed as a fast alternative for topic extraction, based on on-line stochastic optimization, while sentiment analysis is often used to track the polarity of posts. In this paper, we propose an online technique, integrating Online LDA and sentiment analysis to extract more refined polarity-aware topics within an online learning framework from continuous Twitter streams.

This study analyzes 926 articles related to the Beijing-Tianjin-Hebei (Jingjinji) urban agglomeration published in the People's Daily from 2014 to 2024 using topic modeling techniques. By applying the Latent Dirichlet Allocation (LDA) algorithm, six major topics were identified and the results were visualized through topic maps and word clouds. The analysis revealed that the main topics of Jingjinji integrated development are: CPC leadership and national governance, environmental management and disaster response, sports tourism and satellite navigation industry, Xiong'an New Area development and transportation infrastructure, fiscal policy and social security systems, and legal framework and public services. The chronological analysis showed a shift in focus from environmental problem-solving and institutional foundation building in the initial stage (2014-2015), to Xiong'an New Area development and transportation infrastructure in the middle stage (2016-2019), and finally to a comprehensive approach combining physical development with legal and institutional integration in recent years (2023-2024). This transformation demonstrates the policy evolution from ‘problem-solving’ to ‘active construction’ to ‘comprehensive integration.’ By systematically identifying the main agendas and changes in the development of the Jingjinji urban agglomeration through topic modeling, this study contributes to deepening the understanding of China's regional integration strategy.

Mining user reviews to discover what the user likes and dislikes is vital to understanding user behaviors. Topic modeling techniques have been extensively used to discover meaningful topics for user reviews and to discover user behaviors. Extracted topics may be a mixture of different concepts and hence very likely to be less coherent and unclear, especially when extracting a relatively small number of topics. As such, we propose a method that extracts a relatively large number of topics using a topic modeling technique and relies on hierarchical clustering to exploit semantic distances between topics, to generate a small number of highly coherent and clear topics. We also compare this set of topics representing hidden user concerns extracted by our approach with those derived using LDA (Latent Dirichlet Allocation) and a hierarchical variant called Pachinko Allocation Model (PAM) and show that our method generates more coherent user concerns. Further, we also demonstrate how a hierarchical model of user concerns can be automatically generated by exploiting our approach. Such a hierarchy may help capture the conceptual distances between various user concerns and inherent similarities between users having those concerns.

Integrating Big Data Into Evaluation: R Code for Topic Identification and Modeling

Latent Dirichlet Allocation (LDA) is a widely used machine learning technique in topic modeling and data analysis. Training large LDA models on big datasets involves dynamic and irregular computation patterns and is a major challenge to both algorithm optimization and system design. In this paper, we present a comprehensive benchmarking of our novel synchronized LDA training system HarpLDA+ based on Hadoop and Java. It demonstrates impressive performance when compared to three other MPI/C++ based state-of-the-art systems, which are LightLDA, F+NomadLDA, and WarpLDA. HarpLDA+ uses optimized collective communication with a timer control for load balance, leading to stable scalability in both shared-memory and distributed systems. We demonstrate in the experiments that HarpLDA+ is effective in reducing synchronization and communication overhead and outperforms the other three LDA training systems.

BackgroundBioinformatics is an interdisciplinary field at the intersection of molecular biology and computing technology. To characterize the field as convergent domain, researchers have used bibliometrics, augmented with text-mining techniques for content analysis. In previous studies, Latent Dirichlet Allocation (LDA) was the most representative topic modeling technique for identifying topic structure of subject areas. However, as opposed to revealing the topic structure in relation to metadata such as authors, publication date, and journals, LDA only displays the simple topic structure.MethodsIn this paper, we adopt the Tang et al.’s Author-Conference-Topic (ACT) model to study the field of bioinformatics from the perspective of keyphrases, authors, and journals. The ACT model is capable of incorporating the paper, author, and conference into the topic distribution simultaneously. To obtain more meaningful results, we use journals and keyphrases instead of conferences and bag-of-words.. For analysis, we use PubMed to collected forty-six bioinformatics journals from the MEDLINE database. We conducted time series topic analysis over four periods from 1996 to 2015 to further examine the interdisciplinary nature of bioinformatics.ResultsWe analyze the ACT Model results in each period. Additionally, for further integrated analysis, we conduct a time series analysis among the top-ranked keyphrases, journals, and authors according to their frequency. We also examine the patterns in the top journals by simultaneously identifying the topical probability in each period, as well as the top authors and keyphrases. The results indicate that in recent years diversified topics have become more prevalent and convergent topics have become more clearly represented.ConclusionThe results of our analysis implies that overtime the field of bioinformatics becomes more interdisciplinary where there is a steady increase in peripheral fields such as conceptual, mathematical, and system biology. These results are confirmed by integrated analysis of topic distribution as well as top ranked keyphrases, authors, and journals.

Funding agencies such as the U.S. National Science Foundation (NSF), U.S. National Institutes of Health (NIH), and the Transportation Research Board (TRB) of The National Academies make their online grant databases publicly available which document a variety of information on grants that have been funded over the past few decades. In this paper, based on a quantitative analysis of the TRB’s Research In Progress (RIP) online database, we explore the feasibility of automatically estimating the appropriate funding level, given the textual description of a transportation research project. We use statistical Text Mining (TM) and Machine Learning (ML) technologies to build this model using the 14,000 or more records of the TRB’s RIP research grants big data. Several Natural Language Processing (NLP) based text representation models such as the Latent Dirichlet Allocation (LDA), Latent Semantic Indexing (LSI) and the Doc2Vec Machine Learning (ML) approach are used to vectorize the project descriptions and generate semantic vectors. Each of these representations is then used to train supervised regression models such as Random Forest (RF) regression. Out of the three latent feature generation models, we found LDA gives the least Mean Absolute Error (MAE) using 300 feature dimensions and RF regression model. However, based on the correlation coefficients, it was found that it is not very feasible to accurately predict the funding level directly from the unstructured project abstract, given the large variations in source agencies, subject areas, and funding levels. By using separate prediction models for different types of funding agencies, funding levels were better correlated with the project abstract.

Topic modelling is an important technique for extracting meaningful insights from large volumes of unstructured text data. The paper presents a federated technique for topic modelling that is based on a novel approach of the Latent Dirichlet Allocation (LDA) method for topic model generation. The proposed approach enables a topic model to be developed in a distributed environment using data continually generated from multiple sources without the need for sharing actual data. The first iteration of the topic modelling uses unsupervised LDA at each device generating the data. The results of each device are aggregated at a central server to generate a set of seed words that are used for guided LDA by the subsequent iterations of topic modelling. The proposed work, Federated LDA (F-LDA) has been evaluated using two datasets: a text dataset of dialogues between patients and doctors based on factual conversations and another comprising tweets related to depression. Comparing the performance of F-LDA with that of a centralized LDA, it was observed that F-LDA results in improved coherence score as well as diversity score in comparison to centralized LDA. This indicates that F-LDA achieves better interpretable topics covering a wide range of themes without redundancy.