Application of Text Mining Technology to Solve Project Management Problems

With the overwhelming increase in the amount of texts on the web, it is almost impossible for people to keep abreast of up-to-date information. Text mining is a process by which interesting information is derived from text through the discovery of patterns and trends. Text mining algorithms are used to guarantee the quality of extracted knowledge. However, the extracted patterns using text or data mining algorithms or methods leads to noisy patterns and inconsistency. Thus, different challenges arise, such as the question of how to understand these patterns, whether the model that has been used is suitable, and if all the patterns that have been extracted are relevant. Furthermore, the research raises the question of how to give a correct weight to the extracted knowledge. To address these issues, this paper presents a text post-processing method, which uses a pattern co-occurrence matrix to find the relation between extracted patterns in order to reduce noisy patterns. The main objective of this paper is not only reducing the number of closed sequential patterns, but also improving the performance of pattern mining as well. The experimental results on Reuters Corpus Volume 1 data collection and TREC filtering topics show that the proposed method is promising.

With the growing popularity of electronic commerce, for enterprises to enhance the competitiveness, enterprise decision makers and active innovation, to build a distinctive, individualized and user characteristics of e-commerce network platform, e-commerce personalized recommendation was thus born. Enterprises make use of the data mining technology, search, record, analysis server, browser the user’s browsing history and purchase records, selected users a sense of interest from the market information, provide differentiated network marketing strategy. Presented in this paper based on text mining e-commerce website personalized recommendation technology is the A, this paper through the comparison of text mining and Web Mining Technology Analysis of the text mining technology of personalized recommendation technology of precision, which information data of the study of mining technology has great reference value.

Background: While it is commonly recognized that almost every work and research discipline utilize their own taxonomy, the language used within a specific discipline may also vary depending on numerous factors, including the desired effect of the information being communicated and the intended audience. Different audiences are reached through publication of information, including research results, in different types of publication outlets such as newspapers, newsletters, magazines, websites, and journals. Prior research has shown that students, both undergraduate and graduate, as well as faculty may have a difficult time locating information in different publication outlet types (e.g., magazines, newspapers, journals). The type of publication may affect the ease of understanding and also the confidence placed in the acquired information. A text analytics tool for classifying the source of research as a newsletter (used as a substitute for newspaper articles), a magazine, or an academic journal article has been developed to assist students, faculty, and researchers in identifying the likely source type of information and classifying their own writings with respect to these possible publication outlet types. Literature Review: Literature on information literacy is discussed as this forms the motivation for the reported research. Additionally, prior research on using text mining and text analytics is examined to better understand the methodology employed, including a review of the original Scale of Theoretical and Applied Research system, adapted for the current research. Research Questions: The primary research question is: Can a text mining and text analytics approach accurately determine the most probable publication source type with respect to being from a newsletter, magazine, or journal? Methodology: A text mining and text analytics algorithm, STAR’ (System for Text Analytics-based Ranking), was developed from a previously researched text mining tool, STAR (Scale of Theoretical and Applied Research), that was used to classify the research type of articles between theoretical and applied research. The new text mining method, STAR’, analyzes the language used in manuscripts to determine the type of publication. This method first mines all words from corresponding publication source types to determine a keyword corpus. The corpus is then used in a text analytics process to classify full newsletters, magazine articles, and journal articles with respect to their publication source. All newsletters, magazine articles, and journal articles are from the library and information sciences (LIS) domain. Results: The STAR’ text analytics method was evaluated as a proof of concept on a specific LIS organizational newsletter, as well as articles from a single LIS magazine and a single LIS journal. STAR’ was able to classify the newsletters, magazine articles, and journal articles with 100% accuracy. Random samples from another similar LIS newsletter and a different LIS journal were also evaluated to examine the robustness of the STAR’ method in the initial proof of concept. Following the positive results of the proof of concept, additional journal, magazine, and newsletter articles were used to evaluate the generalizability of STAR’. The second-round results were very positive for differentiating journals and newsletters from other publication types, but revealed potential issues for distinguishing magazine articles from other types of publications. Discussion: STAR’ demonstrates that the language used for transferring information within a specific discipline does differ significantly depending on the intended recipients of the research knowledge. Further work is needed to examine language usage specific to magazine articles. Conclusions: The STAR’ method may be used by students and faculty to identify the likely source of research or discipline-specific information. This may improve trust in the reliability of information due to different levels of rigor applied to different types of publications. Additionally, the STAR’ classifications may be used by students, faculty, or researchers to determine the most appropriate type of outlet and correspondingly the most appropriate type of audience for the reported information in their own manuscripts, thereby improving the chance for successful sharing of information to appropriate audiences who will deem the information to be reliable, through publication in the most relevant outlet type.

Purpose– The use of social media and in particular community Question Answering (Q & A) websites by learners has increased significantly in recent years. The vast amounts of data posted on these sites provide an opportunity to investigate the topics under discussion and those receiving most attention. The purpose of this paper is to automatically analyse the content of a popular computer programming Q & A website, StackOverflow (SO), determine the exact topics of posted Q & As, and narrow down their categories to help determine subject difficulties of learners. By doing so, the authors have been able to rank identified topics and categories according to their frequencies, and therefore, mark the most asked about subjects and, hence, identify the most difficult and challenging topics commonly faced by learners of computer programming and software development.Design/methodology/approach– In this work the authors have adopted a heuristic research approach combined with a text mining approach to investigate the topics and categories of Q & A posts on the SO website. Almost 186,000 Q & A posts were analysed and their categories refined using Wikipedia as a crowd-sourced classification system. After identifying and counting the occurrence frequency of all the topics and categories, their semantic relationships were established. This data were then presented as a rich graph which could be visualized using graph visualization software such as Gephi.Findings– Reported results and corresponding discussion has given an indication that the insight gained from the process can be further refined and potentially used by instructors, teachers, and educators to pay more attention to and focus on the commonly occurring topics/subjects when designing their course material, delivery, and teaching methods.Research limitations/implications– The proposed approach limits the scope of the analysis to a subset of Q & As which contain one or more links to Wikipedia. Therefore, developing more sophisticated text mining methods capable of analysing a larger portion of available data would improve the accuracy and generalizability of the results.Originality/value– The application of text mining and data analytics technologies in education has created a new interdisciplinary field of research between the education and information sciences, called Educational Data Mining (EDM). The work presented in this paper falls under this field of research; and it is an early attempt at investigating the practical applications of text mining technologies in the area of computer science (CS) education.

Approximately 90% of the world’s data is held in unstructured formats which mean that it is getting harder for people to extract information from huge amount of data. Motivated by the practical need of specific types of real world data analysis problems, many mining algorithms and knowledge management systems are designed and studied. The management of a vast amount of unstructured customer-related knowledge in academic libraries has become area of significant recent interest. In this paper, we proposed a scheme organizing customer knowledge in academic libraries by using text mining technology. The entity extraction phase called Named Entity Recognition aims to discover proper names, their variations and classes. A database with related entities needs to be created for entity extraction and correlation processes. Benefited from LRD method for text mining, the scheme provides a formal and explicit specification to deliver a shared conceptualization of customer knowledge.KeywordsText miningKnowledge managementAcademic libraries

This paper aims to identify the trends in machine learning research using text mining. The researcharticles contain significant knowledge and research results. However, they are long and have many noisy results such that it takes a lot of human efforts to analyze them. Text mining can be used to analyze and extract useful information from a large number of research articles quickly and automatically. Text mining is the method of defining innovative, and unseen knowledge from unstructured, semi-structured and structured textual data. This knowledge contributed to very important information that can derive from textual data. In this paper, text mining methods are applied to detect trends of terms that occur in the research articles and how they varies over time. We collected 21,906 scientific papers from six top journals in the field of machine learning published in period 1988-2017 and analyzed them using text mining. Our result analysis shows a changing trend of various terms in Machine learning research in three decades. The analysis of our study helps the upcoming researchers to explore the significant research area of machine learning.

To provide scientific institutions with comprehensive and well-maintained documentation of their research information in a current research information system (CRIS), they have the best prerequisites for the implementation of text and data mining (TDM) methods. Using TDM helps to better identify and eliminate errors, improve the process, develop the business, and make informed decisions. In addition, TDM increases understanding of the data and its context. This not only improves the quality of the data itself, but also the institution’s handling of the data and consequently the analyses. This present paper deploys TDM in CRIS to analyze, quantify, and correct the unstructured data and its quality issues. Bad data leads to increased costs or wrong decisions. Ensuring high data quality is an essential requirement when creating a CRIS project. User acceptance in a CRIS depends, among other things, on data quality. Not only is the objective data quality the decisive criterion, but also the subjective quality that the individual user assigns to the data.

Recent trends in educational technology have led to emergence of methods such as teaching analytics (TA) in understanding and management of the teaching–learning processes. Didactically, teaching analytics is one of the promising and emerging methods within the Education domain that have proved to be useful, towards scholastic ways to make use of substantial pieces of evidence drawn from educational data to improve the teaching–learning processes and quality of performance. For this purpose, this study proposed an educational process and data mining plus machine learning (EPDM + ML) model applied to contextually analyze the teachers’ performances and recommendations based on data derived from students’ evaluation of teaching (SET). The EPDM + ML model was designed and implemented based on amalgamation of the Text mining and Machine learning technologies that builds on the descriptive decision theory, which studies the rationality behind decisions the learners are disposed to make based on the textual data quantification and statistical analysis. To this effect, the study determines pedagogical factors that influences the students' recommendations for their teachers, what role the sentiment and emotions expressed by the students in the SET play in the way they evaluate the teachers by taking into account the gender of the teachers. This includes how to automatically predict what a student’s recommendation for the teachers may be based on information about the students’ gender, average sentiment, and emotional valence they have shown in the SET. Practically, we applied the Text mining technique to extract the different sentiments and emotions (intensities of the comments) expressed by the students in the SET, and then utilized the quantified data (average sentiment and emotional valence) to conduct an analysis of covariance and Kruskal Wallis Test to determine the influential factors, as well as, how the students’ recommendation for the teachers differ by considering the gender constructs, respectively. While a large proportion of the comments that we analyzed (n = 85,378) was classified to be neutral and predominantly interpreted to be positive in nature considering the sentiments (76.4%), and emotional valence (88.2%) expressed by the students. The results of our analysis shows that for the students’ comments which contain some kind of positive or negative sentiment (23.6%) and emotional valence (11.8%); that females students recommended the teachers taking into account the sentiments (p = .000). While the males appear to be slightly borderline in terms of emotions (p = .056) and sentiment (p = .077). Also, the EPDM + ML model showed to be a good predictor and efficient method in determining what the students’ recommendation scores for the teachers would be, going by the high and acceptable values of the precision (1.00), recall (1.00), specificity (1.00), accuracy (1.00), F1-score (1.00) and zero error-rate (0.00) which we validated using the k-fold cross-validation method, with 63.6% of optimal k-values observed. In theory, we note that not only does the proposed method (EPDM + ML) proves to be useful towards effective analysis of SET and its implications within the educational domain. But can be utilized to determine prominent factors that influences the students’ evaluation and recommendation of the teachers, as well as helps provide solutions to the ever-increasingly need to advance and support the teaching–learning processes and/or students’ learning experiences in a rapidly changing educational environment or ecosystem.

We recently released a plugin to Pathvisio to assist the curator with automatic integration of heterogeneous pathway knowledge. Pathway curation involves gathering this knowledge not only through discussions with peers, but also from biological databases and the scientific literature. Integrating this all into one pathway is challenging and to date still mainly a laborious manual process.More and more biological databases provide webservices (ie. REST or SOAP). The main objective of webservices is to provide an interface for other computer applications. Our plug-in uses these interfaces to gather binary relationships of a given pathway entity and present the relation in star shaped figures to the curator. The plug-in also uses functionality of Pathvisio to project and visualize research data on pathways. If research data exists, this can then be projected on to the suggested potential novel entities. The curator can then make an educated decision on extending pathways with database knowledge.Different text mining methods exist to process large quantities of available literature. Text mining results can be very valuable in pathway curation. Integrating them in a similar way as with biological databases is difficult. It requires a text-mining output standard. In our current plug-in we included text-mining support in our current version of the plug-in.We like to propose here a standard for text mining output. The output standard is a data object that captures the relationships found with text mining, the document identifiers, the sentences from where the results originate, the method used, and its metrics. Such an output standard would enable a user-oriented comparison of text mining results. Currently, text-mining results are mainly evaluated on their performance in finding relevant results. By enabling a text mining out standard, the results become available to the larger user community. The data is immediately accessible to a wide range of applications. If a text mining method supports our output standard, the pathway curation community can use it in Pathvisio through our plug-in.

We recently released a plugin to Pathvisio to assist the curator with automatic integration of heterogeneous pathway knowledge. Pathway curation involves gathering this knowledge not only through discussions with peers, but also from biological databases and the scientific literature. Integrating this all into one pathway is challenging and to date still mainly a laborious manual process.More and more biological databases provide webservices (ie. REST or SOAP). The main objective of webservices is to provide an interface for other computer applications. Our plug-in uses these interfaces to gather binary relationships of a given pathway entity and present the relation in star shaped figures to the curator. The plug-in also uses functionality of Pathvisio to project and visualize research data on pathways. If research data exists, this can then be projected on to the suggested potential novel entities. The curator can then make an educated decision on extending pathways with database knowledge.Different text mining methods exist to process large quantities of available literature. Text mining results can be very valuable in pathway curation. Integrating them in a similar way as with biological databases is difficult. It requires a text-mining output standard. In our current plug-in we included text-mining support in our current version of the plug-in.We like to propose here a standard for text mining output. The output standard is a data object that captures the relationships found with text mining, the document identifiers, the sentences from where the results originate, the method used, and its metrics. Such an output standard would enable a user-oriented comparison of text mining results. Currently, text-mining results are mainly evaluated on their performance in finding relevant results. By enabling a text mining out standard, the results become available to the larger user community. The data is immediately accessible to a wide range of applications. If a text mining method supports our output standard, the pathway curation community can use it in Pathvisio through our plug-in.

Our goal is to support users who want to discover or create knowledge from a large amount of text data. Text mining is a process that extracts novel knowledge from unstructured data. A variety of applications for text mining, such as Total Environment for Text Data Mining, have been proposed. However, to the best of our knowledge, these methods are not effective at conducting text mining tasks aimed at finding novel knowledge. In this paper, we discuss characteristics of the text mining process and propose a design principle for building text mining applications based on two concepts: (1) text mining is an exploratory search task, and (2) text mining is a process for creative knowledge work.

Business intelligence is a critical in defining the strategy and roadmap of organizations. However, business intelligence covers too much wide coverage to consider all of fields such as data analytics, text mining, predictive analytics, and so on. Among these fields, the most important is information analysis and prediction. Therefore, we suggest a business intelligence application based on the adaptive recognition of user intention and usage patterns in the mobile environment. This application is named InSciTe Adaptive and is based on text mining and semantic web technologies. It supports technology-focused analysis and predictions, such as technology trends analysis, element technology analysis, and convergence technology discovery, as well as adaptive recognition of the user's intention by using semi-automatic user modeling processes. Through adaptive user modeling, this application can provide a more dynamic service flow and more up-to-date analysis results based on the user's intention, compared to existing applications, which provide static analysis results and service flow.

Funding agencies such as the U.S. National Science Foundation (NSF), U.S. National Institutes of Health (NIH), and the Transportation Research Board (TRB) of The National Academies make their online grant databases publicly available which document a variety of information on grants that have been funded over the past few decades. In this paper, based on a quantitative analysis of the TRB’s Research In Progress (RIP) online database, we explore the feasibility of automatically estimating the appropriate funding level, given the textual description of a transportation research project. We use statistical Text Mining (TM) and Machine Learning (ML) technologies to build this model using the 14,000 or more records of the TRB’s RIP research grants big data. Several Natural Language Processing (NLP) based text representation models such as the Latent Dirichlet Allocation (LDA), Latent Semantic Indexing (LSI) and the Doc2Vec Machine Learning (ML) approach are used to vectorize the project descriptions and generate semantic vectors. Each of these representations is then used to train supervised regression models such as Random Forest (RF) regression. Out of the three latent feature generation models, we found LDA gives the least Mean Absolute Error (MAE) using 300 feature dimensions and RF regression model. However, based on the correlation coefficients, it was found that it is not very feasible to accurately predict the funding level directly from the unstructured project abstract, given the large variations in source agencies, subject areas, and funding levels. By using separate prediction models for different types of funding agencies, funding levels were better correlated with the project abstract.

Text mining technologies are increasingly providing an effective response to the growing demand for faster access to the vast amounts of information hidden in the literature. Several tools are becoming available which offer the capability to mine the literature for specific information, such as for example protein-protein interactions or drug-disease relationships. The biomedical text mining community regularly verifies the progress of such systems through competitive evaluations, such as BioCreative, BioNLP, i2b2, CALBC, CLEF-ER, BioASQ, etc. The OntoGene system is a text mining system which specialises in the detection of entities and relationships from selected categories, such as proteins, genes, drugs, diseases, chemicals. The quality of the system has been tested several times through participation in some of the community-organised evaluation campaigns. In order to make the advanced text mining capabilities of the OntoGene system more widely accessible without the burden of installation of complex software, we are setting up a web service that will allow any remote user to submit arbitrary documents. The results of the mining service (entities and relationships) are then delivered back to the user as XML data, or optionally can be inspected via a flexible web interface.

DEFINING TEXT MINING Text mining can be broadly defined as a knowledge-intensive process in which a user interacts with a document collection over time by using a suite of analysis tools. In a manner analogous to data mining, text mining seeks to extract useful information from data sources through the identification and exploration of interesting patterns. In the case of text mining, however, the data sources are document collections, and interesting patterns are found not among formalized database records but in the unstructured textual data in the documents in these collections. Certainly, text mining derives much of its inspiration and direction from seminal research on data mining. Therefore, it is not surprising to find that text mining and data mining systems evince many high-level architectural similarities. For instance, both types of systems rely on preprocessing routines, pattern-discovery algorithms, and presentation-layer elements such as visualization tools to enhance the browsing of answer sets. Further, text mining adopts many of the specific types of patterns in its core knowledge discovery operations that were first introduced and vetted in data mining research. Because data mining assumes that data have already been stored in a structured format, much of its preprocessing focus falls on two critical tasks: Scrubbing and normalizing data and creating extensive numbers of table joins. In contrast, for text mining systems, preprocessing operations center on the identification and extraction of representative features for natural language documents.