Demo: Using RapidMiner for Text Mining

With the growing popularity of electronic commerce, for enterprises to enhance the competitiveness, enterprise decision makers and active innovation, to build a distinctive, individualized and user characteristics of e-commerce network platform, e-commerce personalized recommendation was thus born. Enterprises make use of the data mining technology, search, record, analysis server, browser the user’s browsing history and purchase records, selected users a sense of interest from the market information, provide differentiated network marketing strategy. Presented in this paper based on text mining e-commerce website personalized recommendation technology is the A, this paper through the comparison of text mining and Web Mining Technology Analysis of the text mining technology of personalized recommendation technology of precision, which information data of the study of mining technology has great reference value.

Text mining technologies are increasingly providing an effective response to the growing demand for faster access to the vast amounts of information hidden in the literature. Several tools are becoming available which offer the capability to mine the literature for specific information, such as for example protein-protein interactions or drug-disease relationships. The biomedical text mining community regularly verifies the progress of such systems through competitive evaluations, such as BioCreative, BioNLP, i2b2, CALBC, CLEF-ER, BioASQ, etc. The OntoGene system is a text mining system which specialises in the detection of entities and relationships from selected categories, such as proteins, genes, drugs, diseases, chemicals. The quality of the system has been tested several times through participation in some of the community-organised evaluation campaigns. In order to make the advanced text mining capabilities of the OntoGene system more widely accessible without the burden of installation of complex software, we are setting up a web service that will allow any remote user to submit arbitrary documents. The results of the mining service (entities and relationships) are then delivered back to the user as XML data, or optionally can be inspected via a flexible web interface.

ABSTRACTStemming is the basic operation in Natural language processing (NLP) to remove derivational and inflectional affixes without performing a morphological analysis. This practice is essential to extract the root or stem. In NLP domains, the stemmer is used to improve the process of information retrieval (IR), text classifications (TC), text mining (TM) and related applications. In particular, Urdu stemmers utilize only uni-gram words from the input text by ignoring bigrams, trigrams, and n-gram words. To improve the process and efficiency of stemming, bigrams and trigram words must be included. Despite this fact, there are a few developed methods for Urdu stemmers in the past studies. Therefore, in this paper, we proposed an improved Urdu stemmer, using hybrid approach divided into multi-step operation, to deal with unigram, bigram, and trigram features as well. To evaluate the proposed Urdu stemming method, we have used two corpora; word corpus and text corpus. Moreover, two different evaluation metrics have been applied to measure the performance of the proposed algorithm. The proposed algorithm achieved an accuracy of 92.97% and compression rate of 55%. These experimental results indicate that the proposed system can be used to increase the effectiveness and efficiency of the Urdu stemmer for better information retrieval and text mining applications.

This paper improves the traditional text mining technology which cannot understand the text semantics. The author discusses the text mining methods based on ontology and puts forward text mining model based on domain ontology. Ontology structure is built firstly and the “concept-concept” similarity matrix is introduced, then a conception vector space model based on domain ontology is used to take the place of traditional vector space model to represent the documents in order to realize text mining. Finally, the author does a case and draws some conclusions.KeywordsOntologytext miningdomain ontologyvector space model

Business intelligence is a critical in defining the strategy and roadmap of organizations. However, business intelligence covers too much wide coverage to consider all of fields such as data analytics, text mining, predictive analytics, and so on. Among these fields, the most important is information analysis and prediction. Therefore, we suggest a business intelligence application based on the adaptive recognition of user intention and usage patterns in the mobile environment. This application is named InSciTe Adaptive and is based on text mining and semantic web technologies. It supports technology-focused analysis and predictions, such as technology trends analysis, element technology analysis, and convergence technology discovery, as well as adaptive recognition of the user's intention by using semi-automatic user modeling processes. Through adaptive user modeling, this application can provide a more dynamic service flow and more up-to-date analysis results based on the user's intention, compared to existing applications, which provide static analysis results and service flow.

Recent trends in educational technology have led to emergence of methods such as teaching analytics (TA) in understanding and management of the teaching–learning processes. Didactically, teaching analytics is one of the promising and emerging methods within the Education domain that have proved to be useful, towards scholastic ways to make use of substantial pieces of evidence drawn from educational data to improve the teaching–learning processes and quality of performance. For this purpose, this study proposed an educational process and data mining plus machine learning (EPDM + ML) model applied to contextually analyze the teachers’ performances and recommendations based on data derived from students’ evaluation of teaching (SET). The EPDM + ML model was designed and implemented based on amalgamation of the Text mining and Machine learning technologies that builds on the descriptive decision theory, which studies the rationality behind decisions the learners are disposed to make based on the textual data quantification and statistical analysis. To this effect, the study determines pedagogical factors that influences the students' recommendations for their teachers, what role the sentiment and emotions expressed by the students in the SET play in the way they evaluate the teachers by taking into account the gender of the teachers. This includes how to automatically predict what a student’s recommendation for the teachers may be based on information about the students’ gender, average sentiment, and emotional valence they have shown in the SET. Practically, we applied the Text mining technique to extract the different sentiments and emotions (intensities of the comments) expressed by the students in the SET, and then utilized the quantified data (average sentiment and emotional valence) to conduct an analysis of covariance and Kruskal Wallis Test to determine the influential factors, as well as, how the students’ recommendation for the teachers differ by considering the gender constructs, respectively. While a large proportion of the comments that we analyzed (n = 85,378) was classified to be neutral and predominantly interpreted to be positive in nature considering the sentiments (76.4%), and emotional valence (88.2%) expressed by the students. The results of our analysis shows that for the students’ comments which contain some kind of positive or negative sentiment (23.6%) and emotional valence (11.8%); that females students recommended the teachers taking into account the sentiments (p = .000). While the males appear to be slightly borderline in terms of emotions (p = .056) and sentiment (p = .077). Also, the EPDM + ML model showed to be a good predictor and efficient method in determining what the students’ recommendation scores for the teachers would be, going by the high and acceptable values of the precision (1.00), recall (1.00), specificity (1.00), accuracy (1.00), F1-score (1.00) and zero error-rate (0.00) which we validated using the k-fold cross-validation method, with 63.6% of optimal k-values observed. In theory, we note that not only does the proposed method (EPDM + ML) proves to be useful towards effective analysis of SET and its implications within the educational domain. But can be utilized to determine prominent factors that influences the students’ evaluation and recommendation of the teachers, as well as helps provide solutions to the ever-increasingly need to advance and support the teaching–learning processes and/or students’ learning experiences in a rapidly changing educational environment or ecosystem.

Funding agencies such as the U.S. National Science Foundation (NSF), U.S. National Institutes of Health (NIH), and the Transportation Research Board (TRB) of The National Academies make their online grant databases publicly available which document a variety of information on grants that have been funded over the past few decades. In this paper, based on a quantitative analysis of the TRB’s Research In Progress (RIP) online database, we explore the feasibility of automatically estimating the appropriate funding level, given the textual description of a transportation research project. We use statistical Text Mining (TM) and Machine Learning (ML) technologies to build this model using the 14,000 or more records of the TRB’s RIP research grants big data. Several Natural Language Processing (NLP) based text representation models such as the Latent Dirichlet Allocation (LDA), Latent Semantic Indexing (LSI) and the Doc2Vec Machine Learning (ML) approach are used to vectorize the project descriptions and generate semantic vectors. Each of these representations is then used to train supervised regression models such as Random Forest (RF) regression. Out of the three latent feature generation models, we found LDA gives the least Mean Absolute Error (MAE) using 300 feature dimensions and RF regression model. However, based on the correlation coefficients, it was found that it is not very feasible to accurately predict the funding level directly from the unstructured project abstract, given the large variations in source agencies, subject areas, and funding levels. By using separate prediction models for different types of funding agencies, funding levels were better correlated with the project abstract.

Approximately 90% of the world’s data is held in unstructured formats which mean that it is getting harder for people to extract information from huge amount of data. Motivated by the practical need of specific types of real world data analysis problems, many mining algorithms and knowledge management systems are designed and studied. The management of a vast amount of unstructured customer-related knowledge in academic libraries has become area of significant recent interest. In this paper, we proposed a scheme organizing customer knowledge in academic libraries by using text mining technology. The entity extraction phase called Named Entity Recognition aims to discover proper names, their variations and classes. A database with related entities needs to be created for entity extraction and correlation processes. Benefited from LRD method for text mining, the scheme provides a formal and explicit specification to deliver a shared conceptualization of customer knowledge.KeywordsText miningKnowledge managementAcademic libraries

Objectives : The purpose of this study is to analyze research trends based on text mining technology from the published papers in the Journal of Korean Society of Environmental Engineering.Methods : From 2000 to 2019, a total of 2,743 published papers were analyzed using text mining techniques. Term frequency, TF-IDF for document classification, word association analysis were applied to find the characteristics of text data.Results and Discussion : When confirming the high appearance of the word in the published paper during 2000-2019, ‘adsorption’, ‘heavy metals’, ‘activated carbon’, ‘sediment’, ‘sewage sludge’ was found in order. It implies that large number of studies in the journal were focused on the water quality field mainly. TF-IDF analysis classified the studies into five groups; 1) drinking water treatment field, 2) water quality modeling field, 3) heavy metal adsorption field, 4) biological sewage treatment field, 5) environmental catalyst field. These results by TF-IDF show that a large proportion of studies were published in the field of water quality modeling and biological sewage treatment. When we analyzed the term frequency every five years, “adsorption” and “heavy metals” were the highly-frequency occurrence words from 2000 to 2009, but in the last 5 years, new words such as “fine dust”, “cesium”, and “ecological toxicity” were appeared. It seems that the research was reflected in the recent environmental issues.Conclusions : A lot of studies has been focused on the field of water quality but in recent years, new research topics are being studied related to atmosphere, toxicity, and radiation. Applying a more sophisticated and diverse text mining technique will be of great help to improve the environmental engineering research field.

This paper studies the classification of the coal supply-demand information that comes from the Internet by using text mining technology. Firstly, it introduces the concept of Web text mining and the basic theories of the classification process. Then, it makes a further discussion about related technologies in all aspects of Web text mining process, and focuses on the analysis of the text classification algorithm. At the same time, this paper explores the current development of coal industry and the application of Web text mining on the coal supply-demand market. At last, it creates the information classification model based on the text classification algorithm of Web text mining, which visualizes the process of classifying the online coal supply-demand information.

As information systems are dealing with a growing amount of unstructured text information, text mining technologies and their corresponding software tools have become increasingly widespread. However, text mining is still a young discipline that is starting to witness the advent of flexible, robust, and general software architectures that leverage the abundance of research results in benefiting application developers. In this article, we introduce Plato, a new software architecture that is unique in the set of text mining functionalities it provides to general purpose applications. This system is described and compared to other classification, information retrieval, and information extraction systems. We also include two case studies of applications developed with Plato and currently in use.

The objective of this case study is to illustrate how text mining of open-ended responses and sentimental expressions (positive or negative) from student survey could yield valuable information for improving student experience management (SEM). The concept of student SEM was borrowed from the notion of customer experience management (CEM), which aims for ongoing improvement of customer relations through understanding of the customer's point of view. With the advance of text mining technology, which is based upon artificial intelligence and machine learning, textual data that were previously underutilized are found to be valuable in CEM. To illustrate how text mining can be applied to SEM, the authors discuss an example from a campus-wide survey conducted at Arizona State University. The purpose of this survey was to better understand student experiences with instructional technology in order for administrators to make data-driven decisions on its implementation. Rather than imposing the researchers' preconceived suppositions on the students by using force-option survey items, researchers on this project chose to use open-ended questions in order to elicit a free emergence of themes from the students. The most valuable lesson learned from this study is that students perceive an ideal environment as a web of mutually supporting systems. Specifically, online access should be augmented by use of laptops and availability of course materials, whereas virtual classes should be balanced by human interactions.

The indiGo project aims at improving process knowledge by successive consolidation of feedback, ranging from private annotation, through structured communication in communities of practice, to improved process models and lessons learned. It develops a methodology and integrates previously independent software for process modeling, moderated discourses, experience management and text mining. Both will be evaluated in case studies.KeywordsText MiningBusiness Process ModelProcess DescriptionPrivate AnnotationStart PageThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This paper analyzes the possibilities of using text mining methods to solve project management problems. Text mining is an area of artificial intelligence. The use of text mining technology allows to find useful, previously unknown information in text arrays. Due to the fact that during the implementation of projects a lot of textual data is required, presented in a certain form (paper and electronic documents, letters, between project participants, etc.) and often unstructured, technical tools are needed that can process this information and extract useful knowledge from it that can help projects achieve their goals faster. The article includes the technology of text mining itself, the main approaches and methods that are currently used in this area. Things are impossible due to the fact that it is the use of text mining algorithms that can help in solving project management problems.

Purpose– The use of social media and in particular community Question Answering (Q & A) websites by learners has increased significantly in recent years. The vast amounts of data posted on these sites provide an opportunity to investigate the topics under discussion and those receiving most attention. The purpose of this paper is to automatically analyse the content of a popular computer programming Q & A website, StackOverflow (SO), determine the exact topics of posted Q & As, and narrow down their categories to help determine subject difficulties of learners. By doing so, the authors have been able to rank identified topics and categories according to their frequencies, and therefore, mark the most asked about subjects and, hence, identify the most difficult and challenging topics commonly faced by learners of computer programming and software development.Design/methodology/approach– In this work the authors have adopted a heuristic research approach combined with a text mining approach to investigate the topics and categories of Q & A posts on the SO website. Almost 186,000 Q & A posts were analysed and their categories refined using Wikipedia as a crowd-sourced classification system. After identifying and counting the occurrence frequency of all the topics and categories, their semantic relationships were established. This data were then presented as a rich graph which could be visualized using graph visualization software such as Gephi.Findings– Reported results and corresponding discussion has given an indication that the insight gained from the process can be further refined and potentially used by instructors, teachers, and educators to pay more attention to and focus on the commonly occurring topics/subjects when designing their course material, delivery, and teaching methods.Research limitations/implications– The proposed approach limits the scope of the analysis to a subset of Q & As which contain one or more links to Wikipedia. Therefore, developing more sophisticated text mining methods capable of analysing a larger portion of available data would improve the accuracy and generalizability of the results.Originality/value– The application of text mining and data analytics technologies in education has created a new interdisciplinary field of research between the education and information sciences, called Educational Data Mining (EDM). The work presented in this paper falls under this field of research; and it is an early attempt at investigating the practical applications of text mining technologies in the area of computer science (CS) education.