A Pattern Discovery Model for Effective Text Mining

Nowadays, the big challenge for community of Information Retrieval domain is to discover the relevance feature in text document which helps to decide whether document is relevant or irrelevant. Most existing text mining methods are based on term-based approaches which extract terms from a training set for describing relevant feature. In termbased approach multiple meaning of same word in different context leads to generate polysemy and synonymy issue. The term based approach also suffers from low level support problem. Even though pattern based text mining approach solve low level support problem but still this approach suffers from large number of noise pattern. In the propose work, a pattern discovery approach for text mining is explored. This approach discovers frequent sequential pattern and closed sequential patterns in text documents for identifying the most information contents of the documents and extract useful features for text mining. It also classifies extracted terms into three categories: positive terms, general terms, and negative terms. In this way, we update extracted features using multiple revising strategies. This technique discovers positive and negative patterns in text documents as higher level features in order to accurately weight low-level terms based on their specificity.

With the overwhelming increase in the amount of texts on the web, it is almost impossible for people to keep abreast of up-to-date information. Text mining is a process by which interesting information is derived from text through the discovery of patterns and trends. Text mining algorithms are used to guarantee the quality of extracted knowledge. However, the extracted patterns using text or data mining algorithms or methods leads to noisy patterns and inconsistency. Thus, different challenges arise, such as the question of how to understand these patterns, whether the model that has been used is suitable, and if all the patterns that have been extracted are relevant. Furthermore, the research raises the question of how to give a correct weight to the extracted knowledge. To address these issues, this paper presents a text post-processing method, which uses a pattern co-occurrence matrix to find the relation between extracted patterns in order to reduce noisy patterns. The main objective of this paper is not only reducing the number of closed sequential patterns, but also improving the performance of pattern mining as well. The experimental results on Reuters Corpus Volume 1 data collection and TREC filtering topics show that the proposed method is promising.

It is a big challenge to develop effective methods that can discover high quality and useful features in text documents. Most existing information retrieval and text mining methods focuses on term-based approach that often suffers from the problems of term variation and noise. This paper illustrates an innovative approach that discovers relevant knowledge to precisely describe text features for retrieving web information. In particular, it extracts precise text patterns by considering both relevant and irrelevant documents. Then, the discovered patterns are used to find accurate relevant features in a training set. The proposed approach has been evaluated through the implementation of a novel information filtering model and a comparative evaluation is conducted by invoking state-of-the-art models. The experimental results obtained based on the Reuters Corpus Volume 1 and TREC topics show that the proposed approach significantly outperforms the best baseline method.

PurposeIn business, online reviews have an economic impact on firm performance. Customers’ data in the form of online reviews was used to understand the appreciation and service complaints written by previous customers. The study is an analysis of the online reviews written by the customers about Indian heritage hotels. This study aims to understand the dimensions of service appreciation and service complaints by comparing positive- and negative-rated reviews and find the patterns in the determinants of the satisfaction and dissatisfaction of the customers.Design/methodology/approachA total of 23,643 online reviews about heritage hotels were collected from the TripAdvisor website by using a Web crawler developed in Python. A total of 1000 reviews were randomly selected for further analysis to eliminate the bandwagon effect. Unsupervised text mining techniques were used to analyze reviews and find out the interesting patterns in text data.FindingsBased on Herzberg two-factor theory, this study found satisfied and dissatisfied determinants separately. The study revealed some common categories discussed by satisfied and dissatisfied customers. The factors which satisfy the customers may also dissatisfy the customers if not delivered properly. Satisfied customers mentioned about tangible features of the hotel stay, which includes physical signifiers, traditional services, staff behavior and professionalism and core products (rooms, food). However, most of the customers complained about intangible service problems, such as staff attitude, services failure, issues with reservation and food, value for money and room condition. The results are contradicting with commercial hotels-based studies owing to the unique services provided by heritage hotels.Practical implicationsThe dimensions for satisfaction and dissatisfaction among customer of heritage hotels provide marketers to understand the real emotion and perception of the customers. As these dimensions were extracted through text mining of the reviews written by the customer of heritage hotels, the results would certainly give better insights to the hotel marketers.Originality/valueThe study is a rare attempt to study online reviews of customers on heritage hotels through a text mining approach and find the patterns in the behavior and the determinants of satisfaction and dissatisfaction of customers.

Data fusion has been widely used in various fields of automation. This paper describes a multisensor integration system: range and intensity image processing system, which can be used for object recognition and classification. In the data fusion processing, a new method called generalized evidence inference method is used by the system. The method presented here unifies both Bayesian theory and Dempster-Shafer's evidential reasoning (DSER) for the combination of information from diversified sources, and overcomes the disadvantages of both approaches. At the same time, we adopt these three approaches: the Bayesian theory, the DSER, and the unified approach to fuse the reports in the system for object recognition and classification, the results are compared and analyzed.

Background: While it is commonly recognized that almost every work and research discipline utilize their own taxonomy, the language used within a specific discipline may also vary depending on numerous factors, including the desired effect of the information being communicated and the intended audience. Different audiences are reached through publication of information, including research results, in different types of publication outlets such as newspapers, newsletters, magazines, websites, and journals. Prior research has shown that students, both undergraduate and graduate, as well as faculty may have a difficult time locating information in different publication outlet types (e.g., magazines, newspapers, journals). The type of publication may affect the ease of understanding and also the confidence placed in the acquired information. A text analytics tool for classifying the source of research as a newsletter (used as a substitute for newspaper articles), a magazine, or an academic journal article has been developed to assist students, faculty, and researchers in identifying the likely source type of information and classifying their own writings with respect to these possible publication outlet types. Literature Review: Literature on information literacy is discussed as this forms the motivation for the reported research. Additionally, prior research on using text mining and text analytics is examined to better understand the methodology employed, including a review of the original Scale of Theoretical and Applied Research system, adapted for the current research. Research Questions: The primary research question is: Can a text mining and text analytics approach accurately determine the most probable publication source type with respect to being from a newsletter, magazine, or journal? Methodology: A text mining and text analytics algorithm, STAR’ (System for Text Analytics-based Ranking), was developed from a previously researched text mining tool, STAR (Scale of Theoretical and Applied Research), that was used to classify the research type of articles between theoretical and applied research. The new text mining method, STAR’, analyzes the language used in manuscripts to determine the type of publication. This method first mines all words from corresponding publication source types to determine a keyword corpus. The corpus is then used in a text analytics process to classify full newsletters, magazine articles, and journal articles with respect to their publication source. All newsletters, magazine articles, and journal articles are from the library and information sciences (LIS) domain. Results: The STAR’ text analytics method was evaluated as a proof of concept on a specific LIS organizational newsletter, as well as articles from a single LIS magazine and a single LIS journal. STAR’ was able to classify the newsletters, magazine articles, and journal articles with 100% accuracy. Random samples from another similar LIS newsletter and a different LIS journal were also evaluated to examine the robustness of the STAR’ method in the initial proof of concept. Following the positive results of the proof of concept, additional journal, magazine, and newsletter articles were used to evaluate the generalizability of STAR’. The second-round results were very positive for differentiating journals and newsletters from other publication types, but revealed potential issues for distinguishing magazine articles from other types of publications. Discussion: STAR’ demonstrates that the language used for transferring information within a specific discipline does differ significantly depending on the intended recipients of the research knowledge. Further work is needed to examine language usage specific to magazine articles. Conclusions: The STAR’ method may be used by students and faculty to identify the likely source of research or discipline-specific information. This may improve trust in the reliability of information due to different levels of rigor applied to different types of publications. Additionally, the STAR’ classifications may be used by students, faculty, or researchers to determine the most appropriate type of outlet and correspondingly the most appropriate type of audience for the reported information in their own manuscripts, thereby improving the chance for successful sharing of information to appropriate audiences who will deem the information to be reliable, through publication in the most relevant outlet type.

A sensor fusion scheme for mobile robot environment recognition that incorporates range data and contour data is proposed. An ultrasonic sensor provides coarse spatial description but guarantees open space (with no obstacle) within a sonic cone with relatively high belief. A laser structured light system provides a detailed contour description of the environment but is prone to light noise and is easily affected by surface reflectivity. We present a sensor fusion scheme that can compensate the disadvantages of both sensors. Line models from the laser structured light system play a key role in environment description. The overall fusion process is composed of two stages: noise elimination and belief updates. Dempster-Shafer's evidential reasoning is applied at each stage. Open space estimation from sonar range measurements brings elimination of noisy lines from the laser sensor. Comparing actual sonar data to the simulated sonar data enables data of two disparate sensors to be fused in the unified feature space. Experimental results demonstrate the effectiveness of the proposed method.

News is new information or information about something that is currently happening, presented in the form of print media, broadcasts, social media or word of mouth to third parties or multiple people. The spread of this news is now very much happening both from social media or messages. However, the problem is that no one can guarantee the truth of the news received. Text mining is the application of data mining concepts and techniques to look for patterns in text, the process of analyzing text to find useful information for a particular purpose. Another definition related to text mining is that text mining is data mining in the form of text where the data source is usually obtained from documents and the goal is to find words that can represent the contents of the document so that an analysis of the connectivity between documents can be carried out. To make it easier to identify news hoax requires a text mining algorithm, using a text mining algorithm can be useful to get real news. Based on the calculations of the TF-IDF algorithm, it can be concluded that the similarity of the meaning of the news whose truth is not yet known is closest to the news ofmerahputih.com with a confidence level of 23.11%.

Gigantic growth of text mining is becoming a potential source of crowd wisdom extraction and analysis especially in terms of text pre-processing and analysis. The ubiquitous growth in the text mining field is unprecedented where social media mining is playing a significant role. Therefore, it’s a significant task of current research purview which not only provides an appropriate platform for the contributors but also for active information seekers. Text mining has gained the widespread attention of internet users which is almost on an explosion level. Accessibility of data on such a behemoth scale mandates regular and critical analysis of this information for various perspectives’ plausibility. Rich patterns of online social text can be exploited to extract relevant information effectively. As text mining plays a significant and crucial role in discovery of these insights, therefore its challenges and contribution to social media analysis must be explored extensively. In this chapter, we provide a brief summary of the modern state of text mining in pre-processing and machine learning mechanism for COVID-19. Emphasis is placed on the resources and learning mechanism available to researchers and practitioners, as well as the major text mining tasks of interest to the community. Thus, the main aim of this chapter is to expound and intellectualize the domains of social media which are accessible on an extraordinary range in the field of text mining for COVID-19.

Data Fusion for ITS: Techniques and Research Needs

It is a big challenge to guarantee the quality of extracted features in text documents to describe user interests or preferences due to large amounts of noise. Over the years, pattern mining-based approaches to RF have attracted great interest to discover knowledge of user interest from text documents. However, the data mining approaches often produce a large set of patterns, which include a lot of noisy patterns, reducing the effective use of pattern mining. In this paper, we present a novel pattern mining approach to RF. This approach mines patterns in both positive and negative feedback and then classifies them into clusters to find user-specific patterns. We also propose a novel pattern deploying method that effectively uses the discovered patterns for improving the performance of searching relevant documents. Experiments are conducted on Reuters Corpus Volume 1 data collection (RCV1) and TREC filtering topics. The results show that the proposed approach achieves promising performance comparing with state-of-the-art term-based methods and pattern-based ones.

BackgroundVast numbers of domestic violence (DV) incidents are attended by the New South Wales Police Force each year in New South Wales and recorded as both structured quantitative data and unstructured free text in the WebCOPS (Web-based interface for the Computerised Operational Policing System) database regarding the details of the incident, the victim, and person of interest (POI). Although the structured data are used for reporting purposes, the free text remains untapped for DV reporting and surveillance purposes.ObjectiveIn this paper, we explore whether text mining can automatically identify mental health disorders from this unstructured text.MethodsWe used a training set of 200 DV recorded events to design a knowledge-driven approach based on lexical patterns in text suggesting mental health disorders for POIs and victims.ResultsThe precision returned from an evaluation set of 100 DV events was 97.5% and 87.1% for mental health disorders related to POIs and victims, respectively. After applying our approach to a large-scale corpus of almost a half million DV events, we identified 77,995 events (15.83%) that mentioned mental health disorders, with 76.96% (60,032/77,995) of those linked to POIs versus 16.47% (12,852/77,995) for the victims and 6.55% (5111/77,995) for both. Depression was the most common mental health disorder mentioned in both victims (22.25%, 3269) and POIs (18.70%, 8944), followed by alcohol abuse for POIs (12.19%, 5829) and various anxiety disorders (eg, panic disorder, generalized anxiety disorder) for victims (11.66%, 1714).ConclusionsThe results suggest that text mining can automatically extract targeted information from police-recorded DV events to support further public health research into the nexus between mental health disorders and DV.

BackgroundInternet is a particularly dynamic way to quickly capture the perceptions of a population in real time. Complementary to traditional face-to-face communication, online social networks help patients to improve self-esteem and self-help.ObjectiveThe aim of this study was to use text mining on material from an online forum exploring patients’ concerns about treatment (antidepressants and anxiolytics).MethodsConcerns about treatment were collected from discussion titles in patients’ online community related to antidepressants and anxiolytics. To examine the content of these titles automatically, we used text mining methods, such as word frequency in a document-term matrix and co-occurrence of words using a network analysis. It was thus possible to identify topics discussed on the forum.ResultsThe forum included 2415 discussions on antidepressants and anxiolytics over a period of 3 years. After a preprocessing step, the text mining algorithm identified the 99 most frequently occurring words in titles, among which were escitalopram, withdrawal, antidepressant, venlafaxine, paroxetine, and effect. Patients’ concerns were related to antidepressant withdrawal, the need to share experience about symptoms, effects, and questions on weight gain with some drugs.ConclusionsPatients’ expression on the Internet is a potential additional resource in addressing patients’ concerns about treatment. Patient profiles are close to that of patients treated in psychiatry.

The red swamp crayfish (Procambarus clarkii) is the most widely introduced crayfish species worldwide. Its spread causes massive environmental impacts on native species and ecosystems. Owing to its commercial value and impacts in introduced countries, the literature on P. clarkii has increased exponentially, making necessary studies for synthesising the collected information. In this study, a text mining and topic modelling approach was used to analyse the scientific literature on P. clarkii as an invasive species, to identify the interests and trends of research lines through time and space. A keyword-based search of research databases was performed, and the abstracts of 519 articles were analysed to identify the most characterising terms. These included both descriptive terms of the species and terms related to its invasiveness and impacts. The analysis of the scientific literature allowed us to identify 14 research topics focusing on the ecological impacts of P. clarkii and its geographical distribution, with a growing interest in invasion mechanisms and dispersal pathways. On the contrary, management studies are less abundant, despite their scientific value is highlighted by their increasing number of citations. This reflects the fact that most management-related studies are found in the grey literature and are therefore underrepresented in the scientific literature. The limited accessibility to data and the complexity of control strategies are obstacles to the effective management of this species. Improvement in information sharing through the creation of digital platforms that support distribution data suitable for modelling species spread, and the integration of knowledge from non-academic sources may help fill current knowledge gaps and provide new perspectives for the management of P. clarkii.

A very large percentage of business and academic data is stored in textual format. With the exception of metadata, such as author, date, title and publisher, these data are not overtly structured like the standard, mainly numerical, data in relational databases. Parallel to data mining, which finds new patterns and trends in numerical data, text mining is the process aimed at discovering unknown patterns in free text. Owing to the importance of competitive and scientific knowledge that can be exploited from these texts, “text mining has become an increasingly popular and essential theme in data mining” (Han & Kamber, 2001, p. 428).