Challenges from Clustering Analysis to Knowledge Discovery in Molecular Biomechanics

Chapter 14 - Knowledge Discovery and Data Mining

In recent years, the exponential growth of both structured and unstructured data has highlighted the importance of efficient knowledge discovery techniques. Among these, image mining and data mining play pivotal roles in extracting meaningful patterns and insights from different forms of information. At the same time, data mining focuses on structured datasets such as databases and transaction records, while image mining deals with unstructured visual content, requiring advanced feature extraction, pattern recognition, and machine learning methods. This paper presents a comparative analysis of image mining and data mining techniques, emphasising their methodologies, applications, and challenges. The comparison explores key dimensions, including data preprocessing, feature selection, algorithmic approaches, and application domains such as healthcare, security, business intelligence, and multimedia. Experimental results show clustering accuracy of 87% for data mining and image classification accuracy of 92% for image mining, highlighting the effectiveness of specialised approaches. The study highlights both the similarities and differences in knowledge discovery processes, demonstrating how integrating image mining and data mining can enhance decision-making in diverse fields. The findings provide a comprehensive understanding of their complementary roles, offering valuable insights for researchers and practitioners aiming to develop hybrid approaches to knowledge discovery.

Analysing datasets using data mining techniques can enhance decision making in organizations. However, to ensure that the full potential of these techniques is realised it is important that decision makers understand there are Knowledge Discovery and Data Mining (KDDM) processes that are mature enough to be adopted. This paper demonstrates the benefits of using a KDDM process to evaluate survey data for internet banking users in Jamaica which includes demographic as well as attitudinal and behavioral variables. The major benefits of following this process include the selection of a set of models, rather than a single model, which are more relevant to the business/research objectives and use of a more targeted knowledge discovery process as the data mining analyst is now directed to consider the effects the decisions in each phase will have on subsequent phases. This leads to more relevant knowledge being extracted from the data mining process.

Data Mining refers to a set of methods applicable to large and complex databases to eliminate the randomness and discover the hidden pattern. Datamining (DM), also known as knowledge discovery from databases (KDD), is the extraction of new knowledge from huge databases. Data mining involves the use of sophisticated data analysis tools to discover previously unknown, valid patterns and relationships in large data sets. Data mining tools can forecast the future trends and activities to support the decision of people. The scope of datamining is associated with Uncovering trends and patterns are a great power for the businesses of all sectors and industries. Modern intrusion detection applications are confronted with a variety of issues. These applications must be reliable, extensible, manageable, and minimal in maintenance costs. Data mining-based intrusion detection systems (IDSs) have shown high accuracy, good generalisation to novel types of intrusion, and stable behaviour in a changing environment in recent years. The number of hidden layers in various neural network topologies is evaluated in order to discover the best neural network. The technique of attempting to discover instances of network attacks by comparing current behaviour to the expected actions of an intruder is known as misuse detection. Artificial neural networks have the ability to detect and classify network activity using data that is limited, incomplete, and nonlinear. The main purpose of this work is to identify privacy and security concerns among cloud computing participants and consumers in a distributed environment. Techniques like Machine Learning, Natural Language Processing (NLP), and Data Mining are combined to automatically identify and uncover patterns from many sorts of materials. Predictive analytics is capable of dealing with both continuous and discontinuous changes. Classification, prediction, and to some extent, affinity analysis constitute the analytical methods employed in predictive analytics. The contemporary study in text or document mining is focusing on syntactic components and the semantic environment. In order to accomplish this, and with the motivation gained from our previous research contributions, we investigated a mining model to classify documents based on the Order of Context, Concept, and Semantic Relations (OCCSR). The use of data mining techniques based on Cloud computing will enable users to retrieve meaningful information from virtually integrated data warehouses, lowering infrastructure and storage costs. Data mining can extract useful and potentially useful information from the cloud. Big Data is typically defined by three characteristics known as the 3Vs (Volume, Velocity and Variety). The surveys approaches, environments, and technologies in key areas for Big Data analytics capabilities and discusses how they aid in the development of analytics solutions for Clouds. The clustering technique belongs to an unsupervised learning and it is used to discover a new set of categories. Grid-based clustering has the shortest processing time, which is typically determined by the size of the grid rather than the data. We compare the performance of three clustering algorithms: hierarchical clustering, density-based clustering, and K Means clustering. The majority of current approaches to detecting misuse involve the use of rule-based expert systems to identify indicators of known attacks. We provide a brief overview of the use of various Artificial Intelligence techniques and their advancements in the design, development, and application of Intrusion Detection Systems (IDS) for protecting computer and communication networks from intruders. The goal of Knowledge Discovery in Data (KDD) is to extract information that is not obvious by using careful and detailed analysis and interpretation. To drive decisions and actions, analytics employs KDD, data mining, text mining, statistical and quantitative analysis, explanatory and predictive models, and advanced and interactive visualisation techniques.

The voluminous amount of data stored in databases contains hidden knowledge which could be valuable to improve decision making process of any organization. As it is not humanely possible to analyze large databases, it has become essential to apply advanced data mining algorithms for extracting patterns (models) from data to support decision making. A number of data mining algorithms produce information of a statistical nature that allows the user to assess how accurate and reliable the discovered knowledge is? However, in many cases this is not enough for the users. Even if the discovered knowledge is highly accurate from a statistical point of view, it might not be interesting to the user. Therefore the process of knowledge discovery in databases (KDD) aims at discovering knowledge that is interesting and useful to the user. Most of the data mining algorithms so far have paid lot of attention to discovery of accurate and comprehensible knowledge. Though, the question of interestingness has been addressed time to time, it is being increasingly realized by data mining community that this subject needs a renewed focus. This paper is an attempt to review the measures of interestingness used in the data mining literature. The main contribution of the paper is to improve the understanding of interestingness measures for discovery of knowledge and identify the unresolved problems to set the directions for the future research in this area.

The real-time system of the JT-60 has a cascade structure with the major and the minor control loops. The multi-variable feedback control is adopted in the major control loop in order to adjust non-linear effects which will appear in the high-/beta/region. By applying the concept of 'phase shift control' to the control program of the major control loop, a flexible tokamak discharge control is expected to be performed with taking account of machine protection.

Data Mining is a process of exploring the huge data in search of reliable patterns and methodical relationship among variables. As a result, the findings may be validated through applying the detected patterns to a novel subset of the data. In simple words, Data Mining is referred as extracting the useful information as large dataset and transforming into reliable structure for future use. Data Mining has shown its incredible performance in various fields to a greater extent, out of which, Educational Data Mining (EDM) is one among them. Many researchers have addressed huge number of problems in EDM and applied various techniques to reveal the useful and hidden information that helped in the process of decision making. Students getting employed during and after graduation are one of the important parts of their life. Students, based on their academic performances, are getting employed in companies they deserve. But still, the probability of getting employed is very less in this competitive world. In this paper, a real-time scenario has been chosen for analyzing various factors for getting employed/unemployed. Various clustering and classification techniques have been implemented and their performances are studied. A hybrid approach is presented in this paper that integrates the benefits of particle swarm optimization (PSO) and fuzzy clustering means (FCMs). The results obtained show that the proposed technique helps in obtaining higher accuracy to other clustering techniques. The proposed clustering algorithm PSO-FCM, accuracy is 34.4%, 36.45% and 28.45% higher than the existing method, time complexity shows 45%, 33% and 49% lower than the existing [Formula: see text]-means clustering, Naïve Bayes clustering and SVM clustering algorithms, respectively.

Data mining, also known as knowledge discovery in databases, is a statistical analysis technique used to find hidden patterns and identify untapped value in large datasets. Clustering is a principal data discovery technique in data mining that segregates a dataset into subsets or clusters so that data values in the same cluster have some common characteristics or attributes. A number of clustering techniques have been proposed in the past by many researchers that can identify arbitrary shaped cluster; where a cluster is defined as a dense region separated by the low-density regions and among them DBSCAN is a prime density-based clustering algorithm. DBSCAN is capable of discovering clusters of any arbitrary shape and size in databases which even include noise and outliers. Many researchers have attempted to overcome certain deficiencies in the original DBSCAN like identifying patterns within datasets of varied densities and its high computational complexity; hence a number of augmented forms of DBSCAN algorithm are available. We present an incremental density-based clustering technique which is based on the fundamental DBSCAN clustering algorithm to enhance its computational complexity. Our proposed algorithm can be used in different knowledge domains like image processing, classification of patterns in GIS maps, x-ray crystallography and information security.

Nowadays doctors are generating a huge amount of raw data. These data, analyzed with data mining techniques, could be sources of new knowledge. Unluckily such tasks need skilled data analysts, and not so much researchers in Medicine are also data mining experts. In this paper we present a web based system for knowledge discovery assistance in Medicine able to advice a medical researcher in this kind of tasks. The user must define only the experiment specifications in a formal language we have defined. The system GUI helps users in their composition. Then the system plans a Knowledge Discovery Process (KDP) on the basis of rules in a knowledge base. Finally the system executes the KDP and produces a model as result. The system works through the co-operation of different web services specialized in different tasks. The system is still under development.

Computational scientists must understand results from experimental, observational and computational simulation generated data to gain insights and perform knowledge discovery. As systems approach the petascale range, problems that were unimaginable a few years ago are within reach. With the increasing volume and complexity of data produced by ultra-scale simulations and high-throughput experiments, understanding the science is largely hampered by the lack of comprehensive I/O, storage, acceleration of data manipulation, analysis, and mining tools. Scientists require techniques, tools and infrastructure to facilitate better understanding of their data, in particular the ability to effectively perform complex data analysis, statistical analysis and knowledge discovery. The goal of this work is to enable more effective analysis of scientific datasets through the integration of enhancements in the I/O stack, from active storage support at the file system layer to MPI-IO and high-level I/O library layers. We propose to provide software components to accelerate data analytics, mining, I/O, and knowledge discovery for large-scale scientific applications, thereby increasing productivity of both scientists and the systems. Our approaches include 1) design the interfaces in high-level I/O libraries, such as parallel netCDF, for applications to activate data mining operations at the lower I/O layers; 2) Enhance MPI-IO runtime systems to incorporate the functionality developed as a part of the runtime system design; 3) Develop parallel data mining programs as part of runtime library for server-side file system in PVFS file system; and 4) Prototype an active storage cluster, which will utilize multicore CPUs, GPUs, and FPGAs to carry out the data mining workload.

Pattern mining algorithms for data streams using itemset

Data mining (sometimes called data or knowledge discovery) is the process of analyzing data from different perspectives and summarizing it into useful information - information that can be used to increase revenue, cuts costs, or both. Data mining software is one of a number of analytical tools for analyzing data. It allows users to analyze data from many different dimensions or angles, categorize it, and summarize the relationships identified. Technically, data mining is the process of finding correlations or patterns among dozens of fields in large relational databases. The term data mining is often used to apply to the two separate processes of knowledge discovery and prediction. Knowledge discovery provides explicit information that has a readable form and can be understood by a user (e.g., association rule mining). Forecasting, or predictive modeling provides predictions of future events and may be transparent and readable in some approaches (e.g., rule-based systems) and opaque in others such as neural networks. Moreover, some data-mining systems such as neural networks are inherently geared towards prediction and pattern recognition, rather than knowledge discovery. In Future different Scope of data mining are. 1. Developing a unifying theory of data mining. 2. Scaling up for high dimensional data and high speed data streams. 3. Data mining in a network setting. 4. Data mining for biological and environmental problems. 5. Security, privacy and data integrity. 6. Dealing with non-static, unbalanced and cost-sensitive data

The aim of this study is to highlight the importance of the unsupervised learning in Data mining and CRM fields. Data mining commonly known by its acronym KDD: knowledge discovery in data base, it refers to all methods and algorithms used for data exploration or prediction in large data bases volumes, Data mining is very important in various fields such as science, business and other areas deal with a large data set. CRM: Customer Relationship Management is an integrated information system that is used to plan, schedule and control the pre-sales and post-sales activities in an organization, both CRM and data mining techniques helps organizations maximize the value of every customer interaction and drive superior corporate performance. Clustering is one of the favoured used methods in data mining: The objective of this study is to implement the clustering algorithm K-Medoids via a shell script applied on a set of Analytical CRM data stored in Teradata environment.

This article presents a case study involving data mining (DM) and the knowledge discovery (KD) process for the Western US power grid. With the installation of phasor measurement units (PMUs) in the US power systems, there is potential to improve wide-area situational awareness using data analytics. The KD process is challenging due to the complex nature of the power grid, data propriety issues, the file size of the time-series data, and questions of how to extract meaningful features. This case study demonstrates how the KD process can be implemented in light of these challenges to detect the anomalies of interest.

Adding the dimension of time to databases produces time series databases (TSDB) and introduces new aspects and difficulties to data mining and knowledge discovery. In this correspondence, we introduce a general methodology for knowledge discovery in TSDB. The process of knowledge discovery in TSDR includes cleaning and filtering of time series data, identifying the most important predicting attributes, and extracting a set of association rules that can be used to predict the time series behavior in the future. Our method is based on signal processing techniques and the information-theoretic fuzzy approach to knowledge discovery. The computational theory of perception (CTP) is used to reduce the set of extracted rules by fuzzification and aggregation. We demonstrate our approach on two types of time series: stock-market data and weather data.