Knowledge discovery in time series databases

Time series data is a series of observation data accord- ing to a certain time sequence. It has been penetrate various field. This paper applies Rough set to the knowledge dis- covery of time series. The process of knowledge discovery in time series includes preprocessing of time series data, at- tributes selection and similarity sequence searching. Then, the time series is partitioned to a set of pattern(each pattern represents a trend of time series)by mobile window method. An information table is formed by the most important pre- dicting attributes and target attribute which in the trend- ing structure sequence identified from each pattern. This information table is suitable for the Rough set to discover knowledge. The extracted rules can predict the time series behavior in the future. We demonstrate our method on time series stock market data.

Notice of Violation of IEEE Publication Principles<br><br>"Knowledge Discovery from Virtual Enterprise Model Based on Semantic Annotation,"<br>by Chengzhu Sun, Xiaofei Xu, Xiangyang Li, Shengchun Deng,<br>in the Proceedings of the Fifth International Conference on Fuzzy Systems and Knowledge Discovery, 2008. FSKD '08, vol.5, pp.546-551, Oct. 2008<br><br>After careful and considered review of the content and authorship of this paper by a duly constituted expert committee, this paper has been found to be in violation of IEEE's Publication Principles.<br><br>This paper contains significant portions of original text from the paper cited below. The original text was copied without attribution (including appropriate references to the original author(s) and/or paper title) and without permission.<br><br>"A Controlled Language for Semantic Annotation and Interoperability in e-Business Applications"<br>by M.Missikoff, F.Schiappelli , F.Taglino<br>in the Proceedings of the Semantic Integration Workshop, collocated with the Second International Semantic Web Conference (ISWC-03), October 2003<br><br> <br/> Discovering knowledge from virtual enterprise model is becoming increasingly important, as numerical models established for virtual enterprise are difficult to support interoperability of virtual enterprise. To solve this problem, a knowledge discovery method based on semantic annotation is put forward in this paper. A graphic ontology representation schema for virtual enterprise model is described, which is called ontology structure graph (OSG). Based on applications of semantic annotation, the process of knowledge discovery in the virtual enterprise model base is given and activities such as model selection, semantic annotation, data transformation, knowledge extraction and ontology interoperation in knowledge discovery process are illustrated in detail. Then, several critical issues influencing knowledge discovery are explained, including the organization of domain vocabulary, the definition of semantic annotation rules and semantic affinity function, the formulation of reference ontology. Finally, an instance is given to demonstrate the knowledge discovery method and results of knowledge discovery are presented.

Assessing reliability of Big Data Knowledge Discovery process

This paper presents our ongoing work in the Virtual Interiors project, which aims to develop 3D reconstructions as geospatial interfaces to structure and explore historical data of seventeenth-century Amsterdam. We take the reconstruction of the entrance hall of the house of the patrician Pieter de Graeff (1638–1707) as our case study and use it to illustrate the iterative process of knowledge creation, sharing, and discovery that unfolds while creating, exploring and experiencing the 3D models in a prototype research environment. During this work, an interdisciplinary dataset was collected, various metadata and paradata were created to document both the sources and the reasoning process, and rich contextual links were added. These data were used as the basis for creating a user interface for an online research environment, taking design principles and previous user studies into account. Knowledge is shared by visualizing the 3D reconstructions along with the related complexities and uncertainties, while the integration of various underlying data and Linked Data makes it possible to discover contextual knowledge by exploring associated resources. Moreover, we outline how users of the research environment can add annotations and rearrange objects in the scene, facilitating further knowledge discovery and creation.

Similarity join on time series databases is an essential operation for data analysis applications. Due to the curse of dimensionality, it is not suitable to use traditional index techniques, such as R-tree and kd-tree. In the paper, a dynamic segment index (i.e., DSTree) is utilized to reduce the huge comparison cost on the similarity join on time series databases. However, the DSTree is designed for similarity search and only supports bound estimations between a time series and a batch of time series in a DSTree node. To make the DSTree suitable for the similarity join on time series databases, it is necessary to have tight bounds for the nodes to achieve a better pruning power, where the biggest challenge is that the DSTree nodes may have different segmentations. To solve the problem aforementioned, a segmentation alignment and synopsis evaluation method is proposed to support the estimation of DSTree nodes to significantly reduce the time cost by pruning unnecessary comparisons. Moreover, to make our approach I/O efficient, a caching strategy is proposed by taking advantage of both graph partitioning and the locality of the DSTree index. The efficiency and effectiveness of the proposed approaches are verified by experiments on real-life datasets.

Knowledge discovery in databases is a process that aims at the discovery of associations within data sets. The analysis of geo-referenced data demands a particular approach in this process. This chapter presents a new approach to the process of knowledge discovery, in which qualitative geographic identifiers give the positional aspects of geographic data. Those identifiers are manipulated using qualitative reasoning principles, which allows for the inference of new spatial relations required for the data mining step of the knowledge discovery process. The efficacy and usefulness of the implemented system — Padrão — has been tested with a bank dataset. The results support that traditional knowledge discovery systems, developed for relational databases and not having semantic knowledge linked to spatial data, can be used in the process of knowledge discovery in geo-referenced databases, since some of this semantic knowledge and the principles of qualitative spatial reasoning are available as spatial domain knowledge.

PhilDB is an open-source time series database that supports storage of time series datasets that are dynamic, that is it records updates to existing values in a log as they occur. PhilDB eases loading of data for the user by utilising an intelligent data write method. It preserves existing values during updates and abstracts the update complexity required to achieve logging of data value changes. It implements fast reads to make it practical to select data for analysis. Recent open-source systems have been developed to indefinitely store long-period high-resolution time series data without change logging. Unfortunately such systems generally require a large initial installation investment before use because they are designed to operate over a cluster of servers to achieve high-performance writing of static data in real time. In essence, they have a 'big data' approach to storage and access. Other open-source projects for handling time series data that avoid the 'big data' approach are also relatively new and are complex or incomplete. None of these systems gracefully handle revision of existing data while tracking values that changed. Unlike 'big data' solutions, PhilDB has been designed for single machine deployment on commodity hardware, reducing the barrier to deployment. PhilDB takes a unique approach to meta-data tracking; optional attribute attachment. This facilitates scaling the complexities of storing a wide variety of data. That is, it allows time series data to be loaded as time series instances with minimal initial meta-data, yet additional attributes can be created and attached to differentiate the time series instances when a wider variety of data is needed. PhilDB was written in Python, leveraging existing libraries. While some existing systems come close to meeting the needs PhilDB addresses, none cover all the needs at once. PhilDB was written to fill this gap in existing solutions. This paper explores existing time series database solutions, discusses the motivation for PhilDB, describes the architecture and philosophy of the PhilDB software, and performs a simple evaluation between InfluxDB, PhilDB, and SciDB.

Analysing datasets using data mining techniques can enhance decision making in organizations. However, to ensure that the full potential of these techniques is realised it is important that decision makers understand there are Knowledge Discovery and Data Mining (KDDM) processes that are mature enough to be adopted. This paper demonstrates the benefits of using a KDDM process to evaluate survey data for internet banking users in Jamaica which includes demographic as well as attitudinal and behavioral variables. The major benefits of following this process include the selection of a set of models, rather than a single model, which are more relevant to the business/research objectives and use of a more targeted knowledge discovery process as the data mining analyst is now directed to consider the effects the decisions in each phase will have on subsequent phases. This leads to more relevant knowledge being extracted from the data mining process.

Chapter 14 - Knowledge Discovery and Data Mining

The voluminous amount of data stored in databases contains hidden knowledge which could be valuable to improve decision making process of any organization. As it is not humanely possible to analyze large databases, it has become essential to apply advanced data mining algorithms for extracting patterns (models) from data to support decision making. A number of data mining algorithms produce information of a statistical nature that allows the user to assess how accurate and reliable the discovered knowledge is? However, in many cases this is not enough for the users. Even if the discovered knowledge is highly accurate from a statistical point of view, it might not be interesting to the user. Therefore the process of knowledge discovery in databases (KDD) aims at discovering knowledge that is interesting and useful to the user. Most of the data mining algorithms so far have paid lot of attention to discovery of accurate and comprehensible knowledge. Though, the question of interestingness has been addressed time to time, it is being increasingly realized by data mining community that this subject needs a renewed focus. This paper is an attempt to review the measures of interestingness used in the data mining literature. The main contribution of the paper is to improve the understanding of interestingness measures for discovery of knowledge and identify the unresolved problems to set the directions for the future research in this area.

Throughout endless experimental work, short records of dynamic molecular data are generated from time to time. Biomechanics data mining and knowledge discovery have become an important study area to turn the abundance of generated raw data into pieces of information. In data mining, researchers often encounter challenging issues and constraints, ranging from nature of the collected microarray data and developed clustering algorithms to informative discovery for rhythmic data decision-making processes. This article presents the review of the commonly practiced clustering techniques in molecular biomechanical systems towards better applications in bioengineering research. It highlights the constraints and challenges encountered in temporal molecular bioengineering mechanisms. The findings revealed that the molecular data are commonly analyzed based on data mining computation and mathematical applications to link both developmental stages interfaces and the mechanical principles of living organisms. In this area, mathematical analyses are extensively carried out to investigate dynamic microarray using clustering techniques. The main goal is to extract informative knowledge. Therefore, in order to derive collective patterns and reliable information from microarray, there is a need to consider effects from the nature of data, clustering algorithms and knowledge discovery processes which require substantial understanding on biological systems. Keywords: Clustering, data mining, gene expression, information, knowledge discovery, molecular biomechanics, molecular data, bioengineering mechanisms, mathematical applications, microarray data, Clustering Algorithms.

This paper develops a knowledge discovery procedure for extracting knowledge of soil-landscape models from a soil map. It has broad relevance to knowledge discovery from other natural resource maps. The procedure consists of four major steps: data preparation, data preprocessing, pattern extraction, and knowledge consolidation. In order to recover true expert knowledge from the error-prone soil maps, our study pays specific attention to the reduction of representation noise in soil maps. The data preprocessing step has exhibited an important role in obtaining greater accuracy. A specific method for sampling pixels based on modes of environmental histograms has proven to be effective in terms of reducing noise and constructing representative sample sets. Three inductive learning algorithms, the See5 decision tree algorithm, Naïve Bayes, and artificial neural network, are investigated for a comparison concerning learning accuracy and result comprehensibility. See5 proves to be an accurate method and produces the most comprehensible results, which are consistent with the rules (expert knowledge) used in producing the soil map. The incorporation of spatial information into the knowledge discovery process is found not only to improve the accuracy of the extracted knowledge, but also to add to the explicitness and extensiveness of the extracted soil-landscape model.

In this paper we deal with association rule mining in the context of a complex, interactive and iterative knowledge discovery process. After a general introduction covering the basics of association rule mining and of the knowledge discovery process in databases we draw the attention to the problematic aspects concerning the integration of both. Actually, we come to the conclusion that with regard to human involvement and interactivity the current situation is far from being satisfying. In our paper we tackle this problem on three sides: First of all there is the algorithmic complexity. Although today’s algorithms efficiently prune the immense search space the achieved run times do not allow true interactivity. Nevertheless we present a rule caching schema that significantly reduces the number of mining runs. This schema helps to gain interactivity even in the presence of extreme run times of the mining algorithms. Second, today the mining data is typically stored in a relational database management system. We present an efficient integration with modern database systems which is one of the key factors in practical mining applications. Third, interesting rules must be picked from the set of generated rules. This might be quite costly because the generated rule sets normally are quite large whereas the percentage of useful rules is typically only a very small fraction. We enhance the traditional association rule mining framework in order to cope with this situation.KeywordsAssociation RuleMining AlgorithmFrequent ItemsetsAssociation Rule MiningGeneralize Association RuleThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The knowledge discovery and data mining (KDDM) process models describe the various phases (e.g. business understanding, data understanding, data preparation, modeling, evaluation and deployment) of the KDDM process. They act as a roadmap for implementation of the KDDM process by presenting a list of tasks for executing the various phases. The checklist approach of describing the tasks is not adequately supported by appropriate tools, which specify ‘how’ the particular task can be implemented. This may result in tasks not being implemented. Another disadvantage is that the long checklist does not capture or leverage the dependencies that exist among the various tasks of the same and different phases. This not only makes the process cumbersome to implement, but also hinders possibilities for semi-automation of certain tasks. Given that each task in the process model serves an important goal and even affects the execution of related tasks due to the dependencies, these limitations are likely to negatively affect the efficiency and effectiveness of KDDM projects. This paper proposes an improved KDDM process model that overcomes these shortcomings by prescribing tools for supporting each task as well as identifying and leveraging dependencies among tasks for semi-automation of tasks, wherever possible.

This article presents a case study involving data mining (DM) and the knowledge discovery (KD) process for the Western US power grid. With the installation of phasor measurement units (PMUs) in the US power systems, there is potential to improve wide-area situational awareness using data analytics. The KD process is challenging due to the complex nature of the power grid, data propriety issues, the file size of the time-series data, and questions of how to extract meaningful features. This case study demonstrates how the KD process can be implemented in light of these challenges to detect the anomalies of interest.