Incremental Mining Algorithm of Sequential Patterns Based on Sequence Tree

Sequential pattern mining is an important mining task for discovering sequential patterns along with their insight relationships in many real-world applications. In practice, sequence databases are kept changing over the time along with their business. For some reasons, some sequences in the database are asked to be deleted from the database. In order to have a synchronization of discovered sequential patterns with the database from which they have been discovered, the sequential pattern mining task is re-considered with many challenges. As the number of deleted sequences is often smaller than the size of the entire database, re-mining from scratch the updated database might incur a high cost because sequential pattern mining is a computationally expensive task. In this paper, our work aims at an efficient incremental mining solution to the sequential pattern mining task with sequence deletions. Different from the existing works, we propose an expanded prefix tree by extending the existing prefix tree with additional structures for capturing more necessary information for the incremental mining process. Based on this tree, we propose an incremental sequential pattern mining algorithm, SPMD, for finding a complete set of sequential patterns with no re-scanning the original database, when a number of sequences in the database are deleted. Experimental results on the benchmark databases have confirmed that our SPMD algorithm outperforms the re-mining from scratch with the PrefixSpan algorithm with less running time.

Real life sequential databases are usually not static. They tend to grow incrementally. So after every update a frequent pattern may no longer remains frequent while some infrequent patterns may appear as frequent in updated database. It is not a good idea to mine sequential database from scratch every time as the update occurs. It would be better if we can use the knowledge of already mined sequential patterns to find the complete set of sequential patterns for updated database. An incremental mining algorithm does the same thing. The main goal of an incremental mining algorithm is to reduce the time taken to find out the frequent patterns significantly i.e. it should mine the set of frequent patterns in significantly less time than a non-incremental mining algorithm. In this work we have approached using weight constraints, in time and space, of an idea of already existing algorithm called WSM.

Common sequential pattern mining algorithms handle static databases. Once the data change, the previous mining result will be incorrect, and we need to restart the entire mining process for the new updated sequence database. Previous approaches, within either Apriori-based or projection-based framework, mine patterns in a forward manner. Considering the incremental characteristics of sequence-merging, we develop a novel technique, called backward mining, for efficient incremental pattern discovery. We propose an algorithm, called BSPinc, for incremental mining of sequential patterns using a backward mining strategy. Stable sequences, whose support counts remain unchanged in the updated database, are identified and eliminated from the support counting process. Candidate sequences generated using backward extensions can be mined recursively within the ever-shrinking space of the projected sequences. The experimental results show that BSPinc worked an average of 2.5 times faster than the well-known IncSpan algorithm and outperformed SPAM an average of 3 times faster.

In this paper, we propose a new SQL based incremental distributed algorithm for predicting the next location of a mobile user in a mobile web environments. Parallel and Distributed data mining algorithm is applied on moving logs stored in geographically distributed data grid to generate the mobility pattern, which provides various location based services to the mobile users. One of the existing works for deriving mobility pattern is re-executing the algorithm from scratch results in excessive computation. In our work, we have designed new incremental algorithm by maintaining infrequent mobility patterns, which avoids unnecessary scan of full database. We built data grid system on a cluster of workstation using open source Globus Toolkit (GT) and Message Passing Interface extended with Grid Services (MPICH-G2). The experiments were conducted on original data sets with incremental addition of data and the computation time was recorded for each data sets. We analyzed our results with various sizes of data sets and it shows the time taken to generate mobility pattern by incremental mining algorithm is less than re-computing approach.

Among various data mining techniques, sequential-pattern mining is used to discover the frequent subsequences from a sequence database. Most research handles the static database in batch mode to discover the desired sequential patterns. Transactions or customer sequences are, however, dynamically changed in real-world applications. In the past, the FUSP tree was designed to maintain and update the discovered information based on Fast UPdated (FUP) approach with sequence insertion and sequence deletion. The original customer sequences is still required to be rescanned if it is necessary. In this paper, the prelarge concept is adopted to maintain and update the built FUSP tree with sequence deletion. When the number of deleted customers is smaller than the safety bound of the prelarge concept, the original database is unnecessary to be rescanned but the sequential patterns can still be actually maintained and updated. Experiments are also conducted to show the performance of the proposed algorithm in terms of execution time and number of tree nodes.Keywordsprelarge conceptsequential patternsdeletiondynamic databasesFUSP tree

Comparative analysis of sequence weighting approaches for mining time-interval weighted sequential patterns

Mining frequent sequential patterns from sequence databases has been a central research topic in data mining and various efficient mining sequential patterns algorithms have been proposed and studied. Recently, a novel sequential pattern mining research, called mining repetitive gapped subsequences, has attracted the attention of many researchers. However, the number of repetitive gapped subsequences generated by even these closed mining algorithms may be too large to understand for users, especially when support threshold is low. In this paper, we propose the problem of how to compress repetitive gapped sequential patterns. A novel distance measure of repetitive gapped sequential patterns and an efficient representative pattern checking scheme, δ-dominate sequential pattern checking are proposed. We also develop an efficient algorithm, CRGSgrow ( Compressing Repetitive Gapped Sequential pattern grow), including an efficient pruning strategy, SyncScan. An empirical study with both real and synthetic data sets clearly shows that the CRGSgrow has good performance.Keywordsrepetitive gapped sequential patterncompressing frequent patterns

The purpose of mining sequential patterns problem with weighted constraints is to find high-valued patterns, including infrequent patterns but having items which appear in the pattern of high importance in the sequence database (SD). Therefore, weighted sequential pattern mining will collect a set of more complete patterns with items of low support but of high importance. This paper proposes a new algorithm called WSPM_PreTree to find highly weighted sequential patterns. To collect a set of complete sequential patterns with the stricter weighted constraints of sequential patterns, the proposed algorithm uses both the minimum support constraint and the actual values of items appearing in the SD. To increase the performance of the finding weighted sequential patterns process, the algorithm uses the parent–child relationship on the prefix tree structure to create candidates and combines the weighted mean of the sequential 1-patterns that is calculated from the actual value of items in the SD as conditions to find the weighted sequential patterns. Experimental results show that the proposed algorithm is more efficient than sequential patterns mining with weight constraint (SPMW) algorithm [Ref. 20 ] in the runtime.

Web mining is one of the mining technologies, which applies data mining techniques in large amount of web log data. Web navigational mining discovers users’ access patterns from web logs. This information can be used to identify the behavior of the web user. However, the web data will grow rapidly in the short time, and some of the web data may be antiquated. The user behavior may be changed when the new web data is inserted into and the old web data is deleted from web logs. Therefore, the user behavior must be re-discovered from the updated web logs. However, it is very time-consuming to re-find the users’ access patterns. Hence, many researchers pay attention to the incremental mining, which utilizes the previous mining results and finds new patterns just from the inserted or deleted part of the web logs such that the mining time can be reduced. The present paper proposes an efficient incremental web navigational mining algorithm for discovering web navigational patterns when the user sequences are inserted into and deleted from original database. It avoids re-finding the original web navigational patterns and re-counting the original candidate sequences. It uses lattice structure to keep the previous mining results such that just new candidate sequences need to be computed. Hence, the web navigational patterns can be obtained rapidly when the navigational sequence database is updated. Besides, maximal web navigational patterns can also be obtained easily by traversing the lattice structure. The experimental results show that the present algorithm is more efficient than the other approaches.

Incremental algorithms can manipulate the results of earlier mining to derive the final mining output in various businesses. This study proposes a new algorithm, called the New Fast UPdate algorithm (NFUP) for efficiently incrementally mining association rules from a large transaction database. NFUP is a backward method that only requires scanning incremental database. Rather than rescanning the original database for some new generated frequent itemsets in the incremental database, we accumulate the occurrence counts of newly generated frequent itemsets and delete infrequent itemsets obviously. Thus, NFUP need not rescan the original database and to discover newly generated frequent itemsets. NFUP has good scalability in our simulation.

Efficient algorithms for incremental maintenance of closed sequential patterns in large databases

In this paper, we propose an incremental clustering algorithm called INCLUS for high dimensional sparse transactional data using a newly defined similarity measure and a notion of cluster representatives based on locally frequent items of each cluster. INCLUS seeks structures in transactional data with respect to the support and similarity constraints specified by the users. The effectiveness and the order-independence property of INCLUS are empirically studied and compared with two state- of-art algorithms. Though it is a one-pass algorithm without any iterative refinement, INCLUS is not only effective and scalable, but also insensitive to the order of transactions, which is a crucial property for an incremental algorithm. globally frequent items are neglected during clustering, clusters embedded in such transactions cannot be discovered. In this paper, we define a new notion of cluster representative based on frequent items within each cluster that is easy to comprehend by users, and introduce a new similarity measure especially suitable for sparse transactional data. Using these, an incremental clustering algorithm INCLUS is proposed to cope with high cardinality of transactional datasets. INCLUS is a structure seeking algorithm as it produces clusters based on the support and similarity constraints specified by the users. The domains of input parameters (support and similarity thresholds) are well defined and can be easily chosen by users. INCLUS is significantly different from LargeItem, SUMMARY and TrK-Means which also employ frequent items for clustering. LargeItem uses global optimization to minimize both overlapping of frequent items among clusters and the number of infrequent items for a clustering. INCLUS uses frequent items for local optimization such that similarities for a clustering are maximized with respect to the similarity constraints. In SUMMARY, the frequent items used are globally frequent in the whole data set while in INCLUS they are only locally frequent within a cluster. Unlike INCLUS, TrK-Means does not use similarity constraints and it produces k or more clusters based on the user input of k. Squeezer (7) is an incremental clustering algorithm for categorical data. It uses a different similarity function from INCLUS, which is defined as the number of matched attribute values in a pair of tuples.

BackgroundIn order to maintain the most comprehensive structural annotation databases we must carry out regular updates for each proteome using the latest profile-profile fold recognition methods. The ability to carry out these updates on demand is necessary to keep pace with the regular updates of sequence and structure databases. Providing the highest quality structural models requires the most intensive profile-profile fold recognition methods running with the very latest available sequence databases and fold libraries. However, running these methods on such a regular basis for every sequenced proteome requires large amounts of processing power.In this paper we describe and benchmark the JYDE (Job Yield Distribution Environment) system, which is a meta-scheduler designed to work above cluster schedulers, such as Sun Grid Engine (SGE) or Condor. We demonstrate the ability of JYDE to distribute the load of genomic-scale fold recognition across multiple independent Grid domains. We use the most recent profile-profile version of our mGenTHREADER software in order to annotate the latest version of the Human proteome against the latest sequence and structure databases in as short a time as possible.ResultsWe show that our JYDE system is able to scale to large numbers of intensive fold recognition jobs running across several independent computer clusters. Using our JYDE system we have been able to annotate 99.9% of the protein sequences within the Human proteome in less than 24 hours, by harnessing over 500 CPUs from 3 independent Grid domains.ConclusionThis study clearly demonstrates the feasibility of carrying out on demand high quality structural annotations for the proteomes of major eukaryotic organisms. Specifically, we have shown that it is now possible to provide complete regular updates of profile-profile based fold recognition models for entire eukaryotic proteomes, through the use of Grid middleware such as JYDE.

Using physiologically based pharmacokinetic modelling for in silico–in vitro–in vivo extrapolation to predict chemical exposure

All biological products are required to demonstrate the absence of adventitious viruses (AVs), which may be inadvertently introduced at different steps involved in the manufacturing process. The currently recommended in vitro and in vivo virus detection assays have limitations for broad detection and are lengthy and laborious. Additionally, the use of animals is discouraged by the global 3 R's initiative for replacement, reduction, and refinement. High-throughput or next-generation sequencing (HTS/NGS) technologies can rapidly detect known and novel viruses in biological materials. There are, however, challenges for HTS detection of AVs due to differential abundance of viral sequences in public databases, which led to the creation of a non-redundant, Reference Viral Database (RVDB) containing all viral, viral-like, and viral-related sequences, with a reduced cellular sequence content. In this paper, we describe improvements in RVDB, which include the transition of RVDB production scripts from the original Python 2 to Python 3 codebase, updating the semantic pipeline to remove misannotated non-viral sequences and irrelevant viral sequences, use of taxonomy for the removal of phages, and inclusion of a quality-check step for SARS-CoV-2 genomes to exclude low-quality sequences. Additionally, RVDB website updates include search tools for exploring the database sequences and implementation of an automatic pipeline for providing annotation information to distinguish non-viral and viral sequences in the database. These updates for refining RVDB are expected to enhance HTS bioinformatics by reducing the computational time and increasing the accuracy for virus detection.IMPORTANCEHigh-throughput sequencing (HTS) has emerged as an advanced technology for demonstrating the safety of biological products. HTS can be used as an alternative adventitious virus detection method for replacing the currently recommended in vivo and PCR assays and supplementing or replacing the in vitro cell culture assays. However, HTS bioinformatics analysis for broad virus detection, including both known and novel viruses, depends on using a comprehensive and accurately annotated database. In this study, we have refined our original comprehensive Reference Virus Database (RVDB) for greater accuracy of virus detection with a reduced computational burden. Additionally, the production script for automating the generation of RVDB was updated to facilitate reliable database production and timely availability.