Collusion-Free Privacy Preserving Data Mining

Traditional methods for frequent itemset mining typically assume that data is centralized and static. Such methods impose excessive communication overhead when data is distributed, and they waste computational resources when data is dynamic. We present what we believe to be the first unified approach that overcomes these assumptions. Our approach makes use of parallel and incremental techniques to generate frequent itemsets in the presence of data updates without examining the entire database, and imposes minimal communication overhead when mining distributed databases. Further, our approach is able to generate both local and global frequent itemsets. This ability permits our approach to identify high-contrast frequent itemsets, which allows one to examine how the data is skewed over different sites.

Electronic Health Records (EHRs) are aggregated, combined and analyzed for suitable treatment planning and safe therapeutic procedures of patients. Integrated EHRs facilitate the examination, diagnosis and treatment of diseases. However, the existing EHRs models are centralized. There are several obstacles that limit the proliferation of centralized EHRs, such as data size, privacy and data ownership consideration. In this paper, we propose a novel methodology and algorithm to handle the mining of distributed medical data sources at different sites (hospitals and clinics) using Association Rules. These medical data resources cannot be moved to other network sites. Therefore, the desired global computation must be decomposed into local computations to match the distribution of data across the network. The capability to decompose computations must be general enough to handle different distributions of data and different participating nodes in each instance of the global computation. In the proposed methodology, each distributed data source is represented by an agent. The global association rule computation is then performed by the agent either exchanging some minimal summaries with other agents or travelling to all the sites and performing local tasks that can be done at each local site. The objective is to perform global tasks with a minimum of communication or travel by participating agents across the network, this will preserve the privacy and the security of the local data. The proposed association rule mining methodology will be used for heart disease prediction using real heart disease data. These real data exist at different clinics and cannot be moved to a central site. The proposed model protects the patient data privacy and achieves the same results as if the data are moved and joined at a central site. We also validate the extracted association rules from all the data providers using an independent test datasets.

The increasing availability of numerous heterogeneous but related sources of data arises in many fields, such as bioinformatics, robotics, computer vision, information retrieval, and many others. Multiple sources of data can be viewed as different, related views of the same learning problem, where dependencies between the views could potentially take on complex structures. This gives rise to interesting and challenging machine learning problems where data sources are combined for learning. This framework encompasses several data fusion tasks and related topics, such as transfer learning, multitask learning, multiview learning, and learning under covariate shift. Often the general concept is to augment training data for each problem of interest with data from other partially related learning problems. Several approaches for inferring and exploiting complex relationships between sources have been presented. The advantages of the multiple source learning paradigm can be seen in situations where individual data sources are noisy, incomplete, and learning from more than one source can filter out problem-independent noise. Additionally, another aspect of multiple source learning is that data sources need only contain partially relevant information to the desired learning problem, making it possible to exploit a large number of data sources. This special issue on “Learning from Multiple Sources” was inspired by a successful (eponymous) NIPS workshop which took place in Whistler (Canada) on December 13, 2008. The workshop received twenty-three submissions, from which seven contributed talks and

MultiDS-MDA: Integrating multiple data sources into heterogeneous network for predicting novel metabolite-drug associations

Analysis of remote sensing imagery in the past has consisted of processing a single image from a single sensor. Capabilities now exist to use more than one data source in a single interpretation (such as multi-sensor integration, multi-temporal analysis, interpretations of combined raster-vector overlays). As more data sources become available, data sets become more pluralized, creating two key challenges in the related analysis. The first challenge is to integrate and manage the physical data and metadata as individual data sources and as compound datasets. The second challenge is to select data subsets and related analysis methodologies which are best suited to solve given problems. It is obvious that a dataset consisting of both an optical image and a microwave image will require different analysis methods for each subset, to meet similar goals. In the less extreme case of a dataset containing both spaceborne and airborne optical images, different methods would apply due to their different resolutions and spectral properties. Current and potential operational users could have very high demands for remote sensing data if they were provided with appropriate interpretation tools. These users would not be interested in the process of choosing analysis methods for their data, they would be more concerned with which method was actually used, and its result compared to the specified input criteria. Ultimately they would not be concerned with even choosing the data, but rather would provide a goal of their desired result for driving the decision of which data to select. In this paper, a technique is presented for addressing high-level criteria for analyzing remote sensing data. The technique consists of selecting methods most likely to achieve the result from among several known candidates.

Comparison of epidemiologic data from multiple sources

SufRec, an algorithm for mining association rules: Recursivity and task parallelism

Finding frequent itemsets is a popular data mining problem, aiming to extract hidden patterns from a transactional database. Several bio-inspired approaches to solve this problem have been proposed to overcome the poor performance of exact algorithms, such as Apriori and FPGrowth. Approaches based on genetic algorithms are among the most efficient ones from the point of view of runtime performance, but they are still inefficient in terms of solution’s quality, i.e., the number of frequent itemsets discovered. To deal with this issue, we propose in this paper a new genetic algorithm for finding frequent itemsets called GA-Apriori, in which the crossover and mutation operators are defined by taking into account the Apriori heuristic principle. The results of our evaluation show that GA-Apriori outperforms other approaches to frequent itemset mining based on genetic algorithms, especially when dealing with large instances. The experiments also show that GA-Apriori is competitive with exact approaches in terms of the number of frequent itemsets discovered.

Privacy preserving data mining is a vibrant area in data mining. The sharing of data between the organizations is found to be beneficial for business growth. However, privacy policies and threats prevent the data owners from sharing the data for mini

The heterogeneous, distributed and voluminous nature of many government and corporate data sources impose severe constraints on meeting the diverse requirements of users who analyze the data. Additionally, communication bandwidth limitations, time constraints, and multiple data formats impose further restrictions on users of these distributed data sources. In this paper, we present an Agent-based Complex QUerying and Information Retrieval Engine (ACQUIRE) for large, heterogeneous, and distributed data sources. ACQUIRE acts as a softbot or interface agent by presenting users with a view of a single, unified, homogenous data source, against which users can pose high-level declarative queries. ACQUIRE translates each such user query into a set of sub-queries by employing a combination of planning and traditional database query optimization techniques. ACQUIRE then spawns a set of mobile agents corresponding to these sub-queries, which in turn retrieve the data from various distributed data sources by dynamically optimizing the retrieval strategy as it is carried out. These mobile agents carry with them data-processing code that can be executed at the remote site, thus reducing the size of data returned by the agent. When all mobile agents have returned, ACQUIRE filters and merges the retrieved data and presents the results to the user. While the system is still very much a work in progress, current validation experiments on simulated NASA Distributed Active Archive Centers (DAACs) have demonstrated that complex queries can be effectively decomposed and retrieved by this approach.

This article describes how privacy preserving data mining has become one of the most important and interesting research directions in data mining. With the help of data mining techniques, people can extract hidden information and discover patterns and relationships between the data items. In most of the situations, the extracted knowledge contains sensitive information about individuals and organizations. Moreover, this sensitive information can be misused for various purposes which violate the individual's privacy. Association rules frequently predetermine significant target marketing information about a business. Significant association rules provide knowledge to the data miner as they effectively summarize the data, while uncovering any hidden relations among items that hold in the data. Association rule hiding techniques are used for protecting the knowledge extracted by the sensitive association rules during the process of association rule mining. Association rule hiding refers to the process of modifying the original database in such a way that certain sensitive association rules disappear without seriously affecting the data and the non-sensitive rules. In this article, two new hiding techniques are proposed namely hiding technique based on genetic algorithm (HGA) and dummy items creation (DIC) technique. Hiding technique based on genetic algorithm is used for hiding sensitive association rules and the dummy items creation technique hides the sensitive rules as well as it creates dummy items for the modified sensitive items. Experimental results show the performance of the proposed techniques.

Current robotic systems carry many diverse sensors such as laser scanners, cameras and inertial measurement units just to name a few. Typically such data is fused by engineering a feature that weights the different sensors against each other in perception tasks. However, in a long-term autonomy setting the sensor readings may change drastically over time which makes a manual feature design impractical. A method that can automatically combine features of different data sources would be highly desirable for adaptation to different environments. In this paper, we propose a novel clustering method, coined Layered Affinity Propagation, for automatic clustering of observations that only requires the definition of features on individual data sources. How to combine these features to obtain a good clustering solution is left to the algorithm, removing the need to create and tune a complicated feature encompassing all sources. We evaluate the proposed method on data containing two very common sensor modalities, images and range information. In a first experiment we show the capability of the method to perform scene segmentation on Kinect data. A second experiment shows how this novel method handles the task of clustering segmented colour and depth data obtained from a Velodyne and camera in an urban environment.

Abstract. The classification of urban objects such as buildings, trees and roads from airborne sensors data is an essential step in numerous mapping and modelling applications. The automation of this step is greatly needed as the manual processing is costly and time consuming. The increasing availability of airborne sensors data such as aerial imagery and LIDAR data offers new opportunities to develop more robust approaches for automatic classification. These approaches should integrate these data sources that have different characteristics to exceed the accuracy achieved using any individual data source. The proposed approach presented in this paper fuses the aerial images data with single return LIDAR data to extract buildings and trees for an urban area. Object based analysis is adopted to segment the entire DSM data into objects based on height variation. These objects are preliminarily classified into buildings, trees, and ground. This primary classification is used to compute the height to ground for each object to help improve the accuracy of the second phase of classification. The overlapping perspective aerial images are used to build an ortho-photo to derive a vegetation index value for each object. The second phase of classification is performed based on the height to ground and the vegetation index of each object. The proposed approach has been tested using three areas in the centre of the city of Vaihingen provided by ISPRS test project on urban classification and 3D building reconstruction. These areas have historic buildings having rather complex shapes, few high-rising residential buildings that are surrounded by trees, and a purely residential area with small detached houses. The results of the proposed approach are presented based on a reference solution for evaluation purposes. The classification evaluation exhibits highly successful classification results of buildings class. The proposed approach follows the exact boundary of trees based on LIDAR data which provide above average classification results for the trees when compared to the assumed ideal circular shaped trees in the reference data.

Chromatin immunoprecipitation (ChIP) experiments followed by array hybridization, or ChIP-chip, is a powerful approach for identifying transcription factor binding sites (TFBS) and has been widely used. Recently, massively parallel sequencing coupled with ChIP experiments (ChIP-seq) has been increasingly used as an alternative to ChIP-chip, offering cost-effective genome-wide coverage and resolution up to a single base pair. For many well-studied TFs, both ChIP-seq and ChIP-chip experiments have been applied and their data are publicly available. Previous analyses have revealed substantial technology-specific binding signals despite strong correlation between the two sets of results. Therefore, it is of interest to see whether the two data sources can be combined to enhance the detection of TFBS. In this work, hierarchical hidden Markov model (HHMM) is proposed for combining data from ChIP-seq and ChIP-chip. In HHMM, inference results from individual HMMs in ChIP-seq and ChIP-chip experiments are summarized by a higher level HMM. Simulation studies show the advantage of HHMM when data from both technologies co-exist. Analysis of two well-studied TFs, NRSF and CCCTC-binding factor (CTCF), also suggests that HHMM yields improved TFBS identification in comparison to analyses using individual data sources or a simple merger of the two. Source code for the software ChIPmeta is freely available for download at http://www.umich.edu/~hwchoi/HHMMsoftware.zip, implemented in C and supported on linux.

Stakeholders in Open Source Software (OSS) projects need to determine whether a project is likely to sustain for a sufficient period of time in order to justify their investments into this project. In an OSS project context, there are typically several data sources and OSS processes relevant for determining project health indicators. However, even within one project these data sources often are technically and/or semantically heterogeneous, which makes data collection and analysis tedious and error prone. In this paper, the authors propose and evaluate a framework for OSS data analysis (FOSSDA), which enables the efficient collection, integration, and analysis of data from heterogeneous sources. Major results of the empirical studies are: (a) the framework is useful for integrating data from heterogeneous data sources effectively and (b) project health indicators based on integrated data analyses were found to be more accurate than analyses based on individual non-integrated data sources.